You don't actually need to avoid branches, you just need to make sure that most of the ones you have are very predictable, then their impact is not that big... Same with memory accesses and most other things... The more predictable, the better...
This is true for serial CPU code, but for SIMD code, or GPU code, avoiding branches at all costs is vital to getting good performance. Branches (can) cause divergence, which leads to wasted cycles for pseudo-threads within a SIMD group.
For branch prediction on CPU, there are no relevant rules of thumb and it's not worth messing with that sort of low level optimization except after careful profiling. Otherwise you're guessing, and the guess may have no effect, have a very small effect, or (at absolute most) it might slow down a tight loop by a factor of ~10.
For branches for GPU code, not understanding how to use them correctly can result in code that's obviously wrong and not useful. If you don't understand this issue and write code incorrectly for the hardware, you may be a million times slower than you expected or the code might not compile at all.
there's no special tax that makes a misspeculation OOMs worse than with normal scalar code on a CPU.
Of course there is: The basic fact that having a data dependent branch in the first place kills SIMD parallelism because the branch is for a single lane while masking / predication processes all lanes in parallel.
Eg. take function y = x3 when x > -0.5 and y = -0.125 when x <= -0.5 (and for the sake of discussion assume clamping instructions don't exist). If you use branching, you need a branch for each value while a simd compare + mask processes four or eight values at a time.
you definitely do not want to avoid branches at all costs on a GPU. there are plenty of cases where branchless tricks will lose to straightforward branching.
I do concede that "at all costs" is somewhat hyperbolic, but it is very important to consider the behaviour of threads grouped together in a warp. Divergence is extremely costly.
137
u/meamZ 23h ago
You don't actually need to avoid branches, you just need to make sure that most of the ones you have are very predictable, then their impact is not that big... Same with memory accesses and most other things... The more predictable, the better...