Performance impact/implications of constant buffers / uniform buffers

Question

Constant buffers stay constant throughout entire dispatches / draw calls, and thus their data can presumably be shared as efficiently as possible between different threads using those buffers.

But I guess this probably doesn't mean they are "free" in terms of memory bandwidth? Are they loaded from memory once per thread group / wave (whatever hardware terminology is fitting), or are they somehow cached more efficiently?

What other performance metrics / considerations do they affect? Is there something like register pressure where CB size can impact how well your shader can be parallelized if the CB is too large (even below the 65k limit)?

Is the size of your constant buffer something you should keep an eye out for when optimizing shaders, or is it shared efficiently enough that it's generally not a point of worry? For example, in many cases one can probably output constant buffer constants as shader code constants (leading to more variants) instead. Or maybe some computation in the shader is the result of two values in the CB - you could add a third value to the CB, or compute it on the fly in the shader for every single pixel / thread but in return you maybe don't have to fetch as much memory... how to reason about this?

How is this impacted by different hardware vendors? For example, in certain cases I saw it recommended to use struct buffers instead of constant buffers, even if all data fetched is the same in an entire thread group, but only on certain hardware (was it AMD GCN?). I think light cluster culling is such a case, where one might want to fetch light list data with a struct buffer.

My overall question is how to reason about the performance implications of constant buffers on various hardware .

Answer 1

There's no real way to know as there is significant hardware variance. AMD hardware for example doesn't have UBOs as a distinct concept; a UBO is implemented as a read-only SSBO, relying on caching to aid performance. Other hardware makers have specialized hardware for this stuff, where it reads from the buffer to populate the execution units, and never again.

What matters is this: if you have data that the shader needs to access which is backed by GPU storage, and it fits within the hardware limitations of a UBO... then use them. They will be as fast as the hardware allows for that purpose. Something else might be equally as fast, but nothing will be faster (because the hardware would have just used that).

Answer 2

On some architectures at least, uniform/constant buffer loads will serialize if threads in a warp fetch from different addresses. Worst case, the load might be 32 or 64 times slower.

This can be a problem if, for example, you're indexing matrices for skinning which are stored in a uniform buffer, from a vertex shader. It's likely that threads/vertices will have a lot of disagreement about which matrices to load, and these loads will serialize.

I'm not sure about the state of things across vendors in 2023, though.

If your access pattern is such that different threads might try to load from different addresses at the same time, I recommend using a regular buffer to be on the safe side.