Modern GPUs have many parallel shading units. I'd like to know how texture cache is managed in that scenario.


At the top level, a GPU is subdivided into a number of shader cores. A small GPU in a notebook or tablet may have only a few cores while a high-end desktop GPU may have dozens.

In addition to the shader cores there are also texture units. They may be grouped together with one texture unit per shader core, or one texture unit shared among two or three shader cores, depending on the GPU.

The whole chip shares a single L2 cache, but the different units will have individual L1 caches. Texture units have texture caches, and shader units have caches for instructions and constants/uniforms, and maybe a separate cache for buffer data depending on whether buffer loads are a separate path from texture loads or not (varies by GPU architecture).

Texture units operate independently and asynchronously from shader cores. When a shader performs a texture read, it sends a request to the texture unit across a little bus between them; the shader can then continue executing if possible, or it may get suspended and allow other shader threads to run while it waits for the texture read to finish.

The texture unit batches up a bunch of requests and performs the addressing math on them—selecting mip levels and anisotropy, converting UVs to texel coordinates, applying clamp/wrap modes, etc. Once it knows which texels it needs, it reads them through the cache hierarchy, the same way that memory reads work on a CPU (look in L1 first, if not there then L2, then DRAM). If many pending texture requests all want the same or nearby texels (as they often do), then you get a lot of efficiency here, as you can satisfy many pending requests with only a few memory transactions. All these operations are pipelined, so while the texture unit is waiting for memory on one batch it can be doing the addressing math for another batch of requests, and so on.

Once the data comes back, the texture unit will decode compressed formats, do sRGB conversion and filtering as necessary, then return the results back to the shader core.

| improve this answer | |
  • 1
    $\begingroup$ Thanks! That was really helpful. Regading the constants/uniform cache. Are they any tips besides precision (mediump, lowp) I could use to improve uniforms cache hit ratio? Does the order in which I declare uniforms make any difference (as for packing more tightly)? $\endgroup$ – Felipe Lira Aug 28 '15 at 18:32
  • 2
    $\begingroup$ @PhilLira Packing can make a difference, yeah. The compiler will insert padding to prevent vectors from being split across 16-byte boundaries, so try to avoid that. I don't think mediump/lowp actually does anything on uniforms, at least on desktop GPUs (maybe it does on mobile). I wouldn't worry too much about uniform cache hit ratio though. That's extremely rarely, if ever, a bottleneck. $\endgroup$ – Nathan Reed Aug 28 '15 at 20:23
  • $\begingroup$ @NathanReed Good answer. But "converting UVs to texel coordinates" What? A UV IS a texture coordinate, which is used to get the texel texture[u][v]. $\endgroup$ – Nikos yesterday
  • $\begingroup$ @Nikos Not quite. UVs are in the [0,1] range and have to be converted to find the set of actual texels needed, given the texture size, wrap mode, and filtering mode. $\endgroup$ – Nathan Reed yesterday
  • $\begingroup$ @NathanReed oh my. Thanks man! $\endgroup$ – Nikos yesterday

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.