Performance impact during texture layer loading

Question

Recently I have been having a rather confusing performance drop with loading in layers of a 2D texture array, but only in one case from what I can tell, leaving me to speculate on why it is happening as I have never seen anything like this before.

Firstly to explain some context for this problem, I implemented a simple version of a virtual texture atlas based on some papers I read by NVIDIA and idTech which in short loads textures (which have been split into fixed size tiles) into various levels of the 2D texture array based on which layers are considered "free" in a CPU-side list of free indices. Sampling in the fragment shader is then preformed by accessing a page table stored as a persistently coherently mapped SSBO (for CPU-side updating) which associates offsets in the UVs with layers of the texture array.

Due to the fact that the texture when created is uninitialized, every layer is considered free, so texture tiles to be loaded will slowly fill up the uninitialized layers (preferring them over previously freed layers). Once all of the uninitialized layers have been used, additional tiles will then begin overwriting previously freed layers, if any exist when needed. Also note that I am using glCompressedTexSubImage3D to load in each layer and that I load in mipmap levels as well manually per layer with this call as well.

This system works fine and was tested rather extensively until I began to notice the problem which this question is about. In short, after all of the uninitialized layers of the texture array were used up, the next 5-15 (I don't know exactly how many, but it's just a few) texture tiles which are loaded suffer massive performance drops. After that, everything returns to normal and the problem never returns (from what I can tell). Also note that this is while some currently drawn textures were also in use at the lower end of the texture array which is right before where the first free tiles begin (due to the first in first out nature of the free list).

Upon examining the problem further, I noticed through using the NVIDIA Nsight graphics profiling tool that the call after glCompressedTexSubImage3D in the cases where performance was impacted blocks on the CPU for very long periods of time, usually 10-40ms per mipmap level (in this case it was glGetError, but any other call following it had the same effect after disabling error checking to ensure that was not the problem).

Since the API was blocking, I assumed that it was some sort of synchronization issue with the GPU perhaps due to the locality of the newly loaded tiles with respect to tiles currently being drawn as it may have considered them "in use". That hypothesis did not add up in my mind due to how long the calls were blocking for as each one was the length of multiple frames at 16ms as well as how this problem never appears to return even after multiple times through this same process, though I'm not ignoring that possibility.

Another odd thing I found was that during the time the next call was blocking, many tiny GPU commands were being executed according to NVIDIA Nsight, which I usually only see under any sort of drawing activity.

At this point I'm pretty much stumped and don't know what could possibly be causing this, so any help would be greatly appreciated. Additional details are provided below (and more can be requested if needed):

Texture Array (there are 3 of them):

• Format: COMPRESSED_RGB_BPTC_UNSIGNED_FLOAT, COMPRESSED_RED_RGTC1, COMPRESSED_RG_RGTC2
• Extents: 256x256x512, 9 mipmap levels

GPU:

GeForce GTX 960M

NVIDIA Nsight Results:

Wide view of 3 long frames where the slow texture loading happens (the line down the screen is centered on the first texture loading call):

Zoomed in view near the very start of the first slow call to glCompressedTexSubImage3D (which cannot be seen as it only blocks for a few microseconds), note the many tiny green lines below the blocking glGetError calls representing GPU commands (as again the line down the screen is centered on the first texture loading call):

Answer 1

From the way the problem only manifests when you start overwriting previously used texture tiles, I suspect you're running afoul of some driver heuristic that's attempting to manage the lifetime of the texture data on the GPU.

In general, when you update buffer or texture data from the CPU, the driver has to make sure it doesn't overwrite anything that might still be in use by the GPU (considering that the CPU and GPU run asynchronously to each other). There are various ways the driver can do this, such as internally double-buffering the texture, inserting sync points (for the CPU to wait on the GPU), or initially copying data to a temporary staging buffer and then later re-copying it to its final destination. All this stuff is under-the-hood and the driver makes its own decisions about what strategy to employ in any given situation, using some heuristics based on texture size, usage patterns, etc.

I can only guess at the details, but I suspect what's happening is that the driver is getting into a state where it thinks those previously-used texture tiles are still in use on the GPU somehow. Then when you start to overwrite them, it decides that it needs to make a second instance of the whole texture, and laboriously copies everything over before finally doing the updates you wanted—or something like that. That could be the many tiny draws you see in Nsight.

It's still a mystery why it's taking tens of milliseconds to do that, though. Maybe the textures are large enough that a second copy of them causes you to run out of VRAM? In that case, you'd get a lot of thrashing as things are spilled to main RAM. That's just a guess, though. As for why the bad behavior only happens for a few tiles and then goes back to normal, it could be that the driver heuristics are deciding to switch to a different strategy at that point.

Anyway, as for how to fix it, you might try using glInvalidateTexSubImage whenever you free a texture tile. This lets the driver know that you don't care what was in that texture tile before, which hopefully will convince it to just update the texture in-place rather than doing anything crazy with it.

Another thing to try is to make sure you don't try to re-use a texture tile in the very next frame after you free it, but give it a frame or two of "downtime" before re-using it. But it sounds like that's probably the case already, given that you prefer unused tiles before you start re-using them.

If that doesn't help, I'd recommend sending an email to [email protected]. The OpenGL driver team there can likely take your Nsight captures and help diagnose the issue, and fix it if it's due to a driver bug.