Virtual Texturing is the logical extreme of texture atlases.
A texture atlas is a single giant texture that contains textures for individual meshes inside it:
Texture atlases became popular due to the fact that changing textures causes a full pipeline flush on the GPU. When creating the meshes, the UVs are compressed/shifted so that they represent the correct 'portion' of the whole texture atlas.
As @nathan-reed mentioned in the comments, one of the main drawbacks of texture atlases is losing wrap modes such as repeat, clamp, border, etc. In addition, if the textures don't have enough border around them, you can accidentally sample from an adjacent texture when doing filtering. This can lead to bleeding artifacts.
Texture Atlases do have one major limitation: size. Graphics APIs place a soft limit on how big a texture can be. That said, graphics memory is only so big. So there is also a hard limit on texture size, given by the size of your v-ram. Virtual textures solve this problem, by borrowing concepts from virtual memory.
Virtual textures exploit the fact that in most scenes, you only see a small portion of all the textures. So, only that subset of textures need to be in vram. The rest can be in main RAM, or on disk.
There are a few ways to implement it, but I will explain the implementation described by Sean Barrett in his GDC talk. (which I highly recommend watching)
We have three main elements: the virtual texture, the physical texture, and the lookup table.
The virtual texture represents the theoretical mega atlas we would have if we had enough vram to fit everything. It doesn't actually exist in memory anywhere. The physical texture represents what pixel data we actually have in vram. The lookup table is the mapping between the two. For convenience, we break all three elements into equal sized tiles, or pages.
The lookup table stores the location of the top-left corner of the tile in the physical texture. So, given a UV to the entire virtual texture, how do we get the corresponding UV for the physical texture?
First, we need to find the location of the page within the physical texture. Then we need to calculate the location of the UV within the page. Finally we can add these two offsets together to get the location of the UV within the physical texture
float2 pageLocInPhysicalTex = ...
float2 inPageLocation = ...
float2 physicalTexUV = pageLocationInPhysicalTex + inPageLocation;
Calculating pageLocInPhysicalTex
If we make the lookup table the same size as the number of tiles in the virtual texture, we can just sample the lookup table with nearest neighbor sampling and we will get the location of the top-left corner of the page within the physical texture.
float2 pageLocInPhysicalTex = lookupTable.Sample(virtTexUV, nearestNeighborSampler);
Calculating inPageLocation
inPageLocation is a UV coordinate that is relative to the top-left of the page, rather than to the top-left of the whole texture.
One way to calculate this is by subtracting off the UV of the top left of the page, then scaling to the size of the page. However, this is quite a bit of math. Instead, we can exploit how IEEE floating point is represented. IEEE floating point stores the fractional part of a number by a series of base 2 fractions.
In this example, the number is:
number = 0 + (1/2) + (1/8) + (1/16) = 0.6875
Now lets look at a simplified version of the virtual texture:
The 1/2 bit tells us if we're in the left half of the texture or the right. The 1/4 bit tells us which quarter of the half we're in. In this example, since the texture is split into 16, or 4 to a side, these first two bits tell us what page we're in. The remaining bits tell us the location inside the page.
We can get the remaining bits by shifting the float with exp2() and stripping them out with fract()
float2 inPageLocation = virtTexUV * exp2(sqrt(numTiles));
inPageLocation = fract(inPageLocation);
Where numTiles is a int2 giving the number of tiles per side of the texture. In our example, this would be (4, 4)
So let's calculate the inPageLocation for the green point, (x,y) = (0.6875, 0.375)
inPageLocation = float2(0.6875, 0.375) * exp2(sqrt(int2(4, 4));
= float2(0.6875, 0.375) * int2(2, 2);
= float2(1.375, 0.75);
inPageLocation = fract(float2(1.375, 0.75));
= float2(0.375, 0.75);
One last thing to do before we're done. Currently, inPageLocation is a UV coordinate in the virtual texture 'space'. However, we want a UV coordinate in the physical texture 'space'. To do this we just have to scale inPageLocation by the ratio of virtual texture size to physical texture size
inPageLocation *= physicalTextureSize / virtualTextureSize;
So the finished function is:
float2 CalculatePhysicalTexUV(float2 virtTexUV, Texture2D<float2> lookupTable, uint2 physicalTexSize, uint2 virtualTexSize, uint2 numTiles) {
float2 pageLocInPhysicalTex = lookupTable.Sample(virtTexUV, nearestNeighborSampler);
float2 inPageLocation = virtTexUV * exp2(sqrt(numTiles));
inPageLocation = fract(inPageLocation);
inPageLocation *= physicalTexSize / virtualTexSize;
return pageLocInPhysicalTex + inPageLocation;
}
