Binding vs bindless

Question

I was trying to understand what exactly is 'binding'. I found an answer on quora on this. From this, binding basically sounds like sending data from to GPU with help from CPU. I got to know more about 'bindless' things as there is something called bindless textures in OpenGL. But how does GPU access the data in this case?

NVIDIA article on bindless graphics talks about how the driver does not need to

dereference a vertex buffer or constant buffer on the CPU in order for the GPU to use it.

in the case of bindless graphics.

So is this dereferencing thing the major difference? How does this help in hugely speeding up the application?

In a nutshell, what is the difference between graphics using binding and 'bindless' graphics?

Answer 1

When a shader accesses a resource (buffer or texture), it needs some information about that resource to be able to do so correctly. On a modern GPU, that information will generally just be some kind of number. It could be something as simple as a GPU virtual memory address (with the particular details of what kind of resource is being used defined by the shading language), or it may be an index into a table of resources.

But all modern GPUs need is a number.

From the CPU side, a resource is defined by some API-defined object. To bind a resource (API-agnosticly) means that you tell the API that the given API object's resource is to used by the shader through some resource binding location.

This process therefore requires the API to take that object and convert it into the number that the shader will receive. The object itself cannot just be that integer because the API needs ancillary information for managing the resource. But when you're rendering with the resource, the integer is all you really need.

So let's say you're rendering a complex scene. You may be rendering 100,000 different objects. That could mean 100,000 different binding operations per-frame. Each bind operation has to dereference an object pointer and fetch a single 64-bit integer from the object behind that pointer. And often, the nature of the API object requires multiple dereferences to get this integer.

This is not a cache-friendly operation, as you're fetching a piece of data from storage that has a bunch of unrelated data near it. So not only is the read likely to be a cache-miss, you're doing a whole memory fetch for just 64-bits of data, that will not be used for anything else before that cache line gets evicted. This represents poor cache coherency.

Across 100,000 objects at 60fps (or greater), that can add up.

A bindless API allows the user to store the integer themselves and marshal it to the shader in whatever way they see fit. Any resource integers for an object's rendering call can be stored near other data that is relevant to the rendering process, thus improving cache coherency.

It should also be noted that a bindless API means that any shader could access any information about any object, so long as it has been given access to the integers that represent those resources. This is rather important when doing something like ray-tracing.

So it's not just about performance.