What is in a GPU driver?

Question

There are 3 kinds of code that are contained in a GPU-utilizing program:

1. CPU code.
2. CPU-to-GPU or GPU-to-CPU bridge code. This is what DirectX actually is.
3. GPU code. In NVidia speak, this is called a "Cuda kernel", and this is where DirectX is actually implemented.

Every program starts in 1 and uses 2 to call pre-compiled 3 on the GPU.

Then, exactly what kinds of code, using the above classification, is contained in GPU "drivers"? Does GPU need a sort of OS-like kernel (not the Cuda kernel mentioned above) just like the CPU? Is this contained in the drivers (often GB-sized) as well?

Answer 1

Your taxonomy is at best imprecise. A more accurate taxonomy would be:

1. Userland code. This is code that's executed by the CPU in user mode.
2. Driver code. This is code that's executed on the CPU which runs with special privileges, allowing direct access to various OS internals and the like.
3. Actual compiled GPU-native code. This is more-or-less arbitrary code that is transferred to the GPU and gets executed by some GPU-invoked process. These could be compute kernels, shaders invoked by a rendering command, etc.

This differs from your taxonomy in many ways. "DirectX" is spread among all of this. Not every DirectX functions is driver code. For D3D12, building command buffers doesn't need driver privileges. It's just creating tokens specific to the GPU which will eventually be given to the GPU. Only the actual act of giving those commands to the GPU needs driver privileges.

Furthermore, DirectX implementations can (and will) insert stuff into even your GPU shader code.

"GPU drivers" therefore live in all three places. The code contained in them exists to implement whatever API abstraction that your userland code is using to communicate to the GPU. The driver uses whichever tools are needed to implement that abstraction.

As for what the GPU is doing, it is generally pretty simple. Most GPUs have one or more queues, to which the driver will (at the request of userland code) submit a sequence of commands. This sequence of commands could be considered a "program", but it's usually simpler than that. They're just commands that the queue will execute using various GPU resources, in order (somewhat), first to last.

There may be a series of state setting commands followed by a rendering command. That rendering command will generate a number of vertex shader invocations, which cascades through the entire rendering pipeline. This operation will involve using a number of resources, including compute units.

A command queue is not a program; it's just a series of operations. There's (probably) no conditional logic, no variables, etc. It's all hard-coded operations. The internals of those operations can be flexible, but which commands are available isn't.

The driver's job (among many other things) is to manage the various queues at the request of user code. It also allocates memory to processes (usually upon request), pages things in and out of virtual GPU memory, etc.

But this is all stuff happening largely on the CPU. The GPU is still a device largely controlled by direct CPU action.