Why can't we have hardware-specific graphics APIs?

Question

Right now, if you want to use a GPU, you have to do it through graphics APIs like DirectX or Vulkan. These APIs are meant to be hardware-independent, but more often than not, they are tied to a specific platform (like DirectX for Windows, or Metal for iOS and macOS). The only modern graphics API which was designed to be cross-platform is Vulkan and even it is not supported by every platform. Both iOS and macOS don't support it (you have to use MoltenVK, but that's an API wrapper, not a driver implementation) and I'm assuming that neither PlayStation nor Xbox have support for it.

It seems like we have solved the problem of hardware independence, but we've imposed the (artificial) problem of platform dependence. The hardware is doing the same thing no matter if it's in a Windows system or an Apple system. Wouldn't it be better if each GPU vendor provided their own graphics API?

So, my question is: Why don't we have hardware-specific graphics APIs instead of platform-specific ones?

Here's my reasoning for it...

More work for engine programmers

Let's say you are a game company and you want to make a game using your own game engine. And let's say you want to ship your game on PC and consoles. This means you want to support Windows, Xbox, and PlayStation. macOS and Linux aren't really a concern because not that many people game on them. The graphics APIs for Windows are DirectX and Vulkan (not counting OpenGL), for Xbox its DirectX (DX12 Ultimate technically is the same for Windows and Xbox), and for PlayStation, it's GNM (Sony's own proprietary graphics API). This means that if you wanted to write a rendering engine with as little effort as possible, you would, at the very least, have to support DirectX 12 and GNM.

This doesn't seem that bad until you consider the fact that this is only the bare minimum. Many game engines also support Vulkan on Windows, not to mention that if you want to port your game to Mac or the Nintendo Switch you'll have to support even more APIs.

Additionally, if you are writing a high-end rendering engine and you want to get the most out of your GPU, you'll need to do architecture-specific optimizations to shaders, which means having multiple variations of each shader, FOR EACH graphics API. You could write a shader translator so that you only need to write your shaders once, but then you get into other problems.

More work for driver implementers

On the other side, we have drivers. You have to support OpenGL, DirectX 11, DirectX 12, and Vulkan (all of these are just for Windows). If you provide hardware to Apple you also have to support Metal (although, Apple seem to be making their own hardware now so maybe they're not a concern anymore) and if you provide hardware for consoles, you also have to support their proprietary API. This becomes even worse when you consider the fact that drivers for mobile GPUs (e.g. for laptops and phones) and discrete GPUs are different.

I don't think hardware vendors write a completely different driver for each API+platform+GPU combination, that would be insane! I'm guessing that drivers have their own internal API, or something similar, and each graphics API implementation is just wrapping around that internal API. However, this still leads to a massive amount of software, which ultimately means more bugs and more money spent on more developers.

The (possibly) ideal solution

I think it makes much more sense for hardware vendors to provide their own APIs. These APIs are tailor-made for their own GPUs. If your new GPUs behave differently than the older ones, you can just change the API to better fit the new GPUs instead of giving programmers hacky workarounds for existing APIs. I also think that these APIs should let programmers write in GPU assembly. Of course, I don't think programmers will write in GPU assembly most of the time, however, it gives them more freedom to do architecture-specific optimizations.

One big problem with current graphics APIs is that they are hardware-independent. Why is that a problem? Because it means that everybody has to conform to the same interface. This isn't always bad. A well-designed API can be general enough that it suits everyone's needs, while also not being too restrictive. However, doing this means collaborating with multiple hardware manufacturers, which can take a lot of time. For example, look at how long it took for raytracing to land in Vulkan. The first raytracing cards came out near the end of 2018 and while raytracing was supported in Vulkan soon after, it was only through an NVIDIA-exclusive extension. The official, cross-vendor extension didn't come until 2020.

Granted, I don't think current graphics APIs should be removed entirely. I think that the responsibility of implementing them should fall in the hands of the people who design/provide them, like Microsoft, Apple, or the Khronos Group, and not in the hands of hardware vendors. Instead of having custom implementations for each API in the driver, APIs like DirectX will be implemented at the application level using the publicly-available hardware-specific APIs provided by GPU vendors. This way, programmers who don't care a lot about performance or cutting-edge features, will still be able to use the current APIs.

The end (Finally!)

This seems like something someone working on graphics APIs would have thought of and since it seems like this approach is much better than the current one, I can't think of a reason why it hasn't happened yet. I expect that there must be a pretty good reason why this isn't the way we do things other than just "we've always done things this way so why change it now", I just can't think of it!

Edit: After reading the responses, I think I may have explained some things a bit poorly. In particular, I'm not suggesting that we remove all abstractions and have APIs that are extremely low level. They should still live in user-land and take advantage of things like sharing resources, scheduling, and TDR. What I'm suggesting is APIs similar to Vulkan or DirectX in terms of abstraction, but that are tied to specific hardware, not a specific platform. Anyway, thank you for the responses!

Answer 1

Wouldn't it be better if each GPU vendor provided their own graphics API?

... no.

First, the GPU doesn't belong to you. It belongs to the platform, the OS. In order for different programs to effectively share hardware resources, those programs have to talk to some intermediary that exists to ensure that no programs are completely starved of resources. Furthermore, there needs to be code to ensure that pathological use of the hardware resources resources can be dealt with.

We know what platforms that don't have these kinds of things look like. One program that does something wrong can force a reboot of the system. The only way to prevent this is for someone to stand between programs and the hardware to make sure everybody plays by the rules.

So regardless, there is going to be some platform-specific aspect of any of this. The platform is going to have to approve at least that there is a direct connection between a program and the driver.

Second, developers benefit from having cross-hardware APIs, even platform-specific ones.

Here's the thing. While Metal, D3D12, and Vulkan are different APIs, they're not all that different, structurally speaking. They all have command buffers, some object that represents the shader pipeline, similar ways of attaching resources, similar mechanisms for allocating memory and using it with objects, etc.

By contrast, hardware-specific APIs will expose their particular hardware in substantially different ways. They will also expose specific details of their hardware that other APIs don't, which will make building hardware-neutral graphics programs incredibly difficult.

To take an example, consider image layouts. This is a concept that Vulkan introduced almost entirely as an affordance to AMD's graphics hardware. Their hardware has a lot of layouts that restrict the use of images.

Their OpenGL/D3D11 drivers would do the behind-the-scenes work to transition layouts as you changed how you used images. But the nature of a command-buffer API makes doing that highly impractical. Therefore, any command-buffer API built to handle AMD hardware must expose the basic concept of layouts to the user.

But NVIDIA hardware has no such concept. Well, it does have similar concepts, but the GPU itself handles transitions automatically in most cases.

So an AMD API would require you to use image layouts, and the NVIDIA version wouldn't. Any platform-neutral engine would have to figure out how to square this circle.

Consider Mantle. It had no concept of UBOs (just read-only SSBOs) or a vertex fetching stage. This is because AMD's hardware of the day had no concept of those things either. But other hardware of the day did have those things. So that's another thing you would have to deal with.

And that doesn't even begin to get into some of the more fundamental differences between a standard rendering GPU and a tile-based GPU.

Eventually, developers would just build some kind of cross-hardware abstraction API. You know, like Vulkan.

Additionally, if you are writing a high-end rendering engine and you want to get the most out of your GPU, you'll need to do architecture-specific optimizations to shaders, which means having multiple variations of each shader, FOR EACH graphics API.

But you're going to have to do that work regardless of the API you're using. Optimizations based on architecture are based on the GPU architecture, not the graphics API.

Windows boxes don't use the same GPUs as consoles. They may be similar to certain specific PC GPUs, but they're not the same. You're going to have to use specific optimizations for those specific GPUs anyway.

Not to mention, there are so many PC GPUs anyway that if you're doing significant hardware-specific optimizations to your shader logic, you won't notice 2 more versions. And since you're doing development on consoles too, you're already going to have to build some mechanism to compile your shaders for different targets.

This would be true if you were coding to the GPU; you'd just have way more targets, since each GPU core has changes to its internal machine code relative to previous generations of GPUs.

For example, look at how long it took for raytracing to land in Vulkan. The first raytracing cards came out near the end of 2018 and while raytracing was supported in Vulkan soon after, it was only through an NVIDIA-exclusive extension. The official, cross-vendor extension didn't come until 2020.

What's your point?

If you were in 2019 and wanted to implement raytracing in a program, you were going to be limited to NVIDIA hardware anyway because... no other GPUs supported it. It took quite a long time for other GPUs to provide the feature. So relying on an NVIDIA extension is just normal.

Furthermore, when that feature did land in Vulkan, it did so in a way that non-NVIDIA GPUs could also implement. And it did so at a time when non-NVIDIA GPUs actually existed that could use it.

Answer 2

In the early years of the PC, it was common for every game (and other piece of software that needed them) to ship with its own set of sound and graphics drivers. This had a number of drawbacks:

• Every game developer had to write and maintain a large set of drivers, even though most games needed exactly the same facilities.
• New hardware would have to list in its user manual which drivers from old software it was most likely to work with.
• Smaller hardware vendors were forced to reverse-engineer their rivals interfaces from documentation, to make these existing drivers work.
• The operating system could not take any control of these drivers to guarantee the security and stability of the system. MS-DOS essentially gave full control of the hardware to the game, and hoped for the best.

This led to the development of what Windows 95 called "virtual device drivers" - rather than a driver interfacing each game to each piece of hardware, a single driver interfaces the OS to the hardware, and the game ships with a single adapter for the OS. In theory, this solves all the above problems:

• The game developer only needs to write and maintain a single driver
• New hardware can ship with a driver that installs into the OS, and immediately be available for all games
• The OS interface is designed and documented for new drivers to be written implementing it
• The OS can limit the access of games to keep the system secure and stable, and allow multiple pieces of software to share the hardware effectively

Abstraction layers like DirectX, OpenGL, etc, work on the same principle: they act as an intermediary between the hardware-specific driver and the game-specific code.

Note that is doesn't really matter whether Nvidia writes a GeForce driver for DirectX, or Microsoft writes a DirectX driver for GeForce; games written for DirectX can use it either way.

As you've pointed out, it's not quite as simple as that:

• Games may wish to target more than one OS (or abstraction layer), so still need multiple drivers.
• Games may wish to use cutting edge or unusual features of a particular piece of hardware, which the abstraction layer doesn't support.

These two problems point in exactly opposite directions: the first suggests that there should be "one abstraction to rule them all"; the second suggests that abstractions should be fragmented into a huge range of device-specific functionality.

Highly vendor-specific APIs might appeal to console developers with exclusive agreements to develop for a particular hardware and software platform; highly portable APIs will be more useful to indie developers who want to make their game available on multiple platforms as cheaply as possible.

The result is that people will keep coming up with new abstractions, and extensions to those abstractions, to suit different situations. But ultimately, somebody has to write the code that makes the different combinations of game+OS+hardware possible.

Answer 3

Featureful Versus Widely Available
You're talking about the economics proposition of high functionality for limited subsets versus having a program (game, image processing, video processing) be widely available, independent of hardware.

This can work but only if the features are compelling. This was the case very early on when there were a large number of graphics card providers and even a modest number of computer platform providers. Amiga with Video Toaster was one. So was Silicon Graphics with their advanced workstations.

The problem is that, as technology advances, the common feature set accumulates enough weight to typically outweigh any comparatively small advantages offered by a specific competitor. Their only option is to provide a common driver with extra features that expose their specific advances to be taken advantage of. That allows program developers who want to the option to take advantage of those. The previous answer talks about ray tracing in NVIDIA hardware as a specific example.

As a developer, I'm rarely going to choose a novel software package that requires significant work beyond a standard approach for several reasons. They all boil down to economics. I will have a hard time justifying paying the premium for the license and the extra work to my management. On top of that, the single vendor is likely to have serious bugs in their implementation that we have to cope with over the long haul. The benefits have to be very, very compelling to take on the risk and extra cost.

The economy of scale pretty much demands that maturing technologies evolve towards a common API so that developers can easily switch between products. It's so well pronounced a phenomenon that it typically starts happening in less than ten years. Expect quantum computing to be the same way.

Answer 4

I think the best example of a hardware-specific graphics API was Glide, by 3dfx. It was fairly common in the late 1990's. If you owned one of their Voodoo Graphics cards, it was consistently the most performant way to play a game.

There were a couple of issues with this:

• It caused vendor lock-in (the best performance required specific hardware), which was good for 3dfx but not so much for the consumer.
• It required more development effort from the game developers, as the rendering pipeline would have to target yet another API (and DirectX, OpenGL, and Software renderers were common options at the time, so this would add yet another code path and set of testing)

Glide faded away when 3dfx did, as a result of questionable business choices by 3dfx coupled with Nvidia releasing superior hardware in their TNT2 and GeForce GPUs.

Answer 5

A good deal of software is only available for a single platform (e.g., Android, iOS, Windows or Mac). The current situation means that authors of single-platform software only need to program for a single graphics API.

If you're producing software for multiple platforms, you're already having to deal with differences between them. Adding multiple graphics APIs makes that situation worse, but not radically so.

Answer 6

I think a summary of other answers is: you would end up with current situation. On top of vendor-specific API you would implement some abstraction layer, which in the long run, with typical forces, would end up similar to current "general" APIs.

Anyway, the world is complicated.

I think current (rendering/compute) GPU architecture is close to Vulkan/Direct3D12 APIs, so perhaps internal APIs are not that different (I'm guessing). Also, much of what is hardware-specific is in extensions. And there are lots of them.

But there are vendor-specific APIs: video encoding APIs. Current GPUs have video encoding and decoding chips. Each vendor has their own encoding API. For example, NVIDIA's NVENC, AMD's AMF, Intel's Quick Sync Video, and probably mobile GPUs too. And there are platform-specific video encoding APIs too! Such as VAAPI and VDPAU on linux, and recently Direct3D 12 Video.

There is some information on low level access to GPU hardware. See for example this question. AMD has published details on their GPUs; more info. NVIDIA has their (compute) CUDA API.