For shader math, why should linear RGB keep the gamut of sRGB?

Question

sRGB is often contrasted with "linear RGB."

• Images are stored on disk and passed to displays in sRGB, which is approximately perceptually uniform in intensity.
• Shader math is done in linear RGB, which is physically uniform in intensity.
• Gamma correction can be applied to convert between the two.

Now, sRGB has a standard that colorimetrically specifies its gamut, saying exactly where pure red, green, blue and white lie. But there is no corresponding standard for just "linear RGB." Any triangle on a chromaticity diagram could be said to be linear, and, indeed, there are several well known gamuts to choose from:

In practice, when we say "linear RGB," we mean "sRGB without the gamma correction." (This is implicitly what we're doing when we apply sRGB gamma correction as a final post-processing step but ignore color spaces for the rest of the rendering pipeline.)

But why is that RGB gamut the correct one to use for interpolation and lighting calculations? It seems arbitrary. If anything, wouldn't we want to use the largest possible gamut for internal calculations, and then clip or scale colors to the output device's gamut at the very end?

Is it that RGB lighting is going to be approximate no matter what, so it doesn't matter which gamut we pick, and we may as well pick the one closest to what the display natively supports? Is it just negligence? Or do calculations in these different gamuts actually yield the exact same results, somehow?

Answer 1

Talking about Linear RGB must be avoided because it does not tell you anything about the RGB colourspace intrinsics, i.e., Primaries, Whitepoint and Colour Component Transfer Functions. A few years ago, assuming it was sRGB was middling but nowadays with DCI-P3 and BT.2020 being very common, it must be ruled out.

The ideal gamut for rendering is the one that will minimize errors in respect to a real-world reference or more conveniently a ground truth spectral render. The first takeaway from this sentence is that the various RGB colourspaces are not equivalent and will not produce similar results.

One might think that performing two renders with the same base colours but one where they are encoded with sRGB/BT.709 and the other one where they are encoded with DCI-P3 and then converting the two resulting images to for example ACES2065-1 will yield the same images but it is not the case. Some mathematical operations because of the nature of linear algebra and matrices are dependent on the given RGB colourspace primaries, i.e., on the colourspaces basis. The same operations performed in different RGB colourspace will yield different tristimulus values once converted back to CIE XYZ color space. For example multiplication, division and power operations are RGB colourspace primaries dependent while addition and subtraction are not.

This image illustrates the effect of multiplying various colours by themselves into different RGB colourspaces: the resulting colours are different. The various samples are generated as follows: 3 random sRGB colourspace values are picked and converted to the three studied RGB colourspaces, they are exponentiated, converted back to sRGB colourspace, plotted in the CIE 1931 Chromaticity Diagram on the left and displayed as swatches on the right.

Tests and research conducted by Ward and Eydelberg-Vileshin (2002), Langlands and Mansencal (2014) and Mansencal (2014) showed that gamuts with primaries closest to the spectral locus, i.e. spectrally sharp primaries, tend to minimize the errors compared to spectral ground truth renders.

Here is an image I recently rendered with Mitsuba to re-validate our findings with Anders:

Those are renders of the same scene using BT.709 primaries (first row), 47 spectral bins (second row), BT.2020 primaries (third row), spectral minus BT.709 primaries renders residuals (fourth row), spectral minus BT.2020 primaries renders residuals (fifth row). The last row showcases composite images assembled with three vertical stripes of respectively the BT.709 primaries, spectral and, BT.2020 primaries renders. Direct illumination tends to match between the renders. Areas that show the effect of multiple light bounces, i.e., the ceiling, in the BT.709 and BT.2020 primaries renders tend to exhibit increased saturation, especially in the BT.709 primaries render or slight loss of energy, especially in the BT.2020 render. Excluding outliers, e.g., the visible light source, the RMSE with the spectral render are 0.0083 and 0.0116 for respectively the BT.2020 primaries and BT.709 primaries renders.

Now it does not mean they will always perform better, and one might be able to produce examples that will exhibit a bias toward BT.709/sRGB. The main takeaway is that RGB renders cannot match spectral renders and sharp wide gamuts tend to perform better. As for choosing a rendering colourspace, I would pick one with a wide gamut that encompasses Pointer's Gamut and DCI-P3, BT.2020 or ACEScg are excellent candidates for that.

Answer 2

In practice, when we say "linear RGB," we mean "sRGB without the gamma correction."

It would be more correct to say that there is the "sRGB colorspace" and the "linearized sRGB colorspace", with the sRGB specification definition the conversion from one to another.

Yes, there are infinitely many "linear RGB" color spaces. But the thing that all of these "linear RGB" colorspaces have in common is that they are linear. This means that, if you double the value of any component, you double the intensity of the light that the component represents. That's essentially what it means to be "linear": there is a linear mapping between color values and the resulting intensity of that light color.

This is important because lighting equations don't work if the color values do not linearly map to light intensities. But the equations don't care which linear colorspace you use; you just have to pick one.

So the linearized sRGB colorspace is not more correct than a linearized Adobe RGB colorspace or a linearized SWOP CMYK colorspace. What matters is precisely two things:

1. The colorspace represents a linear mapping of values to light intensities.
2. The colorspace chosen is consistently used in the lighting equation. That is, all colors used in the lighting equation come from the same (linear) colorspace.

Is it that RGB lighting is going to be approximate no matter what, so it doesn't matter which gamut we pick, and we may as well pick the one closest to what the display natively supports?

That, and the fact that sRGB conversion is built into hardware these days, while other colorspace conversion frequently is not. So if you want to use linearized Adobe RGB colorspace, you have to do a lot of work in your shaders to linearize the texel values and perform bilinear/trilinear interpolation on them correctly (which needs to be done after linearization) before you can even apply them to the lighting equation. And then you have to do the conversion from linearized Adobe RGB to linearized sRGB, so that you can write to an sRGB framebuffer image for display.

Or you can just use linearized sRGB everywhere and have performance. The latter tends to win out.

Answer 3

There are two sides to why sRGB particularly. For input non-HDR images, it is claimed that those should be assumed to be compressed to sRGB (whether this claim is accurate is a different story). Hence, before you can do any linear math operation on them, you need to decompress them from sRGB. It is also possible that an image was captured and compresssd to a different representation that is not sRGB, in which case you need to decompress that specific representation. In any case, the encoding implies a certain gamut that the input image will never escape (because sRGB-stored images are typically truncated to 8-bits per channel), but your shader math doesn't have to stay in that gamut after the input image is decompressed. But in the end you have to consider the display.

If you have an image and it is time to display it, you encode it in a representation that the display device requires. CRTs chose sRGB, then LCDs emulated that, so sRGB compression for monitor display has been the common choice for the last decades, and that restricted the output to stick within sRGB gamut, or else clipping will occur. Wider-range displays don't have to stick to that exact gamut.

(I think the basis for the claim that human-made images are sRGB-encoded is because those images were assumed to have been authored on sRGB displays)

So now you can probably see better why sRGB in particular was supported in hardware for shader math input and image display. It is the common case. Plus, it has good merits for reducing perceptual color banding artifacts, so it is a good way to compress colors in 8 bits and keep them looking plausible to humans.

Answer 4

If you allow values outside the 0..1 range, then even with sRGB's fairly limited primaries, you can still address the entire human visual gamut. So for storing floating point light color values it shouldn't matter too much which primaries you use. However, doing any kind of multiplicative math gets a bit funky though, as the arbitrary coordinates of the primaries act as the scaling 'pivot'. sRGB primaries are generally used because traditionally your input data is sRGB encoded and out output display is sRGB or rec709... With rec2020, half of that has changed, but for now, most of your input data is still probably sRGB encoded, so using the same primaries as your storage is just the easiest option.