16bit half-float linear HDR images as (diffuse/albedo) textures?

Question

So I been thinking about this for a while and tried to google for an answer but without any success.

If all your textures are 8bit LDR images, like JPEGs, couldn't that potentially cause conflicts with exposure control/tone mapping when rendering. That is if you adjust the rendering exposure of your image that should expose detail in the textures that aren't really there, since they been clamped out by the low dynamic range. So wouldn't it make sense to also have the textures as HDR images, saved as .exr, in linear colour space with 16bit half-float to get a good colour representation (32bit "full" float might be overkill?). To have more detailed and correct colour values might also, I figure, have an effect on GI and how colour bleed is calculated?

Or is it simply not necessary since the end result of the rendering we want is probably going to be similar to the exposure level of the texture when it was photographed any way? And since camera mostly shoot in 12-14bit you would have to take multiple exposures of the texture and do all that extra work to piece them all together to one HDRI.

Edit: To clarify, I'm mostly interested in this from a photo realistic rendering point of view, with ray trace renderers (like mental ray, V-Ray, Arnold, etc.) with full light simulations and global illumination, rather then for real-time game engines.

Answer 1

In film production, we almost never use 8-bit textures for color/albedo, because of banding, etc. (JPEG is especially problematic since by spec, it's sRGB rather than linear values.) We either use 'half' (16 bit float) or 16-bit unsigned integer values for color/albedo textures.

Answer 2

Yes, it's possible in some extreme cases for HDR lighting and tonemapping to expose banding issues in color textures. In those cases, having a higher bit depth for the textures could be useful. However, in my experience the majority of materials and ordinary lighting situations don't exhibit this problem, and most textures in a typical game are fine in 8-bit (or even less—games often use BC1 compression, which reduces them to 5-6-5-bit).

People use HDR render targets because a single scene can contain vastly different magnitudes of brightness, such as a dark room in which you can see through a window to a sunlit exterior 10–100 times brighter than the room. However, color textures don't have such a wide range of magnitudes. They represent reflectances, which are inherently in the [0, 1] range, and in practice few everyday materials are lower than about 2–5% reflectance. So an 8-bit image (with gamma encoding) can usually represent diffuse and specular colors with enough precision.

It's true that the combination of a quite dark texture with very bright lighting or an extremely overexposed camera setting can show banding in the final frame, but that would be a more unusual case.

A case where you probably would want an HDR texture is emissive materials, especially for neon signs and similar light sources. The texture would appear with its value amped up to appear as a bright light source in game, so in that case an 8-bit image could easily show banding.

Finally, it can still be useful to work at higher precision (e.g. 16-bit precision) if possible when capturing and creating textures, simply because it gives you more headroom to process the image without causing precision problems. For instance, if you need to adjust levels or color balance, you lose a little precision; that can introduce banding (especially if you do it multiple times) when starting from an 8-bit source image. A 16-bit source would be more resilient to such problems. However, the final texture used in the game would still probably be compressed to 8-bit.

Answer 3

I'd like to invite readers to read this article about Quake 2 engine rasterization technology explained in details, if they have the time.

If TLDR, please pay attention to this image:

What we see is the Albedo channel, that's what you want to encode in 16 bits if I understand your question correctly.
I'm not going to say "if it could be encoded in 256 colors for games in the past, why would we need 281474976710656 (that's 281 trillion) in new games?" even if I want to but that sounds like pixar's grumpy guy from Up. More constructively, if you noted in this image, everything is at the same lighting level. More particularly, the maximum intensity possible that preserves the desired saturation. (meaning in HSV space, V is maxed out)

The emphasis is key, we barely need any bits because albedo has no depth to encode anyway. The dynamic comes from the shading, it makes sense to work in f32 per components within shaders, and output to f16 render targets. But storing albedo textures in f16 that's not only overkill, that's a severe unjustified performance hog for our precious bandwidth.