Is orthonormal tangent space an industry standard for use of tangent space normal maps?

Question

I'm investigating software that uses the unnormalized, interpolated vertex normal for creating its tangent space, creating a scaled tangent space that depends on concavity/convexity of the mesh. It both bakes and reads based around this assumption, so it creates unnormalized normal maps.

These normal maps would be incompatible with a rendering engine that used a normalized tangent space-- the angle from the normal changes with the length of the tangent space's Z basis. Likewise, normal maps built around an orthonormal tangent space would be incompatible with engines using scaled tangent spaces. This means that even if orthonormal space would be preferable, that engines might not use them just to meet a common standard.

How do commonly used engines handle this? Do they use orthonormal space or not? I have investigated Blender and xNormal, and neither use orthonormal tangent spaces, but I'm interested in how more popular commercial software handles their normal maps.

Any information about specific, big name software is appreciated-- names like Maya are basically industry standards all on their own.

Answer 1

The Orthonormal TBN tends to be the most popular. I can't speak to every piece of software out there, that question is better asked on sites specific to the software.

One reason for its popularity is a normal map can be compressed down to two channels since we know that $x^2+y^2+z^2=1$ this allows z to be computed in the shader, it also allows conversion to other formats for compression, such as using quaternions to represent the TBN. (see this link near the bottom of the page)

The values can also be compressed more easily since a normalized vector tends to produce decent results even when the number of bits is reduced all the way down to just 8 bits per component.

Also, be sure to look into derivative maps, a popular alternative to the usual TBN with normal map. Which have several nice advantages and some similarities to what you are describing.