Is orthonormal tangent space an industry standard for use of tangent space normal maps?

I'm investigating software that uses the unnormalized, interpolated vertex normal for creating its tangent space, creating a scaled tangent space that depends on concavity/convexity of the mesh. It both bakes and reads based around this assumption, so it creates unnormalized normal maps.

These normal maps would be incompatible with a rendering engine that used a normalized tangent space-- the angle from the normal changes with the length of the tangent space's Z basis. Likewise, normal maps built around an orthonormal tangent space would be incompatible with engines using scaled tangent spaces. This means that even if orthonormal space would be preferable, that engines might not use them just to meet a common standard.

How do commonly used engines handle this? Do they use orthonormal space or not? I have investigated Blender and xNormal, and neither use orthonormal tangent spaces, but I'm interested in how more popular commercial software handles their normal maps.

Any information about specific, big name software is appreciated-- names like Maya are basically industry standards all on their own.

1 Answer

The Orthonormal TBN tends to be the most popular. I can't speak to every piece of software out there, that question is better asked on sites specific to the software.

One reason for its popularity is a normal map can be compressed down to two channels since we know that $$x^2+y^2+z^2=1$$ this allows z to be computed in the shader, it also allows conversion to other formats for compression, such as using quaternions to represent the TBN. (see this link near the bottom of the page)

The values can also be compressed more easily since a normalized vector tends to produce decent results even when the number of bits is reduced all the way down to just 8 bits per component.

Also, be sure to look into derivative maps, a popular alternative to the usual TBN with normal map. Which have several nice advantages and some similarities to what you are describing.

• Wow, quaternion normals just seem so obvious in hindsight! I've floated the question for specific software, but unfortunately it's way too esoteric for most people, and even though the test to differentiate is easy enough, I'm still asking people to do a bake. Could you maybe mention some specific, big-name software that you know uses orthonormal TBN? If I could even just confirm what Maya does, I'd consider that industry-standard enough on its own. May 17 at 15:28
• TBN with normal maps are used in games so frequently that it seems strange for any piece of modern software to not support it. Most tool chains can extract/generate the TBN regardless of file formats or authoring tool support. Generating materials with tools like Substance almost requires a tool chain that can make a TBN for use with layered/varied materials. May 17 at 23:25
• The TBN is also part of the vertex attributes which are almost always unique to a particular project. That may be the reason you are having trouble finding support for it. May 17 at 23:29