Inverse-Transpose of Vertex Normals

Question

For the past month, I have been learning on my own and writing a little graphics library and am currently working on the transformation of vertex normals to soon support lighting/shading. I understand how if I transform the vertices by a matrix M that I need to apply the inverse-transpose of M (i.e., (M^-1 )^T ) to the surface normals. However, I am struggling to understand how this "rule" can be applied to all of the basic transformations (translate, scale, rotate) ... particularly, translation.

Using a homogeneous (w) coordinate in a 4D vector class, a Translate(x, y, z) would have no effect on a normal since its w-coordinate is 0, indicating that it is a vector, not a point. However, when we transpose the inverse of the Translate then it would have an effect on the normal (which feels incorrect).

For example, let's say that we have created a 3D triangle and want to apply a basic Translate(1, 2, 3) to the three vertices' positions. Based upon the "rule" for transforming vertex normals (i.e., inverse-transpose), we should be multiplying the three vertices' normals by Transpose(Translate(-1, -2, -3)). However, this feels incorrect since a vertex normal should remain unaffected by Translation and only be affected by Rotation and Scaling.

Any help in understanding how the inverse-transpose can be applied to vertex normals would be greatly appreciated!

Answer 1

You are correct, when they are talking about taking the inverse-transpose they are talking about the 3x3 part of the affine transformation leaving the translation since it doesn't affect the normal vector.

Check : Normal / Transforming Normals | Wikipedia

About how or why is it the inverse transpose, a simple explanation, that is also given in the book "Essential Mathematics for Games" is

Suppose we have a Point $P$ and a normal vector $n$ both represented by column matrix. Then we know that $$ n^T P = 0 $$ This is the same as the dot product. Now if we apply a transformation matrix $M$ to point $P$, we'll have to apply some sort of transformation matrix $Q$ to $n$ also. $$ \begin{eqnarray} (Q n)^T M P & = & 0 \\ n^T Q^T M P & = & 0 \end{eqnarray} $$

One solution is $$ \begin{eqnarray} Q^T M & = & I \\ Q & = & {M^{-1}}^T \end{eqnarray} $$ Hence the inverse transpose. This stands for any sort of transformation matrix $M$ be it scaling, rotation etc. Also for pure rotation matrix, since they are orthogonal, the matrix transforming the normals is the same as that of the vertices. as ${M^T}^T = M$

Answer 2

You're right, translation is not relevant in this context: it wouldn't affect the normals.

Consider a non-uniformly scaled object:

We want to get the normal matrix to transform the normals in the vertex buffer into vectors representing world space normals from the world matrix M, which encodes the translation, rotation and scaling (and shearing) of the model object.

---

If you were to interpret the 3x3 of the M in the vertex shader, you would read rotation data with scaling applied. If the world matrix contains non-uniform scaling, your normal vectors will be scaled incorrectly, and you'll get the wrong results.

The reason we take the transpose of the transform is because the rotation matrix, used to create M = TranslationMatrix x RotationMatrix x ScaleMatrix, is an Orthagonal Matrix, which means that the row vectors are orthogonal, unit vectors, i.e. they are linearly-independent and their length is one.

This type of matrices hold a property like the following:

Q^T * Q = Q * Q^T = I

If you remember the inverse rule:

Q^-1 * Q = Q * Q^-1 = I

This also means the following for the orthogonal matrices:

Q^T = Q^-1

This allows us to apply the inverse of the rotations by transposing the matrix.

In summary, taking inverse-transpose of the world matrix does 2 important things:

1. Undo transformation (Inverse)
   • undo the scaling (and shearing)
   • invert rotations
   • translate the object to the world center (irrelevant)

2. Un-undo rotations (Transpose)
   • Now, if we take the transpose of our matrix, we will get our rotations back to their original state because of the orthogonal property of the rotation matrix.

We now have our normal matrix ready to transform model-space normals defined in the vertex buffer into world-space normals in the vertex shader.