# Why does opengl use 4d matrices for everything?

What I managed to figure out is that the first 3*3 is used for rotation and scale and the 1*3 at the end is used for position, but what is the bottom row used for? is it only for clipping related things?

• Commented Mar 29, 2019 at 2:55
• Commented Mar 29, 2019 at 10:51
• Related: gamedev.stackexchange.com/questions/72044/… Why do we use 4x4 matrices to transform things in 3D? Commented Feb 27, 2023 at 9:17

So,

A lot of folks would tell you that a 4x4 matrix is used so you can get a translation component rolled into your linear transformation. Meaning that with a 3x3 matrix you can only compute rotations, scales, and shears. (and some would argue that a shear is really another type of rotation) This same person might also go into great detail about affine transformations and homogeneous coordinate systems. And they would be right.

But that's not the reason.

All this might leave the astute observer thinking, well wouldn't a 3x3 matrix get the transformation out of the way then just use vector math to add our translation on at the end saving lots of special math, hardware, and precious resources?

Well this is perfectly plausible, and a little extra thinking would leave use with the idea of adding a 4th column/row to our matrix giving a 3x4 matrix. Now, a translation is suddenly perfectly plausible and would create a pretty world where you can do all the operations needed to move and deform objects all day long saving lots of math and resource. And you would be right. You can in fact do this very thing when computing your Model and View matrices. Handling vectors would require a little extra trickery but we could manage it.

So, now, on this theoretical video card, since all this has to be implemented in hardware, we have gotten or world models/meshs/blobs whatnots and whathaveyous all into place and are ready to project them from our 3D world onto our 2D screen. And we think: this too isn't a problem since a projection is really just a non-linear scaling operation that squeezes things around so the stuff we want to see falls inside our clipping volume.

A little bit of algebra later...

We have mountains of triangles all of which are ready to have their attributes computed, and interpolations...well interpolated. We then use standard interpolation, check our work very carefully and what we get on our video monitor looks like some kind of star trek space anomaly got hold of it.

This is where the problem comes up: It turns out that you can't just interpolate values directly after performing a perspective projection. Instead you must use a special form of interpolation called...wait for it...Perspective Correct Interpolation. And the solution to perspective correct interpolation leads use to the correct form of the projection matrix which has 4 rows and 4 columns. That matrix after being applied to a 4 component position vector results in a nonzero value in the w component of the vector. This value will be specific to each triangle depending on how it is oriented. And, at long last, the video card hardware can perform interpolation on all the values for our triangles, including colors, texture coordinates, normal vectors, ect. Then when that it is all done, hardware can perform the "perspective divide" resulting in correctly interpolated values for the triangles, along with correctly interpolated attributes.

And that is the real reason everyone everywhere uses 4x4 matrices in hardware and why they are littered throughout API's like OpenGL. Hardware vendors could probably hack their way around some of the math related to translations and transformations but when it comes to projection, you really can't get away from the 4x4 matrix. (but you can sure optimize the heck out of it)

The math for deriving the correct form of a projection isn't to terribly complicated, but the full explanation is fairly long so I will defer to a good book for that, and the best explanation I have seen for deriving a projection matrix that will allow perspective correct interpolation is chapter 6, section 6.2 of the Lengyel book, "Foundations of Game Engine Development".

The bottom row allows you to create perspective foreshortening. That is, it makes lines that are getting further away appear to converge. When arranged this way, we call this a perspective projection matrix.

There are other ways to arrange a projection matrix where that foreshortening doesn't happen. For example in an orthographic projection. This graphic shows some different projections that are possible by changing the values in the last row of a 4x4 matrix.

Mathematically, the difference between a perspective projection and an orthographic projection is that the last row of a perspective matrix has a division in the z component, whereas the orthographic projection has a 0 in the z component.

If we put homogeneous ordinates to the backburner for a second. Then there is also a second reason:

Mathematical completeness. A 3 by 4 or 4 by 3 matrix is not invertible (although that does not mean you couldn't calculate a inverse by other means just not by standard matrix algebra). Inverse is critical for hierarchy calculations.

Now you could posit that the last computer part is always same and be done with it. But having worked for a company that in their main software did just this i would say that it is a mistake. Well, sure now the matrices are invertible but not general. I can now not safely transpose my matrices for example. The truth is that i lose out on a lot of possible mathematics knowledge that i could use because this one part is missing. At the end this was a common source of errors, anger and confusion among clients.

Homogeneous coordinates are just one possible application of this mathematical completeness. It turns out to be very useful if you want to model perspective. But its far from the only benefit.

• homogenious xD Contrary to your belief 3x4 and 4x3 matrices can have a left and right inverse, and you can make it work in terms of CG. Not only that but you can make it work even without that as long as you realize that your matrices are products only of $S,R,T$ matrices. To add to this, you can in fact go with 3x3 matrices which are invertible and keep a translation vector which can be equivalent. Or even use a quaternion, scale, translation representation. The real answer is that you do not have to use 4x4 matrices, it's just one way of going about things. Commented Apr 3, 2019 at 14:38
• @lightxbulb yes off course but then they are not general matrices. And yes you can decompose and have pseudo inverses too but they dont fulfill classical matrix algebra. And yes you CAN assume that the last line is given. I gave you a example where it has been done. I just said it leaves a lot to be desired because i can now not use the data as a general matrix. It turns it has many properties. But then you dont need to have any matrices to do 3D. Its just convenient to have a general purpose container that has clearly define math. AS for SRT well yes but having skew is useful too Commented Apr 3, 2019 at 15:27
• @lightxbulb is it somehow better if i point out that one could calculate the inverses by other means. (such as by negating the last row and transposing the rotate part and inverting scale part). Commented Apr 3, 2019 at 15:37
• My issue is really that you presented this as the main reason as to why 4x4 matrices are used, when it simply is not, and you can have well defined inverses for 3x4 and 4x3 matrices that you just waved off (when there are actual benefits when using those such as bandwidth and ops). Not only is it not the main reason, but there are various other representations that are better in some sense depending on what you're doing (within computer graphics mind you). None of the answers here is really satisfactory, the most upvoted one touches upon a more important aspect, but it's obviously missing... Commented Apr 3, 2019 at 15:47
• ...a lot too. I would've written an exhaustive answer if it wouldn't have taken at least a few hours, and several pages, and there's the fact that most of the major bits and pieces that this answer would've containted are already available in some for or another throughout books and blogs on the subject. The other reason is that I don't believe that many people will appreciate the details that would go into an exhaustive answer, most look for something similar to the answer referring to projections. Commented Apr 3, 2019 at 15:50

3x3 matrices cannot represent 3D translations, but 4x4 matrices can

A simple argument why 3D translations are not possible with 3x3 matrices is that translation can take the origin vector:

0
0
0


away from the origin, say to x = 1:

1
0
0


But that would require a matrix such that:

| a b c |   |0|   |1|
| d e f | * |0| = |0|
| g h i |   |0|   |0|


But that is impossible.

A "more overkill"/"more elegant" argument is the singular Value Decomposition theorem, which says that every matrix can be made up with two rotations and one scaling operation:

• rotate to align with eignevectors
• scale with the eigenvectors
• rotate to the final desired position

No translations there.

Why is it possible to use matrices to represent rotations?

Many modeled objects (e.g. a car chassis) or part of modeled objects (e.g. a car tire, a driving wheel) are solids: the distances between vertexes never change.

The only transformations we want to do to on them are rotations and translations.

Matrix multiplication can encode both rotations (NxN matrix) and translations (we need one extra dimension).

Rotation matrices have explicit formulas, e.g.: a 2D rotation matrix for angle a is of form:

cos(a) -sin(a)
sin(a)  cos(a)


There are analogous formulas for 3D, but note that 3D rotations take 3 parameters instead of just 1.

Translations are less trivial and will be discussed later. They are the reason we need 4D matrices.

As a cultural fact, the set of all rotation matrices in a dimension is called the special orthogonal group (SO(n)), which is also a Lie group, and has fundamental applications in particle physics.

Why is it nice to use matrices at all?

Because the composition of multiple matrices can be pre-calculated by matrix multiplication.

E.g., if we are going to translate one thousand vectors v of our car chassis with matrix T and then rotate with matrix R, instead of doing:

v2 = T * v


and then:

v3 = R * v2


for each vector, we can pre-calculate:

RT = R * T


and then do just one multiplication for every vertex:

v3 = RT * v


Even better: if we then want place the vertexes of tire and driving wheel relative to the car, we just multiply the previous matrix RT by the matrix relative to the car itself.

This naturally leads to maintaining a stack of matrices:

• calculate chassis matrix
• multiply by tire matrix (push)
• remove tire matrix (pop)
• multiply by driving wheel matrix (push)
• ...

This stack was explicitly present in the older deprecated immediate mode OpenGL APIs such as glPushMatrix, but with the advent of shaders this responsibility was passed more directly to the shader authors.

How adding one dimension solves the problem

Let's consider the case from 1D to 2D which is easier to visualize.

A matrix in 1D is just one number, and as we've seen in 3D it can't do a translation, only a scaling.

But if we add the extra dimension as we get:

| 1 dx | * |x|  = | x + dx * y |
| 0  1 |   |y|    | y          |


Now, since we added this extra dimension ourselves, we are free to choose a fixed value for y that suites our needs, so we choose y == 1 to get:

| x + dx |
| 1      |


x + dx


as we wanted.

This 2D transformation is so important that it has a name: shear transformation.

It is cool to visualize this transformation:

Note how every horizontal line (fixed y) is just translated (larger y being translated more).

We just happened to take the line y = 1 as our new 1D line, and translate it with a 2D matrix.

Things are analogous in 3D, with 4D shear matrices of the form:

| 1 0 0 dx |   | x |   | x + dx |
| 0 1 0 dy | * | y | = | y + dy |
| 0 0 1 dz |   | z |   | z + dz |
| 0 0 0  1 |   | 1 |   |      1 |


And our old 3D rotations / scaling are now of form:

| a b c 0 |
| d e f 0 |
| g h i 0 |
| 0 0 0 1 |


This Jamie King video tutorial is also worth watching.

Affine space

Affine space is the space generated by all our 3D linear transformations (matrix multiplications) together with the 4D shear (3D translations).

If we multiply a shear matrix and a 3D linear transformation, we always get something of the form:

| a b c dx |
| d e f dy |
| g h i dz |
| 0 0 0  1 |


This is the most general possible affine transformation, which does 3D rotation / scaling and translation.

One important property is that if we multiply 2 affine matrices:

| a b c dx |   | a2 b2 c2 dx2 |
| d e f dy | * | d2 e2 f2 dy2 |
| g h i dz |   | g2 h2 i2 dz2 |
| 0 0 0  1 |   |  0  0  0   1 |


we always get another affine matrix of form:

| a3 b3 c3 (dx + dx2) |
| d3 e3 f3 (dy + dy2) |
| g3 h3 i3 (dz + dz2) |
|  0  0  0          1 |


Mathematicians call this property closure, and is required to define a space.

For us, it means that we can keep doing matrix multiplications to pre-calculate final transformations happily, which is why use used matrices in the first place, without ever getting more general 4D linear transformations which are not affine.

Frustum projection

But wait, there is one more important transformation that we do all the time: glFrustum, which makes an object 2x further, appear 2x smaller.

First get some intuition about glOrtho vs glFrustum at: https://stackoverflow.com/questions/2571402/explain-the-usage-of-glortho/36046924#36046924

As a quick reminder, a projection is the act of taking 3D points and plotting them on the 2D camera plane, possibly taking into account perspective if we want to be realistic. This is the key step to translate between the 3D world of vertex shaders and the 2D world of fragment shaders: https://stackoverflow.com/questions/17789575/what-are-shaders-in-opengl-and-what-do-we-need-them-for/36211337#36211337

glOrtho can be done just with translations + scaling , but how can we implement glFrustum with matrices?

Suppose that:

• our eye is at the origin, looking at -z
• the screen (near plane) is at z = -1 that is a square of length 2
• the far plane of the frustum is at z = -2

If only we allowed more general 4-vectors of type:

(x, y, z, w)


with w != 0, and in addition we identify every (x, y, z, w) with (x/w, y/w, z/w, 1), then a frustum transformation with the matrix would be:

| 1 0  0 0 |   | x |   |  x |               | x / -z |
| 0 1  0 0 | * | y | = |  y | identified to | y / -z |
| 0 0  1 0 |   | z |   |  z |               |     -1 |
| 0 0 -1 0 |   | w |   | -z |               |      1 |


If we throw away:

• z because projection means to take a 3D point and place it in a 2D plane
• w because it is just a mathematical calculation artifact we invented

we get:

• x_proj = x / -z
• y_proj = y / -z

which is exactly what we wanted! We can verify that for some values, e.g.:

• if z == -1, exactly on the plane we're projecting to, x_proj == x and y_proj == y.
• if z == -2, then x_proj = x/2: objects are half size.

Note how the glFrustum transform is not of affine form: it cannot be implemented just with rotations and translations.

The mathematical "trickery" of adding the w and dividing by it is called homogeneous coordinates