# Why does opengl use 4d matrices for everything?

What I managed to figure out is that the first 3*3 is used for rotation and scale and the 1*3 at the end is used for position, but what is the bottom row used for? is it only for clipping related things?

• – Hubble Mar 29 '19 at 2:55
• – gallickgunner Mar 29 '19 at 10:51

The bottom row allows you to create perspective foreshortening. That is, it makes lines that are getting further away appear to converge. When arranged this way, we call this a perspective projection matrix.

There are other ways to arrange a projection matrix where that foreshortening doesn't happen. For example in an orthographic projection. This graphic shows some different projections that are possible by changing the values in the last row of a 4x4 matrix.

Mathematically, the difference between a perspective projection and an orthographic projection is that the last row of a perspective matrix has a division in the z component, whereas the orthographic projection has a 0 in the z component.

So,

A lot of folks would tell you that a 4x4 matrix is used so you can get a translation component rolled into your linear transformation. Meaning that with a 3x3 matrix you can only compute rotations, scales, and shears. (and some would argue that a shear is really another type of rotation) This same person might also go into great detail about affine transformations and homogeneous coordinate systems. And they would be right.

But that's not the reason.

All this might leave the astute observer thinking, well wouldn't a 3x3 matrix get the transformation out of the way then just use vector math to add our translation on at the end saving lots of special math, hardware, and precious resources?

Well this is perfectly plausible, and a little extra thinking would leave use with the idea of adding a 4th column/row to our matrix giving a 3x4 matrix. Now, a translation is suddenly perfectly plausible and would create a pretty world where you can do all the operations needed to move and deform objects all day long saving lots of math and resource. And you would be right. You can in fact do this very thing when computing your Model and View matrices. Handling vectors would require a little extra trickery but we could manage it.

So, now, on this theoretical video card, since all this has to be implemented in hardware, we have gotten or world models/meshs/blobs whatnots and whathaveyous all into place and are ready to project them from our 3D world onto our 2D screen. And we think: this too isn't a problem since a projection is really just a non-linear scaling operation that squeezes things around so the stuff we want to see falls inside our clipping volume.

A little bit of algebra later...

We have mountains of triangles all of which are ready to have their attributes computed, and interpolations...well interpolated. We then use standard interpolation, check our work very carefully and what we get on our video monitor looks like some kind of star trek space anomaly got hold of it.

This is where the problem comes up: It turns out that you can't just interpolate values directly after performing a perspective projection. Instead you must use a special form of interpolation called...wait for it...Perspective Correct Interpolation. And the solution to perspective correct interpolation leads use to the correct form of the projection matrix which has 4 rows and 4 columns. That matrix after being applied to a 4 component position vector results in a nonzero value in the w component of the vector. This value will be specific to each triangle depending on how it is oriented. And, at long last, the video card hardware can perform interpolation on all the values for our triangles, including colors, texture coordinates, normal vectors, ect. Then when that it is all done, hardware can perform the "perspective divide" resulting in correctly interpolated values for the triangles, along with correctly interpolated attributes.

And that is the real reason everyone everywhere uses 4x4 matrices in hardware and why they are littered throughout API's like OpenGL. Hardware vendors could probably hack their way around some of the math related to translations and transformations but when it comes to projection, you really can't get away from the 4x4 matrix. (but you can sure optimize the heck out of it)

The math for deriving the correct form of a projection isn't to terribly complicated, but the full explanation is fairly long so I will defer to a good book for that, and the best explanation I have seen for deriving a projection matrix that will allow perspective correct interpolation is chapter 6, section 6.2 of the Lengyel book, "Foundations of Game Engine Development".

If we put homogeneous ordinates to the backburner for a second. Then there is also a second reason:

Mathematical completeness. A 3 by 4 or 4 by 3 matrix is not invertible (although that does not mean you couldn't calculate a inverse by other means just not by standard matrix algebra). Inverse is critical for hierarchy calculations.

Now you could posit that the last computer part is always same and be done with it. But having worked for a company that in their main software did just this i would say that it is a mistake. Well, sure now the matrices are invertible but not general. I can now not safely transpose my matrices for example. The truth is that i lose out on a lot of possible mathematics knowledge that i could use because this one part is missing. At the end this was a common source of errors, anger and confusion among clients.

Homogeneous coordinates are just one possible application of this mathematical completeness. It turns out to be very useful if you want to model perspective. But its far from the only benefit.

• homogenious xD Contrary to your belief 3x4 and 4x3 matrices can have a left and right inverse, and you can make it work in terms of CG. Not only that but you can make it work even without that as long as you realize that your matrices are products only of $S,R,T$ matrices. To add to this, you can in fact go with 3x3 matrices which are invertible and keep a translation vector which can be equivalent. Or even use a quaternion, scale, translation representation. The real answer is that you do not have to use 4x4 matrices, it's just one way of going about things. – lightxbulb Apr 3 '19 at 14:38
• @lightxbulb yes off course but then they are not general matrices. And yes you can decompose and have pseudo inverses too but they dont fulfill classical matrix algebra. And yes you CAN assume that the last line is given. I gave you a example where it has been done. I just said it leaves a lot to be desired because i can now not use the data as a general matrix. It turns it has many properties. But then you dont need to have any matrices to do 3D. Its just convenient to have a general purpose container that has clearly define math. AS for SRT well yes but having skew is useful too – joojaa Apr 3 '19 at 15:27
• @lightxbulb is it somehow better if i point out that one could calculate the inverses by other means. (such as by negating the last row and transposing the rotate part and inverting scale part). – joojaa Apr 3 '19 at 15:37
• My issue is really that you presented this as the main reason as to why 4x4 matrices are used, when it simply is not, and you can have well defined inverses for 3x4 and 4x3 matrices that you just waved off (when there are actual benefits when using those such as bandwidth and ops). Not only is it not the main reason, but there are various other representations that are better in some sense depending on what you're doing (within computer graphics mind you). None of the answers here is really satisfactory, the most upvoted one touches upon a more important aspect, but it's obviously missing... – lightxbulb Apr 3 '19 at 15:47
• ...a lot too. I would've written an exhaustive answer if it wouldn't have taken at least a few hours, and several pages, and there's the fact that most of the major bits and pieces that this answer would've containted are already available in some for or another throughout books and blogs on the subject. The other reason is that I don't believe that many people will appreciate the details that would go into an exhaustive answer, most look for something similar to the answer referring to projections. – lightxbulb Apr 3 '19 at 15:50