Is there something wrong with a column camera matrix?

Question

I am writing a simple ray tracer. I was working on my camera matrix when I encountered a problem. When I changed the look vector, so that it headed down, the image, which was supposed to go up, actually went down as well. Likewise, if I changed the look vector so that it headed up, the image went with it. I understand that the camera matrix's job is to change world coordinate into eye coordinate. Since I am using column matrices, here's my camera matrix, with $r$ being right vector, $l$ being lookAt vector and $u$ being up vector

$$ \begin{bmatrix} r_x & u_x & -l_x & 0 \\ r_y & u_y & -l_y & 0 \\ r_z & u_z & -l_z & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix} $$

This is the transpose of the gluLookAt matrix of openGL. I dont understand why openGL use their matrix though, since they are also using column matrices.

Answer 1

It was getting a little big to fit in the comments so posting it as an answer instead. Might not be a solution to your problem but the concept is related.

People usually forget that whenever you define a transformation matrix by placing respective basis vectors in respective columns, you are specifying that with respect to another basis usually the right-handed or left handed world coordinate space. For example your above matrix

\begin{bmatrix} r_x & u_x & -l_x & 0 \\ r_y & u_y & -l_y & 0 \\ r_z & u_z & -l_z & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}

Reason why you put - sign in the third column is you are working with a right handed coordinate system and the camera is looking in the negative direction (the -Z axis). So all these basis vectors are also with respect to another basis vector. This means your camera's Z axis is actually the negative of the viewing direction.

Let's make this matrix simple so we can understand what happens when we multiply a matrix with a vector. Let the matrix $M$ be \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & -1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}

The above matrix is wrong. The correct one is,

\begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}

This is your camera initially without any rotation/translation etc. This means your camera space $+Z$ axis maps to world space $-Z$ axis. Now consider a camera space coordinate $[2,3,6,0]^T$. Multiplying this vector by the matrix gives you $[2,3,-6,0]^T$. This is the world space representation of $[2,3,6,0]^T$.

Actually after clearing my concepts a little more, Both the camera space and the world space are aligned for Right handed coordinate systems. This means the X,Y,Z axis of world space align with X,Y,Z of the camera space.

The 3rd column of the matrix actually points to $-V_{dir}$ which basically says "The Z-axis of the camera space is negative of the viewing direction". Since

$V_{dir} = [0,0,-1]$

The negative of it is simply the positive $Z$ axis. This means, points/vectors in world space remains the same as in camera space (when camera is at initial position)

So as Nathan pointed your matrix transforms from camera space to world space. To do the inverse we just take the inverse of the matrix which is the transpose if the basis is ortho-normal.

One of the reasons the transformation matrix is built this way is because it's much easier. It's easier to think about the transformed basis vectors of camera or any other space with respect to the world, write the matrix then invert it to go from the world space to that target space.

Answer 2

What you have there is the matrix that goes from eye space (camera space) to world space. You can see this by observing that, for instance, if you apply this matrix to the column vector $[1, 0, 0, 0]^T$, which is the right-vector in eye space, then it will return $[r_x, r_y, r_z, 0]^T$, the right-vector in world space.

You want the matrix that transforms the other direction, which would be the inverse of this one. And assuming $r, u, l$ form an orthonormal basis, the inverse will simply be the transpose.