Correcting my "look-at" matrix so that it works on non-camera objects?

I'm familiar with the "look-at" matrix and the goal of being able to rotate a Camera at a specific location towards a specified target.

When I tell the Camera to look at some target, there're no issues. However, when some other non-Camera object needs to "look-at" another object, such as a hypothetical Dog looking at some Food, so that the Dog "points" in its direction, it doesn't work as expected. In this case, when the look-at matrix used for the Camera is used (without modification) for the Dog, the Dog ends up looking in the opposite direction I intended it to face.

What follows is the code I've been using to build my look-at matrix:

public static Matrix4 createLookAtMatrix(
Vector4 eyePosition,
Vector4 targetPosition,
Vector4 upDirection)
{
// forward, side, up
//
// f = normalize(targetPosition - eyePosition);
// s = normalize(f × normalize(upDirection));
// u = s × f;
Vector4 f = targetPosition.sub(eyePosition).normalize();
Vector4 s = f.cross(upDirection.normalize()).normalize();
Vector4 u = s.cross(f);

float[][] values = new float[][] {
{ s.x(),  s.y(),  s.z(), -s.dot(eyePosition) }, // row 0
{ u.x(),  u.y(),  u.z(), -u.dot(eyePosition) }, // row 1
{-f.x(), -f.y(), -f.z(), -f.dot(eyePosition) }, // row 2
{ 0f   ,  0f   ,  0f   ,  1f                 }  // row 3
};

return createFrom(values);
}


What am I missing? Is there a way to build such a "look-at" matrix so that it works for both regardless of who/what the viewer is (i.e. Camera or Dog)?

I'd like to avoid depending on "special corner cases" to detect what kind of object is trying to "look-at" something else, and so on.

My system can have multiple SceneObjects at any given moment connected to the same SceneNode (SceneNodes are the objects that actually contain the transforms) and I really need a way to build a "look-at" matrix that will work regardless of which kind of object happens to get attached to the SceneNode.

This is a screenshot of the reference book I was reading/using when writing my code above:

• oIts the exactly same computation. Feb 6, 2017 at 14:10
• @joojaa "olts"? Not sure I understood your comment. Feb 7, 2017 at 3:58
• Typo, it should read 'its'. The source you use uses a transpose trick just clear the matrix. Usually you shouldnt do this many steps beween calls as it confuses other programmers . Anyway the code is the same minus the extra matrix manipulation. I wont be writing code because your code does not tell me what all your conventions are. Feb 7, 2017 at 4:49
• @joojaa When you say "minus the extra matrix manipulation", do you mean that the matrix from the reference above should simply be transposed so that it ends up in a column-major order or were you referring to something else? My intention is to use column-major matrices, but the one above was arranged in row-major based on how it was shown in the reference (which could've been a consistency issue fixed later in the ref) I've been going over my stuff and re-checking my understanding and other things, which is why it's taking a bit longer than usual for me to reply to comments. Thanks. Feb 7, 2017 at 11:16
• @rayits From the look of it its inverted after initial placement. See the ivnerse of a orthogonal matrix (a pure rotation) is its transpose. And ofcourse the inverse of a move is just negative move. Thats why it looks like its wrog order. Its right, just that many operations are stacked in one line of code. Feb 7, 2017 at 13:09

You don't seem to get the concept right:

A look-at camera matrix is created by defining two points, the "from" point (the dog eyes for example assuming you take the position of the eyes of the dog and average them to get the point in between) and a "to" point which is where the dog looks at. So if your dog is at (3,3,3) and that you want it to look at the some food which is at (-10, 100, 20), then you need to call the lookAt() method with these 2 points and it will create a matrix that will actually orient and position the camera properly.

Now you are getting confused (probably) because most cameras in most 3D packages are positioned at the origin (by default) and looking down the z-axis (by default). You can create such camera with a lookAt() function btw that would look like that:

lookAt(Vec3f(0), Vec3f(0,0,-1));


For example. Now check what the matrix this call will produce with the matrix of the default camera in Maya. You will notice they are different. The second is an identity matrix, the first one is not.

Now for example if you create a default camera in Maya it will be position at (0,0,0) and looking down the negative z-axis. If you look at the 4x4 matrix of this camera you will get identity. And maybe what you are trying to do is to create a 4x4matrix using the lookAt function and multiply this matrix by the Maya's default camera matrix in which case, your camera will point in the wrong direction (it will be flipped along the camera's local coordinate z-axis). That's because the matrix returns by lookAt properly orient the camera (as suggested in the first paragraph) but since the default camera in Maya is already oriented along the negative-z axis, your Maya camera will look the wrong way.

So in summary:

1. If you use the lookAt method in a renderer, it will work as you expect. There's no z problem.
2. if you try to use it on a camera that is already oriented along the negative z-axis and yet has identity matrix yes indeed you will be looking the wrong way. This can be fixed easily by flipping the camera back one more time along the z-axis. Simply create another matrix that scales the object/camera by (1,1,-1).

EDIT

The OP says (see comment):

I did spot what looks like a typo: it says "(mind the direction of this vector: it is To − From not From − To)", but then you go on to write the line of code Vec3f forward = Normalize(from - to);, which contradicts what you were trying to clarify.

But a note in the reference says:

The camera is aligned along the negative z-axis. Does it mean I need to rotate the camera by 180 degrees along the y-axis or scale it up by -1 along the z-axis? Not at all. Transforming a camera is no different from transforming any other object in a scene. Keep in mind that in ray-tracing, we build the primary rays as if the camera was located in its default position. This is explained in the lesson Ray Tracing: Generating Camera Rays. This is when we actually reverse the direction of the rays. In other words, the z-coordinates of the rays' directions at this point are always negative: the camera in its default position, looks down along the negative z-axis. Those primary rays are then transformed by the camera-to-world matrix. Therefore, there is no need to account for the default orientation of the camera when the 4x4 camera-to-world matrix is built.

So if you look at the reference they use by default a right-hand coordinate system, which means x-axis points to the right, y is up and z goes out of the screen (outward, going your direction when x-axis points to the right). So they build a matrix that "follows this convention" which makes sense. Then they say in the note "yes but wait, normally the camera points down the negative z-axis, so you should be doing to - from and not from - to, and they explain that in the case of ray-tracing the fact that the camera looks down the z-axis is performed when the rays are cast (by inverting their z-coordinate).

Vec3f rayDirectionInCamSpace{ x, y, -1 }; // invert here
normalize(rayDirectionInCamSpace);
Vec3f rayDirInWorldSpace = transform(rayDirectionInCamSpace, camToWorldMat);
pixelColor = trace(rayOrigWorldSpace, rayDirInWorldSpace);


So no it's not a typo. Now the reference you are pointing out maybe assume this matrix needs to be used in some OpenGL render in which there is no possibility to invert they ray direction when they cast. So they indeed use to - from instead which is the same as multiplying the cam-to-world matrix you would get from the reference by a matrix that flips the coordinates z-axis (multiply by (1,1,1-)).

So no typo, things are quite logical in both cases. The result you get is always constrained to a lot of small details, such as the handiness of the coordinate system and how is the matrix later used down your graphics pipeline (OpenGL pipeline, ray-tracer, renderer, animation package, etc. which may follow its own conventions, etc.).

EDIT 2

Note that in the reference your provide (screenshot of book) they do target - eye but then they use the negative of that in the matrix construction. Not sure why. Then also no idea why the multiply the eye by the vectors for the translation part of the matrix. It seems to be using row-order matrices for the axis of the coordinate system but then the translation part should be at mat[3][0], mat[3][1] and mat[3][2] and not mat[0][3], mat[1][3] and mat[2][3]. Not sure what book this is but seems to be terribly wrong and confusing.

EDIT 3

Code (that seems to be the only solution):

void lookAt(const Vec3f& from, const Vec3f& to, Matrix44f &V)
{
static const Vec3f tmp(0, 1, 0);
Vec3f forward = normalize(from - to);
Vec3f right = crossProduct(normalize(tmp), forward);
Vec3f up = crossProduct(forward, right);

V[0][0] = right.x;
V[0][1] = right.y;
V[0][2] = right.z;
V[1][0] = up.x;
V[1][1] = up.y;
V[1][2] = up.z;
V[2][0] = forward.x;
V[2][1] = forward.y;
V[2][2] = forward.z;

V[3][0] = from.x;
V[3][1] = from.y;
V[3][2] = from.z;
}

int main(int argc, char **argv)
{
...
lookAt(from, to, camToWorldMatrix);
worldToCamMatrix = camera.camToWorldMatrix.inverse();

Matrix44f projectionMatrix;

Matrix44f openGLWorldToCamMatrix = worldToCamMatrix;
Matrix44f openGLProjectionMatrix = projectionMatrix;

while (!glfwWindowShouldClose(window)) {
glViewport(0, 0, width, height);

glUniformMatrix4fv(projectionLocation, 1, GL_FALSE, (const GLfloat *)&openGLProjectionMatrix[0][0]);
glUniformMatrix4fv(viewLocation, 1, GL_FALSE, (const GLfloat *)&openGLWorldToCamMatrix[0][0]);

...
wwapBuffers(window);

}

return 0;
}

• I'm not familiar with Maya, so I wouldn't know what it looks like there. If it's relevant to you, I wrote the code to load Wavefront models exported from Blender; they load/render fine, so there's no issue there. I did mention that I have a working look-at matrix applied to the camera (which needs 3 parameters, not 2: eyePosition, targetPosition, and upDirection vectors). The transformation is actually contained in a SceneNode that the Camera, or Dog, is attached to, hence the question for a way to make it work regardless of which one needs to 'look' at some specified direction. Feb 5, 2017 at 12:50
• I already replied to your question. Take the dog eye as the eyePosition of your lookout method. If your camera is looking the other way around then apply a negative scale along the camera z-axis. There is no reason one or the other solution should not work. Again if you understand what the lookAt method works, you wouldn't ask the question;-) So do you understand what it does? Feb 5, 2017 at 12:55
• I am using the dog's eye as eyePosition when trying to turn it towards the target position. I'm also using camera position as eyePosition when creating a separate look-at for the camera. However, the logic that allows the camera to look in the correct direction (i.e. face the target) leaves the dog facing in the opposite direction; the dog is not the camera position. The camera is looking at the dog which is trying to look at something else. If I scale the matrix, then the camera looks in the opposite direction I intend it to. Feb 5, 2017 at 13:03
• I've made several updates to the question, which I think now does a better job at explaining the issue, removing ambiguity by showing the code I've been using, and showing the reference on which it was based, etc. I did go over your reference, but it explains the same concepts. I did spot what looks like a typo: it says "(mind the direction of this vector: it is To − From not From − To)", but then you go on to write the line of code Vec3f forward = Normalize(from - to);, which contradicts what you were trying to clarify. Let me know if you still think my question can be improved. Feb 6, 2017 at 12:07
• I replied in the edit. Feb 6, 2017 at 12:24

Both the question and its answer is confusing.

First of all the matrix you have

is a column "Look at" view matrix rather than a model matrix. A correct column "Look At" model matrix would be

$$\begin{bmatrix} side_x&side_y&side_z&0\\ up_x&up_y&up_z&0\\ Forward_x&Forward_y&Forward_z&0\\ eye_x&eye_y&eye_z&1 \end{bmatrix}^T$$, which is what the other answer is saying. This means that matrix in the book is the inverse of the matrix given by me and the other answer.

First matrix will be used to transform the objects in the world space to the view space of the camera. Second matrix will be used to transform the camera from its local space to the world space such that camera is pointing at whatever the "target" vector is.

There is nothing special about the second matrix, you can use it on a dog to point to food, provided its mouth is along the +z direction in its local space.

So in summary, first matrix is a view matrix is used transform objects from world space to view and the second matrix is a regular model matrix which can be applied to any object in the scene to transform it from its local space to the world space with "eye" as its world position and +z direction looking at "target".

There is nothing wrong with either matrices just that they are used for accomplishing total different tasking.