I'm trying to build a raytracer and I use this article on how to build camera system. The problem is that when, after calculating ray direction in camera space, I multiply it by camera-to-world transformation matrix and my camera seems to rotate in opposite directions and works correctly if I inverse transformation matrix before multiplication.
Here is the code (I use glm library and right-handed coordinate system).

Some data

glm::vec3 origin_ = glm::vec3(0.f);// camera origin
cont glm::vec3 kDirection = glm::vec3(0.f, 0.f, -1.f);
cont glm::vec3 kUp = glm::vec3(0.f, 1.f, 0.f);
float aspect_ratio_ = (float)raster_height_ / raster_width_;

// bug !!! rotates in opposite direction  (camera is actually tilted down)
glm::mat4 camera_to_world_ = glm::lookAtRH(origin_, glm::vec3(0.f, 0.2f, -1.f), kUp);
// works !!! (camera is tilted up)
glm::mat4 camera_to_world_ = glm::inverse(glm::lookAtRH(origin_, glm::vec3(0.f, 0.2f, -1.f), kUp));

And function that generates camera rays

// Calculate ray as if camera is located at 0,0,0 and pointing into negative z direction
// Then transform ray direction to desired plase
// x,y - pixel coordinates of raster image
// calculate as if raster image (screen) is 1.0 unit away from origin (eye)
Ray Camera::GenRay(const uint32_t x, const uint32_t y) {
  glm::vec3 ray_direction = kDirection;
  // from raster space to NDC space
  glm::vec2 pixel_ndc((x + 0.5f) / raster_width_, (y + 0.5f) / raster_height_);
  // from NDC space to camera space
  float scale = tan(fov_ / 2.0f);
  ray_direction.x = (2.0f * pixel_ndc.x - 1.0f) * scale; // *aspect_ratio_;
  ray_direction.y = (1.0f - 2.0f * pixel_ndc.y) * scale * aspect_ratio_;
  // apply camera-to-world rotation matrix to direction
  ray_direction = camera_to_world_ * glm::vec4(ray_direction, 0.0f);

  return  Ray(origin_, ray_direction, Ray::Type::kPrimary);

I really can't understand the root of a problem so any help us appreciated.


The problem is that instead of a camera-to-world matrix, a world-to-camera matrix is being made by glm::lookAtRH.

This is because GLM is a math library made for OpenGL. In OpenGL, you do not move the 'camera' as there is no camera in OpenGL. You move the whole world the other way around, so that it lines up in the frustum. The lookAtRH method returns a matrix that moves the whole world in a way that it seems like you have the camera there.

Inversing the matrix is a fine way to get the proper camera-to-world matrix.

Scrathapixel does have an article regarding making your own lookAt function. One thing to note about this article is that they use a row-major notation. It might be that you need to switch it up and use a column-major notation (just swap the x index with the y index).

Placing a Camera: the LookAt Function | Scratchapixel

  • $\begingroup$ Very precise and finely noted point! $\endgroup$ – Amit Tomar Oct 17 '18 at 10:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.