Converting points, vectors, normals between world and object space for a ray tracer gives weird results

Question

I'm writing a ray tracer but having trouble converting rays and normals between world space and object space.

I am creating a Ray, then transforming it with transformation matrices, calculating intersections and return them.

Constructing transformation matrices:

    glm::mat4 rotate = glm::eulerAngleYXZ(- _rotation.y, - _rotation.x, - _rotation.z);
    glm::mat4 translate = glm::translate(- _position);
    glm::mat4 scale = glm::scale(glm::vec3(1.0f) / _scale);

    _transform_world_to_object = translate * rotate * scale;
    _transform_object_to_world = glm::inverse(_transform_world_to_object);

Checking Ray Sphere intersection in object space, and adding according intersections with their normal, position and distance (in world space) to provided list:

    void Sphere::computeAllRayIntersections(Ray& ray, IntersectionList& intersections) {
    //assuming ray has unit direction vector (easier calculation), then compensate for length later in distances
    float ray_to_sphere_ray_dir_dot = glm::dot(glm::normalize(ray.getDirection()), ray.getOrigin());
    float radicand = glm::dot(ray_to_sphere_ray_dir_dot, ray_to_sphere_ray_dir_dot) - (glm::dot(ray.getOrigin(), ray.getOrigin()) - 1.0f);

    if(radicand < 0.0f) {
      return;
    }

    float d1 = (- ray_to_sphere_ray_dir_dot + sqrtf(radicand)) / glm::length(ray.getDirection());

    glm::vec3 position1 = ray.positionAtDistance(d1);
    glm::vec3 normal1 = glm::normalize(ray.positionAtDistance(d1));
    //invert normal when ray starting point lies in sphere:
    if(glm::length(ray.getOrigin()) < 1.0f) {
      normal1 *= -1.0f;
    }

    Ray hit1_data(position1, normal1);
    hit1_data = this->transformHitObjectToWorld(hit1_data);
   intersections.addIntersection(std::make_unique<Intersection>(
      Intersection(
        (IGeometricObject*) this,
        glm::normalize(hit1_data.getDirection()),
        hit1_data.getOrigin(),
        d1)));

    if(radicand == 0.0f) { //stop after calculating only intersection
      return;
    }

    float d2 = (- ray_to_sphere_ray_dir_dot - sqrtf(radicand)) / glm::length(ray.getDirection());

    glm::vec3 position2 = ray.positionAtDistance(d2);
    glm::vec3 normal2 = glm::normalize(ray.positionAtDistance(d2));
    //invert normal when ray starting point lies in sphere:
    if(glm::length(ray.getOrigin()) < 1.0f) {
      normal2 *= -1.0f;
    }

    Ray hit2_data(position2, normal2);
    hit2_data = this->transformHitObjectToWorld(hit2_data);
   intersections.addIntersection(std::make_unique<Intersection>(
      Intersection(
        (IGeometricObject*) this,
        glm::normalize(hit2_data.getDirection()),
        hit2_data.getOrigin(),
        d2)));                                                           
  }

With the following code I transform a point,vector pair between coordinate spaces: The conversion from world to object space doesn't normalize and doesn't uses normal formula, while converting the intersection from object to world space does. This differentiation is based on this presentation.

    Ray Ray::transform(glm::mat4 transform, bool normalize_direction, bool is_normal) {
    glm::vec3 transformed_direction;
    if(is_normal) {
      transformed_direction = glm::transpose(glm::inverse(transform)) * glm::vec4(_direction, 0.0f);
    }
    else {
      transformed_direction = transform * glm::vec4(_direction, 0.0f);
    }
    //transformed_direction = transform * glm::vec4(_direction, 0.0f); 
    if(normalize_direction) {
      transformed_direction = glm::normalize(transformed_direction);
    }
    return Ray(transform * glm::vec4(_origin, 1.0f), transformed_direction);
  }

    Ray BaseGeometricObject::transformRayWorldToObject(Ray& ray) {
    return ray.transform(_transform_world_to_object, false, false);
  }
  Ray BaseGeometricObject::transformHitObjectToWorld(Ray& ray) {
    return ray.transform(_transform_object_to_world, false, true);
  }

When rendering with uniform scaling, the output image looks (disregarding the normal) good:

{
    "Type": "Sphere",
    "Material": "Red",
    "Point" : {
        "x": 0,
        "y": 0,
        "z": 4
    },
    "Rotation": {
        "x": 0,
        "y": 0,
        "z": 0
    },
    "Scale": {
        "x": 1,
        "y": 1,
        "z": 1
    }
}

But when I use non uniform scaling, the sphere gets seemingly distorted on a axis not modified by scale:

{
    "Type": "Sphere",
    "Material": "Red",
    "Point" : {
        "x": 0,
        "y": 0,
        "z": 4
    },
    "Rotation": {
        "x": 0,
        "y": 0,
        "z": 0
    },
    "Scale": {
      "x": 1,
      "y": 1,
      "z": 0.5
     }
}

My Camera is positioned as follows: (it's transformation matrices should't change the ray, they are applied using the Ray transform functions with a matrix built from translate * rotate).

{
    "FOV": 60,
    "Position": {
        "x":  0.0,
        "y":  0.0,
        "z":  0.0
    },
  "Rotation": {
    "x": 0.0,
    "y": 0.0,
    "z": 0.0
  }
}

Now I'm wondering whether my code is correct or not. Have I got something conceptually wrong with the coordinate transformations? I've searched a lot on the internet, but didn't find a complete guide on how to perform that for points, vectors and normals.

Answer 1

I think you have the wrong order of matrix multiplication.

    _transform_world_to_object = translate * rotate * scale;
    _transform_object_to_world = glm::inverse(_transform_world_to_object);

translate * rotate * scale is the appropriate multiplication/composition order for a conventional object-to-world coordinate transformation: first, scale the object, then rotate the scaled object, then translate the rotated object.

But you are instead doing them in the opposite order. Thus, the translation amount is scaled by the scale, so your z distance of 4 becomes 4 × 0.5 = 2, closer to the camera.

Try this instead:

    _transform_world_to_object = scale * rotate * translate;

Or you can compute the object-to-world matrix directly (this will require changing all your minus signs) and world-to-object as the inverse of it. Different path, same result.