# Precision errors with raytracing normal

I'm writing a raytracer in Java that draws a scene containing the elliptic paraboloid defined by the equation $$F(x,y,z)=x^2+z^2-y=0$$, as well as the hyperbolic paraboloid defined by $$G(x,y,z)=x^2-z^2-y=0$$.

To compute the surface normal to this, I'm taking the normalized gradients of the equations: $$\vec{\nabla} F(x,y,z) = \left<2x, -1, 2z\right>\\ \vec{\nabla} G(x,y,z) = \left<2x, -1, -2z\right>$$.

To ensure that the normal is on the correct surface, I take the dot product with the incident ray and flip the normal if the dot product is negative.

However, when I render the image, I observe this strange outline around the object: I can tell that this is a normal-related issue because when I color the objects based on their normals, I can see discontinuities: I have no idea why this is, but I believe it has something to do with rounding errors during this sign-change step. I'm not sure why this would be though; I'm using double-precision floating-point numbers (Java's double type) for all calculations. When I remove this check, the normals are broken, but the outline disappears. How might I be able to fix this?

Here's where the normal is computed:

Vec3 dP = pt.sub(pos);
Vec3 normal = new Vec3(2 * dP.x, -1, 2 * dP.z)
.getNormalized()
.faceForward(ray.rd);


where pos is the object's local position, and ray.rd is the ray's direction.

Here's the code responsible for the check (the faceForward function in the above code):

public Vec3 faceForward(Vec3 incident) {
boolean outside = (this.dot(incident) < 0);
return (outside ? this : this.negate());
}

• I doubt that this has to do with precision. I'd rather question the computation of the incident vector or the reflection model.
– user1703
Aug 24 at 16:10
• You can add the results without the check. Also, if you think this is caused by rounding errors then you can verify by simply change to this.dot(incident) < 0 to something like this.dot(incident) < 1e-4 to see if anything changes. Aug 25 at 1:33