Let's say that I have a raytracer that is rendering a scene that includes a refractive object.
When a ray hits the refractive object, I know that I can use the fresnel equation (Schlick's approximation in my case) as well as the object's base reflectivity amount to figure out how much light is reflected or transmitted at that point on the surface.
At this point, I can get the reflected color using a reflection ray (possibly recursively), an environment map, and also calculate a specular highlight for the light source(s) I have. I sum whatever reflection colors I have, multiplied by the amount of reflection from the fresnel equation.
Next, I calculate what amount of light comes in due to refraction by following the refracted ray (recursively) and multiplying that value by 1.0-reflectionAmount.
That makes sense to me so far, but where I get a little confused is for the refraction rays.
When I follow the refracted ray into the object and hit the back side of the surface the first time, i once again use the fresnel equations to calculate the reflected (internally) and refracted multipliers for the light, taking into account the possibility of total internal refraction (assuming here that the inside of the object has a higher IOR than outside of the object, an ok assumption for my case).
But, at this point, do i apply a specular highlight to the surface that i hit from the inside?
On one hand it's a surface so i feel like i should, but which ray should i use for the lighting - the refracted ray or the reflected ray, or both?
If using the refracted ray, it feels wrong to use the direction to the light source for the specular light, and feels like i should shoot the refracted ray to the surface of the object and see what the specular highlight would look like from there and use that - but do so recursively, using the fresnel equation again at each step.
Can anyone explain a decent way to handle a specular highlight in a ray tracer that supports refraction and total internal reflection?
Since this is a ray tracer, not a path tracer, instead of being limited to physically accurate solutions, I'm looking for any plausible solutions, but of course, physically inspired solutions are likely to be better!