# How to calculate ray-origin and ray-direction in vertex shader working universally for any projection and using interpolators for performance

I have been reading and watching many raymarching tutorials, but almost all of them are only working in the perspective projection. Almost all assume, that to calculate the ray direction, you just have to calculate the rd = hitpos - cameraPos, where hitpos is vertex position. This approach doesn't work when we change the projection to orthographic though.

Because there are a lot of hidden details in calculating and correctly interpolating the rays and positions, how to correctly calculate camera ray direction and ray origin like on the picture below (without any external scripts)? Rd is ray direction, Ro is ray origin, near and far are camera clip planes and Hit will be fragment's position.

I am mainly interested in HLSL implementation, specifically in Unity, but the general approach would be welcome as well. • it has to account for both orthographic and perspective projection at the same time
• interpolating directions in vertex shader can lead to weird interpolation issues later in fragment (looking similarly to issues with perspective-correct texture mapping)
• the non-trivial thing about how interpolators work for data passed from vertex to fragment
• accounting for depth buffer (and how it's not linear)
• using only data already available in the shader routines without passing things like FOV, camera position and similar to shader explicitly
• working in many different coordinate systems (local, world, view, clip)
• if a depth buffer is reversed or not

And I still don't quite understand the correct way of accounting all of those things and calculating things like worldspace pos from depth.

I'm gonna mention, that I have already achieved desired effect, but in the fragment shader (origin on near plane or hit position, depending on control, direction universally calculated for perspective and ortho). It is overcomplicated and most probably very inefficient. It uses clip-space to generate a ray: it takes the fragment coordinates in clip-space, calculates direction vector from projecting this fragment to z=0 and z=1, then calculates an inverse projection matrix to bring this vector back to camera space.

There is no universal solution here since there are many different small variations that can be applied to get a specific result.

One of the things that trips people up when computing ray's in the vertex shader is how a vector's length changes when it is interpolated. This results in vectors that have slightly different lengths arriving in the fragment shader. This is the same problem that happens with lighting vectors. With lighting the solution is to re-normalize the vector in the fragment shader. With ray casting the vector length needs to be recomputed to match the needs of the technique. This means we can effectively ignore ray length in the vertex shader. But the code must be careful to compute the proper length in the fragment shader.

One good technique is to compute the ray direction in the vertex shader and pass it to the fragment shader. The ray origin can then be computed in the fragment shader in a straight forward manner. Where ray direction is computed by finding the point of intersection with the camera space starting plane in 2D (not necessarily the projection near plane) and just plugging the distance from the camera to the plane in as the 'z' value. The vector length will be odd but its going to be changed with interpolation anyway, then transform it into the appropriate space for interpolation.

Ray origin can be computed in the fragment shader after rescaling (or computing a scaling factor) for the interpolated ray. A scaling factor can be computed with something along the lines of

scale = plane_distance * (1/camera_vector_length)

And then the Ray origin computed with:

ray_origin = interpolated_ray_direction * scale + transformed_camera_postion

It can be better to just pass a camera space 2D "ray" and another ray transformed into the same space that the ray will be marched/cast in so that there is no need to do any transformations of the ray in the fragment shader and is the reason I wrote "camera_vector_length" above. The 2D ray is really just interpolating a point across the start plane then the plane distance is plugged in as the 'z' value in the fragment shader. (saving some interpolation hardware) Also, the transformed_camera_position above, is the camera position transformed into whatever space the ray marching will be done in.

Another nice technique is to compute the ray end point, then compute a time t that is used to interpolate between the start and end points. Where t is a step size that can be set in the shader, or passed in as a parameter, computed on the fly, etc. The interpolated point is then used to sample the data. This drops ray direction out of the marching loops altogether.

Yet another reason to use t is that a random offset can be easily added to it which helps "jitter" the sampling points breaking up regular artifacts that crop up often with ray casting. The goal hear is to move the rays slightly from frame to frame rather then within a single frame.

Using these approach's results in shaders that look considerably different from the standard "Rd = hitpos - camerapos" but are more efficient and produce better results.

// out cs_ray == camera space ray
// out t_ray == transformed ray created using transform matrix
//    such as the camera to world matrix, or camera to shadow space matrix
void ComputeRayFromClipSpace( in float2 vp, out float2 cs_ray, out float3 t_ray )
{
float pd; // distance to camera space plane, ie Ray Origin plane
float s;  // aspect ratio
float g;  // focal distance

// Compute point on camera space plane
float3 hitpoint = float3( vp*pd*s/g, vp*pd*1.0/g, pd);

cs_ray = hitpoint.xy;
t_ray  = hitpoint * transform_matrix;
}


This is setup for a "full screen" effect. The inputs would usually be generated from a single clip space triangle that covers the entire vieport. More info here. It is also using info from a perspective frustum but similar info could be generated for a orthographic frustum. The main point here is to compute the points on the plane that intersect with the frustum using whatever method that is appropriate.

This function is leaving some performance on the table for clarity, the constants and transform computations.

float3 camera_ray = float3( cameraRayIn, pd ); // pd == plane distance
float length = sqrt( dot( camera_ray, camera_ray));
float scale = pd * (1.0/length);
float3 scaled_ray = t_ray_in * scale;


The scaled_ray can be used to compute the start point scaled_ray+camera_pos where camera_pos is the position of the camera in whatever space we are working in. The ray can then be rescaled (such as normalized or scaled to the required step distance) or used to compute a ray end point.

The best sources I know of for this technique are talks and books describing ray casting effects used in games like fged2.

• Thanks for a detailed answer, I'll try to wrap my head around it and try the approaches you explained. Could you possibly provide some links to the implementations or some other resources about the two techniques you mentioned above? I noticed that a devil almost always is in the details. If it isn't too much to ask, could you maybe paste some example hlsl/glsl implementation of this in here? Aug 27 at 16:26
• Meanwhile, I have written a visualization in the graphing calculator to check how vectors calculated in vertex shader get interpolated when passed to fragment shader, and like I guessed (I may have done it wrong before, but I don't know anymore), vectors which are somehow changed in vertex shader (in my case normalized) get quite messed up in the fragment shader. Here is a quick demo, green is correct interpolated ray, red is wrong ray that resulted from interpolating normalized vectors geogebra.org/calculator/fppufpcf (Under assumption that I correctly understand interpolation) Aug 27 at 16:28
• I added some code and some more description. Aug 28 at 12:50
• Thank a lot, I'm gonna mark your answer as accepted when I get to test it and I get it to work 👍 for now it must wait a bit for a bit of my spare time :P. Aug 29 at 9:46