3
$\begingroup$

I’ve recently gotten into shader art , using sites like shadertoy and various tutorials for getting started.

One thing I noticed is that a typical setup involves ray marching whereby:

  1. We set a hypothetical camera / eye, typically somewhere on the negative Z axis looking forward at the direction of XY plane
  2. Our (2D) image plane is typically the XY plane
  3. We then project out rays from the origin which is our eye / camera, and for each pixel (x, y) on our image we set the direction of the corresponding ray emanating from the eye and passing through this pixel to be (x, y, 1)

My question is specifically about this ray direction - why is z simply set to 1 irrespective of the actual camera position on the z axis? Is this just a hack that doesn’t matter eg because of some subsequent normalization , does this represent some actual parameter of the camera (focal length etc) or otherwise? Most tutorials gloss over this without explanation so curious where this comes from.

$\endgroup$

1 Answer 1

3
$\begingroup$

It depends on your setup. Of course you can normalize the direction to have length 1, but it is usually unnecessary. The following figure depicts the differences:

Round frustum end face Flat frustum end face
enter image description here enter image description here

Given the same step (for example, 128 ray marching steps, typical in NeRFs), you can guess which one is resulted from the normalized direction. The left one, for sure, since it reflects the spherical wavefront emitted from the camera. If we simply set z to be 1 (or -1), the forward direction will have the same step size, no matter which pixel we are on. This is more conventional in graphics textbooks, when we refer to camera frustum, we should be referring to shape as in the right figure. Normalizing the direction can be of use if you want the ray marching resolution to be direction-independent, since in the right case, the pixels closer to the screen edge will be more sparsely distributed in 3D space.

$\endgroup$
5
  • $\begingroup$ Thank you for detailed response! A more basic clarification perhaps - if my camera is at say (0,0,-5) and (for the invocation of the fragment shader for pixel (x, y)) the ray direction is set at (x, y, 1), it will actually intersect the XY plane at (5x, 5y, 0) - I suppose this is fine since it is a scaling factor that is applied to all positions uniformly ? $\endgroup$
    – giorgio
    Commented Mar 31 at 3:40
  • $\begingroup$ It seems to me like this is de facto a way to control field of view / focal length, eg the lower the z value we set in the ray direction corresponds to larger field of view? $\endgroup$
    – giorgio
    Commented Mar 31 at 3:45
  • 1
    $\begingroup$ "lower the z value we set in the ray direction corresponds to larger field of view" Yes. You can always calculate the FoV angle with your z value and the span in X, Y axis. Result of arctan will get greater as z decreases. $\endgroup$ Commented Mar 31 at 12:04
  • $\begingroup$ Thank you! Your answer about the further considerations to do with step size and the geometry of the frustum have answered the question I would’ve probably ask next now that I understand this :-) I originally got interested in ray marching in the context of 3D deep learning so the comment about usage in NeRFs is also very interesting. $\endgroup$
    – giorgio
    Commented Mar 31 at 16:16
  • 1
    $\begingroup$ For NeRFs, however, you should not tweak the z value and always leave it to 1 or -1 (when generate the ray directions from pixel coords). Since with near, far and number of samples per ray, you can obtain the starting point, end point and other samples on the ray with simple multiplication. For example: near * ray_dir + camera_pos is usually the starting point. Changing z values of rays will change the meaning of near and far (depth). $\endgroup$ Commented Apr 1 at 1:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.