After reading a few articles online I can confidently say I am clueless on how Anti-Aliasing works when using Ray Tracing.

All I understand is that A Single Pixel/Ray is split into 4 sub-pixels and 4 rays rather than 1.

Could somebody please explain how this is done (preferably with code)?


4 Answers 4


I think it's safe to say that there are two different ways of doing AA in raytracing:

1: if you have the final image and the depth image it is possible to apply almost all existing techniques that are used in games (FXAA, etc) Those work directly on the final image and are not related to raytracing

2: the second method is to take into account multiple rays for each pixel and then averaging the result. For a very simple version think of it like this:

  • you render first an image of size 1024x1024, one ray for each pixel (for example)
  • after rendering, you scale the image to 512x512 (each 4 pixels are avereged into one) and you can notice that the edges are smoother. This way you have effectively used 4 rays for each pixel in the final image of 512x512 size.

There are other variations on this method. For example you can adapt the number of samples for pixels that are right at the edge of the geometry meaning that for some pixels you will have only 4 samples , and for others 16.

Check the links in the comments above.

  • $\begingroup$ So basically I render an image to a large size and when saving it to an image, downscale it to a lower size? That seems quite simple :)! Is this the super sampling method? $\endgroup$ Commented Nov 10, 2016 at 14:27
  • 2
    $\begingroup$ @Arjan Singh yes it's en.wikipedia.org/wiki/Supersampling , but this is the slowest of them all , raytracing allows you to easily do adaptive supersampling, which can perform a lot better $\endgroup$
    – Raxvan
    Commented Nov 10, 2016 at 14:54

Raxvan is completely right that "traditional" anti aliasing techniques will work in raytracing, including those that use information such as depth to do antialiasing. You could even do temporal anti aliasing in ray tracing for instance.

Julien expanded on Raxvan's 2nd item which was an explanation of super sampling, and showed how you'd actually do that, also mentioning that you can randomize the location of the samples within the pixel but then you are entering signal processing country which is a lot deeper, and it definitely is!

As Julien said, if you want to do $N$ samples per pixel, you can break the pixel up into $N$ evenly distributed sample points (on a grid basically) and average those samples.

If you do that, you can still get aliasing though. It is better than NOT doing it, because you are increasing your sampling rate, so will be able to handle higher frequency data (aka smaller details), but it can still cause aliasing.

If you instead take $N$ random samples within a pixel, you are effectively trading aliasing for noise. Noise is easier on the eyes and looks more natural than aliasing, so is the preferred result usually. I believe it's even provably the ideal situation with higher sample counts but don't have more info on that ):

When you use just "regular" random numbers like you'd get from rand() or std::uniform_int_distribution, that is called "white noise" because it contains all frequencies, like how white light is made up of all other colors (frequencies) of light.

Using white noise to randomize the samples within a pixel has the problem that sometimes your samples will clump together. For instance, if you average 100 samples in a pixel, but they ALL end up being in the upper left corner of the pixel, you aren't going to get ANY information about the other parts of the pixel, so your final resulting pixel color will be missing information about what color it should be.

A better approach is to use something called blue noise which only contains high frequency components (like how blue light is high frequency light).

The benefit of blue noise is that you get even coverage over the pixel, like you get with a uniform sampling grid, but, you still get some randomness, which turns aliasing into noise and gives you a better looking image.

Unfortunately, blue noise can be very costly to compute, and the best methods all seem to be patented (what the heck?!), but one way to do this, invented by pixar (and patented too i think but not 100% sure) is to make an even grid of sample points, then randomly offset each sample point a small amount - like a random amount between plus or minus half the width and height of the sampling grid. This way you get a sort of blue noise sampling for pretty cheap.

Note that this is a form of stratified sampling, and poisson disk sampling is a form of that too, which is also a way of generating blue noise: https://www.jasondavies.com/poisson-disc/

If you are interested in going deeper you probably will also want to check out this question and answer!

What is the fundamental reasoning for anti aliasing using multiple random samples within a pixel?

Lastly, this stuff is starting to stray into the realm of monte carlo path tracing which is the common method for doing photorealistic raytracing. if you are interested in learning more about that, give this a read!


  • $\begingroup$ As soon as I saw your name, I knew you'd include something on blue noise :) $\endgroup$
    – Hubble
    Commented Mar 25, 2020 at 19:39
  • $\begingroup$ Since this popped up as active again: To explain why this is the case - blue noise sampling patterns excel at integrating functions with most of the energy concentrated in the low frequencies. So by using a blue noise pattern you effectively assume that your signal is mostly smooth/dominated by low frequencies. For high frequency signals, this may actually be counterproductive. Fortunately, natural images have their energy concentrated in the low frequencies. For more details see: sampling.mpi-inf.mpg.de/2019-singh-fourier.html $\endgroup$
    – lightxbulb
    Commented Mar 26, 2020 at 15:39

Let's suppose a fairly typical raytracing main loop:

struct Ray
    vec3 origin;
    vec3 direction;

RGBColor* image = CreateImageBuffer(width, height);

for (int j=0; j < height; ++i)
    for (int i=0; i < width; ++i)
        float x = 2.0 * (float)i / (float)max(width, height) - 1.0;
        float y = 2.0 * (float)j / (float)max(width, height) - 1.0;

        vec3 dir = normalize(vec3(x, y, -tanHalfFov));
        Ray r = { cameraPosition, dir };

        image[width * j + i] = ComputeColor(r);

One possible modification of it to do 4 samples SSAA would be:

float jitterMatrix[4 * 2] = {
    -1.0/4.0,  3.0/4.0,
     3.0/4.0,  1.0/3.0,
    -3.0/4.0, -1.0/4.0,
     1.0/4.0, -3.0/4.0,

for (int j=0; j < height; ++i)
    for (int i=0; i < width; ++i)
        // Init the pixel to 100% black (no light).
        image[width * j + i] = RGBColor(0.0);

        // Accumulate light for N samples.
        for (int sample = 0; sample < 4; ++sample)
            float x = 2.0 * (i + jitterMatrix[2*sample]) / (float)max(width, height) - 1.0;
            float y = 2.0 * (i + jitterMatrix[2*sample+1]) / (float)max(width, height) - 1.0;

            vec3 dir = normalize(vec3(x, y, -tanHalfFov) + jitter);
            Ray r = { cameraPosition, dir };

            image[width * j + i] += ComputeColor(r);

        // Get the average.
        image[width * j + i] /= 4.0;

Another possibility is to do a random jitter (instead of the matrix based one above), but you then soon enter the realm of signal processing and it takes a lot of reading to know how to choose a good noise function.

The idea remains the same though: consider the pixel to represent a tiny square area, and instead of shooting only one ray that passes through the center of the pixel, shoot many rays covering the entire pixel area. The more dense the ray distribution is, the better signal you get.

P.S.: I wrote the code above on the fly, so I'd expect a few errors in it. It's only meant to show the basic idea.

  • $\begingroup$ Great Answer! What would be the benefits of using this method opposed to the method @Raxvan used? Will I get the same results by rendering to a large size and then downscaling to a smaller size? $\endgroup$ Commented Nov 10, 2016 at 15:28
  • $\begingroup$ Fundamentally, with ray tracing you just don't need to render a bigger image then scale it down. That means you have a lot more flexibility: you can have a lot of samples, you can vary the number of samples depending on the region, and simply, you don't have to add the rescale step. $\endgroup$ Commented Nov 10, 2016 at 15:36
  • 2
    $\begingroup$ On the topic of jittering, this turns out to be a fairly complex topic. Here is a great paper analyzing the state-of-the-art a few years back graphics.pixar.com/library/MultiJitteredSampling/paper.pdf $\endgroup$ Commented Nov 10, 2016 at 15:44
  • $\begingroup$ The code sample above uses a 4 Sample MSAA, if I wanted to do 8x MSAA what would the matrix look like then? What would I need to change in the jitter matrix shown above? $\endgroup$ Commented Dec 3, 2016 at 15:59
  • $\begingroup$ You say "4 samples MSAA would be:" Presumably you meant SSAA. MSAA, as used in rasterisation as a cost/quality tradeoff, first determines which object (read triangle) each of the samples hits. If all of them hit the same object, only one shading operation is performed. If 2 different objects are found, then 2 shading operations are performed and the results are weighted and summed by the number of samples hitting each object, and so on for 3 etc objects. What you are describing is standard super sampling AA, i,e, SSAA. $\endgroup$
    – Simon F
    Commented Mar 26, 2020 at 8:14

Just to add to the answers above:

Distributed Ray Tracing (Cook, Porter, & Carpenter). Allows you to simultaneously do spatial AA, temporal AA (i.e. motion blur), and focus/depth of field. Best to read the paper, but basically the N-rays you fire per pixel can also be assigned pseudo-random times (for motion blur) and positions on a lens (to get focus effects).

Adaptive Supersampling: You initially fire a certain number of rays per pixel. However, if the rays in a local neighbourhood return significantly different results, then you can locally increase the sampling rate (say 2x), to improve the results. You can opt to repeat the process. It's not perfect since objects can still 'fall between the samples', but it's possibly cheaper than uniformly sampling at a higher rate.

Ray tracing with cones (Amanatides): Instead of using an infinitely thin ray, each ray is instead approximated by a cone, say with a diameter that covers all of the pixel (and a little of the neighbours) allowing you to do evaluate partial coverage. It also has benefits for texture sampling/antialiasing, soft shadows, and potentially LOD of models. I did an implementation many years ago - it's certainly more difficult to do, but does avoid a lot of additional rays. IIRC there was also a scheme called pencil tracing (but my initial search didn't find a link to the paper)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.