After reading a few articles online I am can confidently say I am clueless on how Anti-Aliasing works when Ray Tracing.

All I understand is that A Single Pixel/Ray is split into 4 sub-pixels and 4 rays rather than 1.

Could somebody please explain how this is done (preferably with code)?


I think it's safe to say that there are two different ways of doing AA in raytracing:

1: if you have the final image and the depth image it is possible to apply almost all existing techniques that are used in games (FXAA, etc) Those work directly on the final image and are not related to raytracing

2: the second method is to take into account multiple rays for each pixel and then averaging the result. For a very simple version think of it like this:

  • you render first an image of size 1024x1024, one ray for each pixel (for example)
  • after rendering, you scale the image to 512x512 (each 4 pixels are avereged into one) and you can notice that the edges are smoother. This way you have effectively used 4 rays for each pixel in the final image of 512x512 size.

There are other variations on this method. For example you can adapt the number of samples for pixels that are right at the edge of the geometry meaning that for some pixels you will have only 4 samples , and for others 16.

Check the links in the comments above.

  • $\begingroup$ So basically I render an image to a large size and when saving it to an image, downscale it to a lower size? That seems quite simple :)! Is this the super sampling method? $\endgroup$ – Arjan Singh Nov 10 '16 at 14:27
  • 1
    $\begingroup$ @Arjan Singh yes it's en.wikipedia.org/wiki/Supersampling , but this is the slowest of them all , raytracing allows you to easily do adaptive supersampling, which can perform a lot better $\endgroup$ – Raxvan Nov 10 '16 at 14:54

Raxvan is completely right that "traditional" anti aliasing techniques will work in raytracing, including those that use information such as depth to do antialiasing. You could even do temporal anti aliasing in ray tracing for instance.

Julien expanded on Raxvan's 2nd item which was an explanation of super sampling, and showed how you'd actually do that, also mentioning that you can randomize the location of the samples within the pixel but then you are entering signal processing country which is a lot deeper, and it definitely is!

As Julien said, if you want to do $N$ samples per pixel, you can break the pixel up into $N$ evenly distributed sample points (on a grid basically) and average those samples.

If you do that, you can still get aliasing though. It is better than NOT doing it, because you are increasing your sampling rate, so will be able to handle higher frequency data (aka smaller details), but it can still cause aliasing.

If you instead take $N$ random samples within a pixel, you are effectively trading aliasing for noise. Noise is easier on the eyes and looks more natural than aliasing, so is the preferred result usually. I believe it's even provably the ideal situation with higher sample counts but don't have more info on that ):

When you use just "regular" random numbers like you'd get from rand() or std::uniform_int_distribution, that is called "white noise" because it contains all frequencies, like how white light is made up of all other colors (frequencies) of light.

Using white noise to randomize the samples within a pixel has the problem that sometimes your samples will clump together. For instance, if you average 100 samples in a pixel, but they ALL end up being in the upper left corner of the pixel, you aren't going to get ANY information about the other parts of the pixel, so your final resulting pixel color will be missing information about what color it should be.

A better approach is to use something called blue noise which only contains high frequency components (like how blue light is high frequency light).

The benefit of blue noise is that you get even coverage over the pixel, like you get with a uniform sampling grid, but, you still get some randomness, which turns aliasing into noise and gives you a better looking image.

Unfortunately, blue noise can be very costly to compute, and the best methods all seem to be patented (what the heck?!), but one way to do this, invented by pixar (and patented too i think but not 100% sure) is to make an even grid of sample points, then randomly offset each sample point a small amount - like a random amount between plus or minus half the width and height of the sampling grid. This way you get a sort of blue noise sampling for pretty cheap.

Note that this is a form of stratified sampling, and poisson disk sampling is a form of that too, which is also a way of generating blue noise: https://www.jasondavies.com/poisson-disc/

If you are interested in going deeper you probably will also want to check out this question and answer!

What is the fundamental reasoning for anti aliasing using multiple random samples within a pixel?

Lastly, this stuff is starting to stray into the realm of monte carlo path tracing which is the common method for doing photorealistic raytracing. if you are interested in learning more about that, give this a read!



Let's suppose a fairly typical raytracing main loop:

struct Ray
    vec3 origin;
    vec3 direction;

RGBColor* image = CreateImageBuffer(width, height);

for (int j=0; j < height; ++i)
    for (int i=0; i < width; ++i)
        float x = 2.0 * (float)i / (float)max(width, height) - 1.0;
        float y = 2.0 * (float)j / (float)max(width, height) - 1.0;

        vec3 dir = normalize(vec3(x, y, -tanHalfFov));
        Ray r = { cameraPosition, dir };

        image[width * j + i] = ComputeColor(r);

One possible modification of it to do 4 samples MSAA would be:

float jitterMatrix[4 * 2] = {
    -1.0/4.0,  3.0/4.0,
     3.0/4.0,  1.0/3.0,
    -3.0/4.0, -1.0/4.0,
     1.0/4.0, -3.0/4.0,

for (int j=0; j < height; ++i)
    for (int i=0; i < width; ++i)
        // Init the pixel to 100% black (no light).
        image[width * j + i] = RGBColor(0.0);

        // Accumulate light for N samples.
        for (int sample = 0; sample < 4; ++sample)
            float x = 2.0 * (i + jitterMatrix[2*sample]) / (float)max(width, height) - 1.0;
            float y = 2.0 * (i + jitterMatrix[2*sample+1]) / (float)max(width, height) - 1.0;

            vec3 dir = normalize(vec3(x, y, -tanHalfFov) + jitter);
            Ray r = { cameraPosition, dir };

            image[width * j + i] += ComputeColor(r);

        // Get the average.
        image[width * j + i] /= 4.0;

Another possibility is to do a random jitter (instead of the matrix based one above), but you then soon enter the realm of signal processing and it takes a lot of reading to know how to choose a good noise function.

The idea remains the same though: consider the pixel to represent a tiny square area, and instead of shooting only one ray that passes through the center of the pixel, shoot many rays covering the entire pixel area. The more dense the ray distribution is, the better signal you get.

P.S.: I wrote the code above on the fly, so I'd expect a few errors in it. It's only meant to show the basic idea.

  • $\begingroup$ Great Answer! What would be the benefits of using this method opposed to the method @Raxvan used? Will I get the same results by rendering to a large size and then downscaling to a smaller size? $\endgroup$ – Arjan Singh Nov 10 '16 at 15:28
  • $\begingroup$ Fundamentally, with ray tracing you just don't need to render a bigger image then scale it down. That means you have a lot more flexibility: you can have a lot of samples, you can vary the number of samples depending on the region, and simply, you don't have to add the rescale step. $\endgroup$ – Julien Guertault Nov 10 '16 at 15:36
  • 2
    $\begingroup$ On the topic of jittering, this turns out to be a fairly complex topic. Here is a great paper analyzing the state-of-the-art a few years back graphics.pixar.com/library/MultiJitteredSampling/paper.pdf $\endgroup$ – Mikkel Gjoel Nov 10 '16 at 15:44
  • $\begingroup$ The code sample above uses a 4 Sample MSAA, if I wanted to do 8x MSAA what would the matrix look like then? What would I need to change in the jitter matrix shown above? $\endgroup$ – Arjan Singh Dec 3 '16 at 15:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.