Algorithms to Remove High Frequency Noise from Path Tracing

I have been working on my own renderer for a while, and I'm wondering if there's any way to remove the Monte Carlo noise from the rendered image, besides waiting for a long time for it to converge?

The way I found is to blur the image, which is not really helpful, since it reduces the quality/sharpness of the image a lot. And I can achieve the same thing by rendering a small image with more samples, then scaling it up.

Is there any algorithm designed to deal with noise in the image in path tracing?

• Are you more interested in post processing to disguise the noise, or ways of speeding up convergence so that less noise is present? Jun 22 '17 at 23:31
• FWIW, Benedikt Bitterli recently released the following twitter.com/tunabrain/status/872174108385136640 based on his denoising paper. Jun 23 '17 at 8:58
• In postprocessing area, there is nice algorithm called bilateral filter shadertoy.com/view/4dfGDH Jun 23 '17 at 9:10

There are, and I am looking forward to seeing the specifics of other answers, but one way to deal with this is to not have the noise (or as much noise) in the source data to begin with.

The noise is coming from the fact that there is high variance in the rendering - the number of samples you've taken haven't converged enough to the actual right answer of the integral, and so some pixels are too high/bright and some are too low/dim (in each color channel).

The problem is this: If you use white noise random numbers to do your sampling, you may get samples clumping together like the image below. Given enough samples, it will converge, but it will take a while before it gives good coverage over the sampling space. Find a region of empty space in the image below (like in the lower right) and imagine that there was a small, bright light there and that the scene was dark everywhere else. You can see how not having any samples there is going to make a problem for rendering.

Alternately, you could sample at even intervals like the below, but that will give you aliasing artifacts instead of noise, which is worse.

One idea is to use low discrepancy sequences and do quasi monte carlo integration (https://en.wikipedia.org/wiki/Quasi-Monte_Carlo_method). Low discrepancy sequences are related to blue noise, which has only high frequency components. Going these routes, you get faster convergence of $O(1/N)$ instead of $O(\sqrt{N})$. These give better coverage of the sample space, but since there is some randomness (or random like qualities) to them, they don't have the aliasing issues that regularly spaced sampling does.

Here is a "jittered grid" where you sample on a grid, but use small random offsets within a cell size. This was invented by pixar and was under patent for a while but is no longer:

Here is a common low discrepancy sequence called the Halton sequence (basically a 2d version of Van Der Corpus)

And here is a poisson disc sampling, using Mitchel's best candidate algorithm:

More information, including the source code that generated these images can be found here: https://blog.demofox.org/2017/05/29/when-random-numbers-are-too-random-low-discrepancy-sequences/

One technique you could use is break the image into blocks and measure each blocks variance - this way you can apply more samples to blocks with higher variance.

The variance can be estimated by using 2 accumulation buffers instead of 1. You render each pass into an alternate buffer. The absolute difference between these buffers (with respect to each block) is proportional to variance. Upon presentation to screen you can add the two buffers together to get your full accumulation buffer back.

• How would you measure the absolute difference? Something like suming sqrt((r1-r2)^2 + (g1-g2)^2 + (b1-b2)^2) for all pixels in a block, or summing (abs(r1-r2) + abs(g1-g2) + abs(b1-b2)) for all pixels, or something more advanced? I've seen (and implemented) methods where you divide the difference in lengths between single pixel values with the sum of the same lengths in two accumulators, and keep sampling that pixel until the ratio is below some limit, but perhaps a smallish patch is better to compare? Mar 15 at 7:44
• I suppose it's subjective when it comes to how many samples is enough, with squared difference you're going to need a lot more samples to converge fireflies away but the quality will be higher. Mar 17 at 3:23
• I would suggest using luminance or some other perceptual metric too. It doesn't need be an average either (fireflies won't stand out much in a 8x8 block average), it could be the max per-pixel difference for a block. Mar 17 at 7:39