Algorithms to Remove High Frequency Noise from Path Tracing

Question

I have been working on my own renderer for a while, and I'm wondering if there's any way to remove the Monte Carlo noise from the rendered image, besides waiting for a long time for it to converge?

The way I found is to blur the image, which is not really helpful, since it reduces the quality/sharpness of the image a lot. And I can achieve the same thing by rendering a small image with more samples, then scaling it up.

Is there any algorithm designed to deal with noise in the image in path tracing?

Answer 1

There are, and I am looking forward to seeing the specifics of other answers, but one way to deal with this is to not have the noise (or as much noise) in the source data to begin with.

The noise is coming from the fact that there is high variance in the rendering - the number of samples you've taken haven't converged enough to the actual right answer of the integral, and so some pixels are too high/bright and some are too low/dim (in each color channel).

The problem is this: If you use white noise random numbers to do your sampling, you may get samples clumping together like the image below. Given enough samples, it will converge, but it will take a while before it gives good coverage over the sampling space. Find a region of empty space in the image below (like in the lower right) and imagine that there was a small, bright light there and that the scene was dark everywhere else. You can see how not having any samples there is going to make a problem for rendering.

Alternately, you could sample at even intervals like the below, but that will give you aliasing artifacts instead of noise, which is worse.

One idea is to use low discrepancy sequences and do quasi monte carlo integration (https://en.wikipedia.org/wiki/Quasi-Monte_Carlo_method). Low discrepancy sequences are related to blue noise, which has only high frequency components. Going these routes, you get faster convergence of $O(1/N)$ instead of $O(\sqrt{N})$. These give better coverage of the sample space, but since there is some randomness (or random like qualities) to them, they don't have the aliasing issues that regularly spaced sampling does.

Here is a "jittered grid" where you sample on a grid, but use small random offsets within a cell size. This was invented by pixar and was under patent for a while but is no longer:

Here is a common low discrepancy sequence called the Halton sequence (basically a 2d version of Van Der Corpus)

And here is a poisson disc sampling, using Mitchel's best candidate algorithm:

More information, including the source code that generated these images can be found here: https://blog.demofox.org/2017/05/29/when-random-numbers-are-too-random-low-discrepancy-sequences/

Answer 2

One technique you could use is break the image into blocks and measure each blocks variance - this way you can apply more samples to blocks with higher variance.

The variance can be estimated by using 2 accumulation buffers instead of 1. You render each pass into an alternate buffer. The absolute difference between these buffers (with respect to each block) is proportional to variance. Upon presentation to screen you can add the two buffers together to get your full accumulation buffer back.