I'm looking at adding SVGF de-noising to my path tracer; currently I'm using plain a-trous but it can't really handle areas like soft shadows well. I've been looking at the SVGF paper https://research.nvidia.com/sites/default/files/pubs/2017-07_Spatiotemporal-Variance-Guided-Filtering%3A//svgf_preprint.pdf but there seem to be a lot of things where the paper assumes the reader already knows what they are talking about and details are missing. I'm currently skipping motion vectors, reprojection, and history rejection; I'm aiming to get a decent image if things don't move.
Here's the most confusing parts:
- Variance estimation; the paper asks for the raw moments of colour luminance. Wikipedia says this is just the mean and standard deviation of the image. Presumably this should not be the entire image, but some small subset like 5x5 or 7x7, but the paper doesn't exactly say. The discussion of low history implies 7x7? To follow on, it's discussed as a temporal variance with a temporal history, and a moment history is mentioned as being saved, but there doesn't seem to be a direct usage of this history? Is it meant to have the moving average applied similar to the illumination? When calculating the variance, this should presumably be the illumination only and not including albedo? The paper also mentions that the moments should be integrated but it's not clear what that means.
- Looking at the description of the a-trous involved, it seems that we are essentially running two a-trous simultaneously, one on the variance and one on the illumination? I'm not clear if I should do the illumination first, using the unfiltered variance, or do the variance first, and then use that for the first illumination a-trous.
- When discussing depth weights, $\nabla z$ is "the gradient of clip-space depth with respect to screen-space co-ordinates". I'm not exactly clear on how to calculate this as the examples I found use known functions, rather than from samples. I guess something like the sum of the vectors to the 8 surrounding pixels multiplied by the change in depth for those pixels?
- Looking at the luminance edge-stopping function, the note states that the Gaussian pre-filter is only used for luminance edge-stopping, and not variance. Does this mean that when running a-trous for variance, $w_l$ should be computed simply without the Gaussian pre-filter, or that it should be skipped entirely? $w(p, q)$ is not defined separately for variance. Should the luminance be an actual luminance calculation e.g. $0.299*R + 0.587*G + 0.114*B$, or is this meant to be another term for illumination?
- The paper discusses using a rasterisation pass to fill in the data in the G-buffer. This seems unnecessary if the path tracer can write the G-buffer itself? The path tracer already accesses all these properties.
So the basic idea should be?
- Run path tracer to compute albedo, illumination, depth, and normal
- Perform exponential moving average on illumination
- Calculate moments from 7x7 kernel of illumination, apply moving average to them, save the new moments, then calculate variance from that
- Perform one a-trous iteration on illumination and variance (so illumination a-trous uses unfiltered variance for first pass), and save the colour for next frame use in moving average
- Perform four more a-trous iterations
- Add in pixel albedo to "finalise" the image ready for any other post-processing steps
Any clarification on the less-than-obvious parts of the paper would be appreciated.
