My path-tracer is generally fine and gives good output for regular test scenes, but I've noticed dense fireflies and dark speckles over fractal surfaces. The noise disappears if I render without light gathers (or zero every gather) and definitely isn't driven by hidden NaNs, so I suspect it's happening because fractals are naturally "rough" while my occlusion tests assume smooth surfaces (with constant shadow-ray offsets).
Would it be possible to evaluate local distance variance at each intersection and scale my shadow-ray offsets appropriately? Would it have a significant effect if so, or would I be better off writing a denoiser?
A few different test images with constant shadow offset; explicit light sampling is always performed with MIS (one BXDF sample, one spatial sample).