Changing colors after raytracing

Question

Intro

Maxwell Render has a feature called multilight, where one can ray-trace a scene with multiple light sources, and change the brightness of the individual light sources afterwards. Luxrender has a similar feature called light groups.

I image that behind the scenes, these renderers keep track of the contributions of the different light sources separately, and then composing a final image (given brightnesses of light sources) just amounts to mixing these per-light contributions with appropriate coefficients. Effectively, instead of keeping track of the color of a ray as a tuple (R,G,B), one keeps track of more numbers (R1, G1, B1, R2, G2, B2, ..., Rk, Gk, Bk) if there are k light sources (let's assume no spectral rendering, just for ease of notation -- it's conceptually identical).

From a mathematical point of view, the final image is a linear combination of the images resulting from the individual light sources.

---

Question

Now it seems interesting to have a similar feature that allows changing the colors of objects in the scene after rendering. This can be done with an entirely similar technique: suppose that we have two objects whose color we want to be able to change after rendering. The final image is a polynomial in the color of these objects, with two variables (one for each object) and of degree n, where n is the maximum number of bounces for a ray.

Specifically, instead of keeping track of tuples (R,G,B), the integrator would keep track of expressions of the form (R00,G00,B00) + (R10,G10,B10)x + (R01,G01,B01)y + (R11,G11,B11)xy + (R21,G21,B21)x²y + ..., for a total of k^n terms, where k is the number of colors you want to be able to change, and n the maximum number of bounces. For modest k and n, this certainly seems tractable. Color calculations become more expensive, as does storing the resulting "polynomial-image", but all the geometric calculations are identical.

Is this feature implemented in any widely available renderers?

• If so, which ones?

• If not, why not? Is this not something that's useful in practice? Is it not computationally feasible?

---

Remark

I hope I've made myself clear in my explanation. I don't know much about computer graphics (mathematician by training), so my notation may be unusual. More general that just changing the colors of objects, one could change more properties of materials after rendering. For example, the transparency of objects could be changed afterwards (though assuming importance sampling is used by the integrator, this comes at the cost of noise, I guess).

Answer 1

What you describe is essentially a deep raster. Most modern renderers implement this in fact it is implemented in several ways. There are even sopecific technical positions that rely on this technology in form of lighting, color grading and compositioning artists

The first technique is multipass rendering which is basically just this dumping of render data out of the renderer for fast defered rendering. All rendering engines on the planet can do this, though all do not implement dumping it out of one render, most do however). What you then do is you integrate these dumped buffers in a compositioning app (Nuke, Fusion, maybe even after effects, you can even do this in photoshop) where you can do final tweaks and changes. This is a very old idea and has been used for more than 30 years. It has certain drawbacks bit overall this means you night be able to pull off even calculation changes that would be too costly to do with a re-render. So color of objects

Many systems also support interactive sessions that cache the computations, in what is essentially a deep raster. This way its possible to recalculate not only color but even the position, decay of individual lights at interactive speeds. The most modern ones of these are frankly quite sophisticated and can also issue the renderer to rerender in background. Animation studios have lots of people working on these to make them better.

You can even use this trick to change camera position for the second stereo camera without recomputing your scene.