# How can I raytrace a scene that does not fit into memory?

If the scene to be raytraced cannot be stored in memory, then without adding more RAM to the machine it seems unrealistic to render it in a practical time span, due to the need to load different parts of the scene from disk potentially several times per pixel.

Is there any way around this? I'm trying to think of some way of performing a large number of the calculations involving a particular subset of the scene all at once, to reduce the number of times it needs to be loaded into memory. Is there any other way of improving the speed in such a case?

If the scene does not entirely fit into memory, you are entering the field of out-of-core rendering. There are essentially two approaches here: a) Generate your scene on-demand b) Load your scene on-demand

The former approach aligns well with most animation workflows, where models are heavily subdivided using e.g. Catmull-Clark and can become very memory-intensive, but the base meshes themselves easily fit into memory. Pixar have a few papers about this (e.g. Ray Differentials and Multiresolution Geometry Caching for Distribution Ray Tracing in Complex Scenes), but the gist of it is that models are only subdivided when they are hit by a ray, and only subdivided as much as is reasonable for such a ray (e.g. diffuse interreflection need less accuracy than mirror reflections). The rest is handled by a geometry cache, which keeps the subdivided models in memory and hopefully makes the process efficient by a good eviction strategy.

As long as all your base meshes comfortably fit into memory, you can easily go out-of-core and render meshes at subdivision levels that would never fit into memory. The geometry cache also scales nicely with the amount of memory you have, allowing you to weigh RAM vs. render times. This was also used in Cars I believe.

The second approach is more general and does not rely on heavy use of subdivision. Instead, it relies on the fact that your scene was most likely made by an artist and already comes partitioned into reasonably small objects that fit into memory individually. The idea is then to keep two hierarchies (kD-tree or bounding volume hierarchy): A top-level hierarchy that only stores bounding boxes of the objects in your scene, and a low-level hierarchy that stores the actual geometry. There is one such low-level hierarchy for each object.

In this approach, you ideally already store a bounding box along with each object on disk. As the scene is loaded, you only build the top-level hierarchy initially, meaning you only have to look at the bounding boxes and not the geometry. You then start tracing rays and traverse them through the hierarchy. Whenever a ray hits a leaf node in the top-level hierarchy (i.e. it hits the bounding box of an object), that object is loaded into memory and its low-level hierarchy is built. The ray then continues down into tracing that object. Combined with an object cache that keeps as much of the low-level hierarchy in memory as possible, this can perform reasonably well.

The first benefit of such an approach is that objects that are never hit are never loaded, meaning that it automatically adapts to the visibility in your scene. The second benefit is that if you are tracing many rays, you don't have to load an object immediately as it is hit by a ray; instead, you can hold that ray and wait until enough rays have hit that object, amortizing the load over multiple ray hits.

You can also combine this approach with a ray sorting algorithm such as Sorted Deferred Shading for Production Path Tracing to avoid thrashing due to incoherent rays. The mentioned paper describes the architecture of Disney's Hyperion renderer, used for Big Hero 6 I believe, so it most likely can handle scenes at production scale.

• This is super interesting! So is the Disney paper you linked. Aug 9 '15 at 23:41
• +1 So many answers to things I've always wondered about! Oct 22 '15 at 20:53

If you organize your scene in a spatial structure (the usual way being a Bounding Volume Hierarchy), you can use a sort of virtual scene (I am making up this term, in reference to virtual textures).

A memory manager would keep only a limited number of bounding boxes loaded at a time, and abstract the operation consisting in retrieving one.

This way, a box would be loaded only as needed: when a ray hits a bounding box, the box get loaded to resolve the collision. Later on when another box needs to be loaded, the unused one is deleted to make room for the new one.

With all these boxes getting loaded and deleted, the ray coherency would be a major factor in speed. I suppose a further improvement could be to defer loading, by reordering rays to treat first the boxes that are already loaded.

• Yes something like this. Aug 9 '15 at 11:40

What you do is you load triangles into memory from disk based on what has been hit previously. You can begin with triangles in close proximity first. The reasoning is that in one area the rays are likely to hit same triangles repeatedly. And eventually you will be somewhat efficient. (For this reason it is a good idea to cache last hit triangle in occlusion tracing that dont care of order)

Second you store the triangles in a spatial tree that allows you to do quick searching from the disk, to renew what portions you are having in memory by proximity. So load only branches that will be in the way of the ray. If its some kind of voxel tree, like a octree you can even sort secondary rays and solve them by coherence. A BSP tree is also somewhat good at pruning areas.

There are cases where this fails but its reasonably efficient in most scene buckets if your not rendering noise...

The approaches mentioned by the 3 answers up to now (May 2020) mention:

• subdivision surfaces
• BVH per object tracing/eviction + ray cache + memory management

There are other orthogonal techniques that can be added or used by themselves. That is decimation based on something else than OpenSubdiv. Like:

A different angle of thought: if your scene doesn't fit in memory, it has an entropy problem. You only have 4k res (~8 Million pixels?), therefore in the sense of Shannon "more than memory" cannot reasonably map to less data than can be projected to these 8Mp. Which means a scene that uses "more than memory" is most likely a super overkill. Thus again, the simplification approach will probably not only speed up things, but also could help aliasing.

• I strongly disagree with your interpretation of information theory. It is true, that in the end you will need #pixels * #channels * #bits_per_channel but you don't want to guess that data, you want to compute them by evaluating the rendering equation. Note that you have many free variables like camera and light parameters, etc. You don't want to pre-bake an optimized scene for each possible camera position or light setup! For example that would mean to precompute all textures for all possible perspectives. May 25 '20 at 9:37
• Another wrong assumption of yours is perhaps that only the visible objects contribute to the final image. However, there may be shadow casters all around, layers of transparent objects that accumulate together, specular highlights and caustics from behind the camera reflecting towards object in front of the camera and possibly back again. In order to compute all the light bounces correctly, you will easily end up with scenes much more information content (orders of magnitude more) than the resulting image. May 25 '20 at 9:41
• The last bit of additional information you need to take into account is that if you use a spectral renderer, you will try to have more than just RGB data for your materials, but the rendering output will be in most cases just RGB. May 25 '20 at 9:50
• @Isolin Your first remark I would argue, yes we do. That's what we do with mipmaps. When we prepare any sort of map, especially impostors, we do pre-bake them for any sort of camera distance or angle. Any sort of LOD of meshes is exactly that, it's a pre-baked optimization for each possible camera position. Now your second remark is interesting. It made me ponder about the universe. We can say that the faint GI caused by galaxies contribute to the final image. Though I doubt that it's chaotic; meaning they can be reduced to few parameters. But caustics are. good catch May 25 '20 at 10:44