Storing 3D animations for ray tracing

Question

I've just finished writing a path tracer that takes OBJ files and renders it. This is convenient for still images, but my end goal was to export a sequence from a 3D application (Blender was what I had in mind), and then render an image sequence from that. Originally I figured an OBJ sequence would do, but that would take a massive amount of space for complex scenes.

I began looking into parsing .fbx, .3ds and other popular file formats, but there's so little out there that I could find. I still don't really understand how those files store animations. I can imagine the mesh is stored and then keyframes indicate which vertices translate...? Maybe? Or whole objects? What happens when you have complex deformation like an ocean wave, fluid, or character where many, many vertices are translating? At that point would it be easier to do some sort of OBJ like sequence? I want to stay away from that if I can since it takes lots of memory/time to export.

Hopefully someone can point me in the right direction or explain how other standalone renderers keep their animation data. Thank you!

Answer 1

I think that it is important to classify between different types of animations here. One rough classification could be

1. Rigid body animations
2. Vertex skinning animations
3. Others...

Rigid body animations

This refers to animations where the actual geometry data does not change. For example, you could have a geometric object, like a unit cube, and apply (usually affine) transformations to it: You can rotate, translate or scale this object. This is usually accomplished by some sort of scene graph: Each node of this scene graph can have information about the rotation, translation and scaling that should be applied to the objects. In general, this is something that can be expressed with a 4x4 transformation matrix.

The key point is that you don't have to store "animated geometry data". You only have to store the geometry data once, in its original state. In addition to that, you store the key frames of the transformation.

In your case, if you wanted to do a rigid body animation, you could store the geometry in an OBJ file, and additionally store the key frame information in an arbitrary form, for example

TIME  TRANSLATION
0.0   ( 0, 0, 0)
0.1   (10, 0, 0)
0.2   (10,10, 0)
0.3   ( 0,10, 0)
0.4   ( 0, 0, 0)

This could describe a movement of the object along a rectangular path, taking 0.4 seconds. Fortunately, you don't have to create your own file format for that. There already are solutions for storing this sort of animation data. For example, the COLLADA format already supports animations like this. A more compact representation of such an animation can be stored in a glTF asset. For example, it can represent animations like this one, from the glTF sample models repository:

Vertex skinning animations

This technique refers, roughly speaking, to cases where you have an "initial" geometry, and you can define "rules" of how this geometry should be deformed based on the pose of a skeleton. As the name suggests, this is mainly intended for character animations, where you define a skeleton consisting of the "bones" of the character, and the geometry defines the "skin" of the character. Vertex skinning is also supported by different file formats like COLLADA or glTF. The effect is shown in this image, again, from the glTF sample models:

As already mentioned by joojaa in his answer, implementing vertex skinning according to the specifications of these file formats may be a bit tricky. Additionally, a considerable amount of the skinning computations is usually done by the vertex shader. So for a ray-tracer, you would have to move these computations into your application code.

Others...

This refers to what you already alluded to in the question: There may be cases where the animation is not a simple transformation of rigid objects, and where you cannot define any "rules" of how the vertices should be transformed. For example, when the vertex positions are computed by a simulator, like a fluid simulator or a soft body simulator.

In doubt, it may simply be necessary to actually store the positions of all vertices, frame for frame. You could achieve this as a sequence of OBJ files. But note that OBJ files contain ASCII data, and thus, are usually large and the geometry data has to be parsed before it can be used. Storing the data in a binary form that can be used by the application directly may be beneficial here.

There are some possible approaches for reducing the amount of data that has to be stored. For example, one could extend the concept of "key frames" here. For example, one could store the initial configuration of al vertices, and then only store the movement of vertices that acutally have changed between two frames. But of course, if (nearly) all vertices changed, the savings may be diminishing.

I'm not aware of any universally applicable, "standardized" solutions for this. Maybe some of the solutions that joojaa referred to in his answer could be helpful here.

Answer 2

Pixar spent quite much time, money and other capital to archive all their renders. What they did was just to store the raw render description files their render engine used. The reason for this is that it's the only way to ensure that the features stay intact. Of course they had a very well documented standardized format to work with.

The benefit of this method is that it stores things relevantly. In other words static objects can be stored static and only animation objects need to be stored frame by frame which is the only way to ensure compatibility for animation without relying on host apps to do this for you.

Alembic works on a similar principle. The downside is big formats. But note that formats such as FBX and Collada (*.dae) are not much better. While it's true that they can contain some of the animation data they cannot always do everything that you want them to do. Which means that in the end it might not do what you want.

The animation exchange formats (FBX, Collada etc...) do not usually store vertex positions, if they did then they would just have same problem as your obj storage. Instead they store the modification history so that they can redo the animation. So for example the bones are stored as transforms and the vertex weights are used to compute their position. The downside is that your reader needs to implement substantial amounts of the animation core. And anything the format does not know how to do is stored either as point caches, or not at all. Realistically speaking for FBX you would use the API to do this. Implementing it yourself is a quite big undertaking, it may be cheaper to just buy more storage space.

Personally I would implement something like the RI spec for my own renderer. If not RI spec itself which would allow you to swap renderer if need be (also the other way around a user could swap to your renderer without many changes).