How to get clean plate from moving footage?

Question

I'm not sure if this question is on-topic or not, based on the tour. Feel free to point me elsewhere.

The short version of the question is: what's the typical technique for generating the clean-plate for footage in which the camera moves?

More detail:

It's easy to understand how computer graphics are superimposed over live footage to cover existing elements (like an actor in green pajamas). For example, it seems easy for the CG version of the Hulk to completely cover Mark Ruffalo without having to replace anything real that wasn't already visible in the shot.

It's also easy to understand where this comes from for footage where the camera doesn't move. With a locked-off shot it's not hard to get a shot without any of the actors/props or whatever is being replaced (a clean-plate, as it were), and match that to the real take.

But I'm baffled about non-locked-off shots. A good example might be a scene from Star Trek: First Contact when the Borg Queen is introduced. The upper half of her body is lowered onto a lower-half kind of rig. Whether the actress is actually lowered or not, there is background behind either her invisible real body, as she's lowered, or behind her invisible upper body as it's being rotoscoped down into position.

Either way, they had to have something to show behind the parts of the actress that weren't supposed to be there in the final footage.

One theory I've imagined is that they run the camera through the shot first without the actors, and then repeat the shot with the actors, but this seems prohibitively prone to imprecision.

Another theory is that they get the clean plate from another frame in the same footage, but this seems like it'd only work in very specific cases.

I've also seen tutorials in which the trick is that the shot IS locked-off, and the camera shake is added in after the composite. But again, this only works for certain kinds of shots, and I have a hard time believing that this is the way it's done for everything.

What is/are the typical technique(s) for generating a clean plate for footage in which the camera moves?

Answer 1

There are a couple of techniques:

1. A motion controlled camera - The camera is on some sort of rig connected to a computer. The computer is programmed to move the camera in a particular way. The camera runs through the moves without the actors, generating a clean set of plates. Then it is run again with the actors, usually wearing green or blue costumes or parts of costumes. For example, in "Forest Gump," one character loses his legs in a war. The actor had working legs, as shown early in the movie, and in some later scenes wore blue stockings that were composited out. (They also used other techniques, such as simply hiding his legs under various things in some scenes. Whatever works!)
2. As you surmise, sometimes a larger scene is filmed with a locked-off camera to get a clean plate, then during editing, they zoom into the scene and pan around in software to create the motion you see on screen. This doesn't work well for anything that involves the camera rotating, but can work for scenes where the camera trucks.
3. Parts of the scenery that the actors are in front of are blocked off with green screens and recreated in post. I've seen outtakes of a few pieces where they did this. If you have the DVD of "The Others" with Nicole Kidman, the "DVD Extras" show how fog was added to scenes that were filmed without it. It basically involved someone walking around behind her holding a large green board. Then, they recreated the scenery in the computer, added fog to it, and replaced the green screen with the foggy scene. I have a vague recollection of them doing something similar in the HBO mini-series John Adams. This would likely involve cameras that record their motion. The motion is then imported into the computer and used to move the virtual camera around the scene they've recreated.

Answer 2

Sometimes FX studios will use projection painting. The technique basically tracks the footage onto basic geometry and use different frames for different areas of the scene(an actor may be in front of a section of the background, but isn't in another frame.)

CGMatter has a great tutorial on it: https://www.youtube.com/watch?v=gz4qV5_zuMo&t=202s