3
$\begingroup$

everyone. I have a piece of footage that I have motion-tracked fairly successfully, solved the camera motion, and made it into a 3D scene with that little button on the left of the motion-tracking layout. I then modeled a basic spaceship-thingy and animated it landing, with flames (specifically, the fire shader from Simon Thommes) emitting light onto the ground plane (which is on the mask layer).

I would like to add the light from the flames into the footage, but it does not show up. I do see that there is no DiffuseIndirect pass in the compositing setup but I do not know how to mix that pass in and, if I can, whether the light from the HDRI will mess it up. Is there a way to add in emission so that the render will look more realistic?

Thank you for your time,

Daniel

$\endgroup$
  • $\begingroup$ Not entirely sure where your particular problem is, but here are some small pieces of advice if you are trying to take the work in this direction. First, if you can, use the camera’s log encoded footage and take it to scene referred for the compositing. Second, specifically regarding emissions, simply use alpha. That is, an emissive-yet-non-occluding value is an RGBA set with zero alpha. It ends up a simple add. Blender is quite broken for many file formats, but EXRs should work here. $\endgroup$ – troy_s May 13 at 15:26
5
$\begingroup$

A method to get believable light interaction with the environment, is to recreate the environment.

result

I have chosen this template image. The process will be the same for moving, tracked footage, but the tracking must be very accurate.

I chose this background image by Hans Weingartz - Self-photographed, CC BY-SA 2.0 de. Click to enlarge.
background template

I have recreated the environment with very basic shapes. This result in the following layer setup.
layers

  • Layer 1: Cg object, which is inserted.
  • Layer 2: Recreated environment with computer generated light. Try to match the reference as close as possible. I used some simple diffuse, glossy and emission shaders, but you can always try to extract color information from your footage. Especially with high contrast areas like checkboard patterns or dark ridges, you should take great care to achieve similiar color values through your artificial lighting of the recreated environment.
  • Layer 3: The environment object, but this time with the camera image mapped onto it for maximum accuracy. The background image is just applied as the emission color and this object will be used for the lighting and reflections of the cg object on layer 1.

Now, I create three Render Layers.
render layers

  • environment_clean This is just the recreated environment with virtual lighting on layer 2. Use the exclude layers to exclude the cg model from layer 1 and also the reflection object from layer 3.
  • environment_shaded This is the same environment as above (layer 2), but this time without excluding the cg model. Although the model is not visible (layer 1 is switched off), it is not excluded and will therefore influence the lighting/shading of the environment on layer 2. Exclude layer 3, since the duplicated environment is still not relevant.
  • cg model This is the render layer which will render the model on layer 1. The recreated environment with virtual lighting is excluded. The environment with the camera mapped footage on layer 3 will not show up (since it is not checked in the render layer), but will influence the lighting of the model, especially visible in reflections.

Illuminating the background footage

Since we have rendered a virtually lit environment with and without influence of the cg model, we can create the difference between the lighting. We will then apply this difference to the background footage.

subtract shading from clean environment

Subtract the environment_shaded from the environment_clean render layer in compositing nodes. It is important the you uncheck clamp, since shadows are going to be values below 0.0 (which we don't want to cut off) and lights are going to be values above 0.0. The subtracted result is the difference in lighting between our environment and the environment influenced by the model. Add this to the background footage. The background image will now appear as if illuminated and shaded by the cg model.

Note, that in the areas, where there are dark colors in the original live action footage, but light color in our virtual environment, the added lighting looks very unnatural.
unnatural lighting
While shadowy areas would naturally receive light, objects with dark colors won't get equally lighter as object with brighter colors. A quick and easy fix is to only add the light/shadow to already bright areas. We can achieve this by using the original image (modified by a colorramp) as the Fac value for the addition.
more natural lighting by reducing the cahgne in dark areas
Obviously a more conving image is accomplished by modeling and shading the virtual environment more accurately. (Or combined both.)

Add the cg model (render layer) on top of this modified version of the background using a mix or alpha over node.

alpha over the cg model

Here is the complete node tree.

enter image description here

Render Time vs Workload

The rendering process will obviously take much longer than a simple shadow catcher or other tricks, because it uses three different layers. The two environment layers will take a lot of render time because their noise will accumulate in the subtraction and they usually need a high sample count for the indirect lighting. The setup will however easily produce a realistic render result in little work time.

$\endgroup$
  • $\begingroup$ It worked! The one thing I would add is a picture that shows the entire node setup. Thank you for your help! $\endgroup$ – The Blender Bender May 15 at 20:07
  • $\begingroup$ @The Added the image. $\endgroup$ – Leander May 16 at 17:12
  • $\begingroup$ One last thing. Under the cg_model render layer settings, set layer 2 as a Mask layer. This makes the cg models disappear if they go behind the models of the environment. (Or was this covered in the mask in the node setup? I didn't have any but setting layer 2 as a mask did the trick for me.) $\endgroup$ – The Blender Bender May 23 at 21:38
  • $\begingroup$ @The Great idea, however, I find it easier to mask out the foreground objects of the footage, than to perfectly replicate the environment. My tracking is usually not subpixelperfect. Feel free to edit the answer to add this info. $\endgroup$ – Leander May 23 at 22:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.