4
$\begingroup$

(As a follow-up to this question)

I'm trying to overlay a a green-screen plate over a 3D background.

I create the key with a Keying node and the overlay with an Alpha Over node.

I can get a clean key by enabling Convert to Premultiplied.

enter image description here

Whereas if I leave the Convert to Premultiplied off, the key does not work.

enter image description here

The reason for this, as explained on this anwser:

The Alpha over node actually assumes associated alpha and performs the correct over operation, which is FG+(1-FG.a)BG.RGB.

Unassociated alpha can do none of these things.

[...] it is prudent to manually associate your alpha and control the merging to prevent data loss in the RGB channels. I still stand by the advice, however; always try to maintain a consistent state of alpha, and when you deviate, do so under scrutiny.

So the question is: what is the correct way to associate the Alpha Channel crated by the keying node, so that it works with the Alpha over node without resorting to the convert to premultiply option?

enter image description here

green screen element can be downloaded from the mango project https://download.blender.org/mango/first-footage.zip

$\endgroup$
  • $\begingroup$ What's the file extension of the input image ? $\endgroup$ – Mr Zak Feb 4 '16 at 18:18
  • $\begingroup$ @ Mr Zak its an EXR file $\endgroup$ – cegaton Feb 4 '16 at 18:19
3
$\begingroup$

There are potentially a few issues with either "Convert Premul", converting alpha, or a simple multiply across the channels, that may not be entirely obvious at first. It is a point of confusion in that that no node should deliver unassociated alpha as the entire compositor assumes associated values. Sadly the node used here, does.

One issue, with particular attention to pulling a key, is that your edges will likely not be of the correct luminance value to blend seamlessly. That is, when a subject is shot against a green screen, the alpha channel that is produced would be a ratio of the foreground against the background, which in this case is some non-black value of the screen used. The resulting output ratio isn't going to be the same as an object set against non-emitting pixels. Those semi-transparents can be extremely tricky to get the ratios correct on without having a good grasp on the behind-the-scenes models.

The second issue would be with associating is that you can destroy RGB information. As per the OpenEXR Technical Introduction, alpha with a greater ratio RGB is entirely valid and represents emission with a lower degree of occlusion. The problem is that if you craft this in an RGB image, zero alpha will destroy this information on the multiply.

Calling the color channels “premultiplied” does not mean that the color values in an image have actually been multiplied by alpha at some point during the creation of the image, or that pixels with zero alpha and non-zero color channels are illegal. Non-zero color with zero alpha is legal; such a pixel represents an object that emits light even though it is completely transparent, for example, a candle flame or a lens flare.

In the visual effects industry premultiplied color channels are the norm, and application software packages typically use internal image representations that are also premultiplied.

To work around this, a carefully crafted RGB against zero emission RGB values can be associated manually.

Once you have a representation of your emission levels in RGB, which is effectively RGB values against black, you can use whatever technique you desire to assign the alpha channel. For example, if you have a blue glow with low or zero occlusion (alpha), you would want the equivalent occlusion values properly set into your base. "Set Alpha" will assign the alpha directly to the alpha channel, and the result should be correct[1].

TL;DR in your sample, "Convert Premul", using the alpha conversion node, or strictly a simple multiply will all work as expected, with a rather large caveat being the first issue above. You would ideally want your image to represent emission on the semi-occluded pixels, which as any keyer would tell you, is quite a dance of artistry. Using that canned After-Effects-Esque "Keying" nightmare node won't help either, as it assumes a single luminance / RGB value to compare against across the whole green screen, which is never the case, and will result in whack or rather cheated values, which will impact your final composite. An alternative that might get you closer is to inpaint and use a distance node on the RGB values, which will at least pull data from the fully non-occluded plate elements and push them into the semi-transparent regions. This will result in a luminance difference that is closer to what was behind the actual pixels in question.

[1] Note that there are largely two primary methods to do this. One is probabilistic, aka A+B-AB and the other is non-overlapping, which is A+B. If you are certain your geometry doesn't overlap for the pixel in question, the latter is correct. If your geometry overlaps, such as a transparent window atop of another transparent window, the former is correct.

$\endgroup$
  • $\begingroup$ "inpaint and use a distance node on the RGB values" can you please show an example of how to do this? $\endgroup$ – cegaton Feb 4 '16 at 18:42
  • $\begingroup$ I realized I left this comment hanging. Basically you would want to have an image that approximates what is behind your semi-transparent regions. Inpaint can do a decent job of this. Once you have an inpainted version of your plate, you can feed that and the original to a distance node to calculate the distance between the RGB background and foreground. That distance ends up being your transparency ratio to combine with your fully opaque regions. $\endgroup$ – troy_s Jul 3 '16 at 4:19
0
$\begingroup$

Use Add>Convertor>Alpha Convert set to Straight to Premul to easily and quickly multiply the R, G, and B components of the image by the alpha. This should give you an image with the associated alpha that alpha over expects.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.