4
$\begingroup$

(As a follow-up to this question)

I'm trying to overlay a a green-screen plate over a 3D background.

I create the key with a Keying node and the overlay with an Alpha Over node.

I can get a clean key by enabling Convert to Premultiplied.

enter image description here

Whereas if I leave the Convert to Premultiplied off, the key does not work.

enter image description here

The reason for this, as explained on this anwser:

The Alpha over node actually assumes associated alpha and performs the correct over operation, which is FG+(1-FG.a)BG.RGB.

Unassociated alpha can do none of these things.

[...] it is prudent to manually associate your alpha and control the merging to prevent data loss in the RGB channels. I still stand by the advice, however; always try to maintain a consistent state of alpha, and when you deviate, do so under scrutiny.

So the question is: what is the correct way to associate the Alpha Channel crated by the keying node, so that it works with the Alpha over node without resorting to the convert to premultiply option?

enter image description here

green screen element can be downloaded from the mango project https://download.blender.org/mango/first-footage.zip

$\endgroup$
2
  • $\begingroup$ What's the file extension of the input image ? $\endgroup$
    – Mr Zak
    Commented Feb 4, 2016 at 18:18
  • $\begingroup$ @ Mr Zak its an EXR file $\endgroup$
    – user1853
    Commented Feb 4, 2016 at 18:19

3 Answers 3

3
$\begingroup$

There are potentially a few issues with either "Convert Premul", converting alpha, or a simple multiply across the channels, that may not be entirely obvious at first. It is a point of confusion in that that no node should deliver unassociated alpha as the entire compositor assumes associated values. Sadly the node used here, does.

One issue, with particular attention to pulling a key, is that your edges will likely not be of the correct luminance value to blend seamlessly. That is, when a subject is shot against a green screen, the alpha channel that is produced would be a ratio of the foreground against the background, which in this case is some non-black value of the screen used. The resulting output ratio isn't going to be the same as an object set against non-emitting pixels. Those semi-transparents can be extremely tricky to get the ratios correct on without having a good grasp on the behind-the-scenes models.

The second issue would be with associating is that you can destroy RGB information. As per the OpenEXR Technical Introduction, alpha with a greater ratio RGB is entirely valid and represents emission with a lower degree of occlusion. The problem is that if you craft this in an RGB image, zero alpha will destroy this information on the multiply.

Calling the color channels “premultiplied” does not mean that the color values in an image have actually been multiplied by alpha at some point during the creation of the image, or that pixels with zero alpha and non-zero color channels are illegal. Non-zero color with zero alpha is legal; such a pixel represents an object that emits light even though it is completely transparent, for example, a candle flame or a lens flare.

In the visual effects industry premultiplied color channels are the norm, and application software packages typically use internal image representations that are also premultiplied.

To work around this, a carefully crafted RGB against zero emission RGB values can be associated manually.

Once you have a representation of your emission levels in RGB, which is effectively RGB values against black, you can use whatever technique you desire to assign the alpha channel. For example, if you have a blue glow with low or zero occlusion (alpha), you would want the equivalent occlusion values properly set into your base. "Set Alpha" will assign the alpha directly to the alpha channel, and the result should be correct[1].

TL;DR in your sample, "Convert Premul", using the alpha conversion node, or strictly a simple multiply will all work as expected, with a rather large caveat being the first issue above. You would ideally want your image to represent emission on the semi-occluded pixels, which as any keyer would tell you, is quite a dance of artistry. Using that canned After-Effects-Esque "Keying" nightmare node won't help either, as it assumes a single luminance / RGB value to compare against across the whole green screen, which is never the case, and will result in whack or rather cheated values, which will impact your final composite. An alternative that might get you closer is to inpaint and use a distance node on the RGB values, which will at least pull data from the fully non-occluded plate elements and push them into the semi-transparent regions. This will result in a luminance difference that is closer to what was behind the actual pixels in question.

[1] Note that there are largely two primary methods to do this. One is probabilistic, aka A+B-AB and the other is non-overlapping, which is A+B. If you are certain your geometry doesn't overlap for the pixel in question, the latter is correct. If your geometry overlaps, such as a transparent window atop of another transparent window, the former is correct.

$\endgroup$
2
  • $\begingroup$ "inpaint and use a distance node on the RGB values" can you please show an example of how to do this? $\endgroup$
    – user1853
    Commented Feb 4, 2016 at 18:42
  • $\begingroup$ I realized I left this comment hanging. Basically you would want to have an image that approximates what is behind your semi-transparent regions. Inpaint can do a decent job of this. Once you have an inpainted version of your plate, you can feed that and the original to a distance node to calculate the distance between the RGB background and foreground. That distance ends up being your transparency ratio to combine with your fully opaque regions. $\endgroup$
    – troy_s
    Commented Jul 3, 2016 at 4:19
1
$\begingroup$

This has been fixed on blender version 2.92.

The output from the keying node now outputs associated alpha, as it should be.

As explained here: https://developer.blender.org/rBf68c3d557aa

Compositor: Ensure keying node result is pre-multiplied

Historically the result of the keying node was violating alpha pre-multiplication rules in Blender: it was simply overriding the alpha channel of input.

This change makes it so keying node mixes alpha into the input, which solves the following issues:

The result is properly pre-multiplied, no need in separate alpha-convert node anymore. Allows to more easily stack keying nodes. This usecase was never really investigated, but since previously alpha is always overwritten it was never possible to easily stack nodes. Now it is at something to be tried. Unfortunately, this breaks compatibility with existing files, where alpha-convert node is to be manually removed.

$\endgroup$
0
$\begingroup$

Use Add>Convertor>Alpha Convert set to Straight to Premul to easily and quickly multiply the R, G, and B components of the image by the alpha. This should give you an image with the associated alpha that alpha over expects.

$\endgroup$

You must log in to answer this question.