Why should I never ever use "Convert Premul" on Cycles renders?

Question

On this question, When trying to overlay images using Alpha Over for green screen work , I used premultiply Alpha, it was the only option that would give me clean key without the background bleeding over. A comment states Never use that "Convert Premul" option on Cycles renders. Ever.

Why?

Answer 1

You misread the answer, and my suggestion wasn't terribly clear. Any imager should be "pro" associated (aka crap term "premultiplied") alpha.

There is a good reason, and I will expand this answer when I find some time later.

The bottom line is that only associated alpha models a real-world correlation, where unassociated doesn't.

Only associated alpha manages to model both occlusion and emission. Glows, convolutions, motion blurs, and a plethora of other transparent / translucent effects can only be modeled with associated alpha.

Think of alpha as selecting the over operation. The Alpha over node actually assumes associated alpha and performs the correct over operation, which is FG+(1-FG.a)BG.RGB.

Unassociated alpha can do none of these things. This is at least one of the reasons that PNGs are never used in a post production pipeline as they are incapable of using associated alpha. From the EXR Technical Introduction:

Calling the color channels “premultiplied” does not mean that the color values in an image have actually been multiplied by alpha at some point during the creation of the image, or that pixels with zero alpha and non-zero color channels are illegal. Non-zero color with zero alpha is legal; such a pixel represents an object that emits light even though it is completely transparent, for example, a candle flame or a lens flare.

In the visual effects industry premultiplied color channels are the norm, and application software packages typically use internal image representations that are also premultiplied.

^{http://www.openexr.com/TechnicalIntroduction.pdf}

From the actual TIFF specification, regarding TIFFTAG_EXTRASAMPLES tag that denotes what type of samples are stored:

The difference between associated alpha and unassociated alpha is not just a matter of taste or a matter of maths.

Associated alpha is generally interpreted as true transparency information.

^{http://www.awaresystems.be/imaging/tiff/tifftags/extrasamples.html}

Again, never, ever, use unassociated (crap term "straight" or "key") alpha, and instead associate it. This is what the unfortunately named "Convert Premul" does in the Alpha Over node. However, on direct Cycles renders, it is already associated and is never needed.

There is exactly one edge case that you must unassociate alpha before processing, and that is on colour corrections. To do so, you must be careful and skip on zero alpha. An alpha value of zero with nonzero RGB is entirely valid, and represents an emission without occlusion. Examples include reflections, glows, glares, flares, fires, etc.

The only time "Convert Premul" should ever be used is if the imager knows she is feeding it unassociated alpha, but like a good imager, she would never be using anything except the One True Alpha, associated alpha.

Further reading:

Zap Anderssen of Autodesk now: http://forums.cgsociety.org/archive/index.php?t-278718.html

Larry Gritz of OpenImageIO and Sony Pictures imaging team: http://oiio-dev.openimageio.narkive.com/aaTdld9U/handling-unassociated-alpha#post2

Jeremy Selan, two time Academy Award winning imager behind OpenColorIO, now at Valve: https://groups.google.com/forum/m/#!topic/ocio-dev/ZehKhUFqhjc

The Infamous Adobe Photoshop thread. Pay key attention to the names, as there are many imaging legends in it, including Zap Anderssen dismantling one of the head developers of Photoshop, only to have Florian Kainz of ILM back up Zap's take on it:

http://forums.adobe.com/thread/369637

EXR Technical Introduction:

http://www.openexr.com/TechnicalIntroduction.pdf

I likely misspoke in the original question as, from memory, I can't remember exactly what that goofy keying node delivers in terms of alpha. I suspect it is unassociated, which makes the "Convert Premul" option work. Generally however, to avoid oopsies in the code etc., it is prudent to manually associate your alpha and control the merging to prevent data loss in the RGB channels. I still stand by the advice, however; always try to maintain a consistent state of alpha, and when you deviate, do so under scrutiny.

Additional information for implementation via nodes or software

There are three states for imagery with emission values encoded in the RGB channel, and an additional occlusion channel.

1. Associated. The RGB emissions are “as is” and represent the final emissions required. This may include zero alpha with nonzero RGB, such as the aforementioned case of reflections, fire, etc. To disassociate the emissions from the degree of occlusion, use a divide skip zero alpha. Leave any RGB emissions in the zero alpha case, as they could be glares, flares, fire, or reflections.
2. Unassociated. Here the alpha is not encoded. That is, the RGB represents the emissions without occlusion taken into account. To associate in the case of Unassociated alpha, use a simple multiply which will scale the RGB emissions by the linear degree of occlusion.
3. Disassociated. Here the alpha started life properly encoded as associated alpha, but has been disassociated to conduct colour transforms. To properly reassociate the data, multiply but skip zero alpha. That is, leave any emissions in the RGB channel as is.

Answer 2

I believe it's that Cycles renders have straight alpha. Straight means that, even though the pixel is transparent, it still has pure, unaltered color information. Like R255 G35 B48 A10.

Converting straight alpha to premultiplied is essentially multiplying the color info with the alpha, so R255*(10/255) G35*(10/255) B48*(10/255) A10 becomes R25 G3 B5 A10.

If you tick 'Convert Premul' you're implying the Cycles render has already been premultiplied (which it hasn't) and Blender will try to readjust color values to accommodate that. The alpha rim gets wonky.

It's not that straight alpha is technically 'better' than premultiplied alpha. Premul alpha saves disk space so it's the preferred option for real-time 3D and like. It's just that the algorithms for combining straight and premul pictures are not compatible. My guess is that Blender prefers the straight variant, so you have the option to convert premul to straight if needed.