This issue stems from the way Blender currently performs color management. HDR color management is a complicated subject, so I recommend some related reading first if you are unfamiliar:
The problems of using color as alpha
The Blender compositor always works on scene-referred values in the Linear color space; this cannot be changed. This means that the RGB values of the rendered image are not in the range [0.0, 1.0], but rather [0.0, +∞), and they have not yet been mapped into the display color space (usually sRGB).
After compositing has finished, Blender applies the selected View Transform, which maps the scene-referred values into the display color space. The exact way this is done is outside the scope of this answer, but you can read the above links for more details.
Alpha makes the already-complicated color management story significantly more complicated. Importantly, the alpha channel is not affected by the view transform; alpha is always linear information stored in the [0.0, 1.0] range. Blender uses associated alpha (aka “premultiplied” alpha), so the meaning of these values is simply the amount that the background is occluded when using Alpha Over compositing.
This is a problem for your compositing setup, since you are trying to use color data directly as an alpha channel:
The color-to-scalar transformation is relatively well-defined—Blender just takes the luminance—but interpreting the result as alpha is more fraught. You are using a scene-referred value in the range [0.0, +∞) where a value in the range [0.0, 1.0] is expected. This has two problems:
Most obviously, the values above 1.0 are simply clipped.
More subtly, even inputs that are in the [0.0, 1.0] range will not be mapped to display-referred values.
The second point is tricky to understand, but its implications are severe. Scene-referred values don’t have any useful absolute interpretation, only a relative one, which means the set of values that fall into the [0.0, 1.0] range is largely arbitrary. In other words, the value you’ve ended up with is largely meaningless, and it only looks close to correct by sheer coincidence.
Imagining a better solution
The previous section outlines why using scene-referred color values as alpha is fundamentally the wrong approach, but what could a better one look like? The answer to that question is different depending on what you’re actually trying to do. Let’s address each of those in turn.
Rendering an image for further compositing
If the goal here is to generate an output image that is subsequently composited on top of another scene, converting to straight alpha at this point isn’t what you want, anyway. Instead, you should save your render in the OpenEXR format, which will save the raw scene-referred values using associated alpha.
This allows you to do your compositing in the scene-referred Linear color space, then convert to a display-referred format only as a final step. This sidesteps all of the problems that make color management of images with alpha channels so awkward to deal with, and it ensures you’ll get the highest-quality result.
Rendering an image for display
If you want your semi-transparent output to be the final render result, intended for direct display, things are more complicated. Converting from associated to straight alpha is not, in general, a lossless process, and doing it properly depends on what background the image will eventually be displayed against.
However, in this particular case, one reasonable interpretation would be to simply assume the image will be displayed upon a solid black background and calculate the appropriate straight alpha values from there. To do that, you would need to do calculations in the display-referred color space, since you care about which absolute color values are eventually produced.
Unfortunately, Blender currently does not currently provide any mechanism for converting from scene- to display-referred data in the compositor. Providing this functionality in a proper way would likely be tricky (since applying a view transform is a whole-image operation that depends on the overall light intensity of the image), but it isn’t in principle impossible. That said, that doesn’t help you much, since there’s no way to do it right now.
If you really want a straight alpha PNG that best captures the visual characteristics of your render, your best bet is probably to render to OpenEXR and use an external tool to “digitally expose” the image in a way that handles alpha more satisfactorily. I unfortunately do not know of such a tool to recommend, but maybe someone else can chime in if they do.
Sorry.
