On trying to answer this question, I can't subtract information from the alpha channel without running into borders that persist from the UV mapping node. Those borders come from the original image's alpha channel.
When I map a mask using the UV mapping node, I would have expected to get only the information from the mask in the image (meaning the mask on a black background), but the image gets some alpha information from the render layer as well. That information I cannot subtract from the alpha channel.
This is the setup I'm after, that shows the error:
and the result of the subtract operation:
If you look at the mask going through the UV map, there are some edges that are not part of the mask at all, but are the edges of the original alpha channel of the image used in the texture.
The object in the scene is just an EXR image imported as plane, mapped to a principled bsdf, that uses the Alpha channel of the image as Alpha on the shader.
Here's the blend file
EDIT:
If I ignore the alpha channel on the original image, the mask shows correctly, but now the edges of the UV map show.
