PNG is an unassociated alpha image format, while Blender, as any other 3D renderer produce associated (often called premultiplied) alpha images.
To produce an unassociated alpha image using the compositor, you have to "pre-divide" the alpha channel and connect the unassociated RGB to the composite output and the alpha socket of the same node.
You can unassociate the alpha channel using an Alpha convert node (using the "premul to straight" mode) or just divide the rendered image by its alpha with a mix node (divide mode).
That's the correct way to save unassociated from Blender if it has to be predivided manually for any reason, otherwise Blender does the predivision automatically when PNG is selected as output (and in that case it's not necessary to manually connect alpha to the output).
PNG will always save the RGB information of transparent pixels because it is an unassociated alpha format.
However, producing an unassociated alpha image from an associated image is a destructive process, because it has to be divided (and alpha zero pixels are discarded).
Saving an associated image as PNG is not allowed by the format, so it's important to keep in mind that any hack used to produce a PNG as described by the OP will produce an invalid image that is likely to produce edge artifacts when composited.
It is strongly advised to respect the alpha association dictated by the format and avoid this kind of hacks if PNG has to be used.