The issue is one of the many problems with the PNG format: Unassociated Alpha
Blender will display the RGB information because it is part of the file and the alpha channel has not been associated correctly, and Blender expects associated alpha.
Quoting from the PNG specifications:
PNG uses "unassociated" or "non-premultiplied" alpha so that images with separate transparency masks can be stored losslessly.
An alpha value of zero represents full transparency, and a value of (2^bitdepth)-1 represents a fully opaque pixel. Intermediate values indicate partially transparent pixels that can be combined with a background image to yield a composite image. (Thus, alpha is really the degree of opacity of the pixel. But most people refer to alpha as providing transparency information, not opacity information, and we continue that custom here.)
The color values stored for a pixel are not affected by the alpha value assigned to the pixel. This rule is sometimes called "unassociated" or "non-premultiplied" alpha.
"premultiplied alpha", stores pixel values premultiplied by the alpha fraction; in effect, the image is already composited against a black background
A different way to think about the problem is to blender sees the RGB information on those pixels as luminous and transparent at the same time... (like the fire on a candle, for example).
What would happen if you switch from Straigth Alpha to Premultiplied?
Not a real solution, because the RGB information is still there (though this will display correctly in rendered view). When you premultiply blender would expect black instead of RGB information.
The proper way to deal with this situation is to multiply the alpha with the RGB information.
Split the RGB information, use a converter>math node in multiply mode to multiply the information of each channel by alpha and recombine the RGB channels.
(click on the image to enlarge)
Or use the color and alpha information on a color Mix node in multiply mode and use 1 as factor.
Other ways to use unassociated alpha information.
Unassociated alpha could then be thought as a mask to determine how two elemements mix. White (1) would be one element, black (0) would be the other and any shade of gray would be a partial mix.
You can then use such a mask to control the mix between two different shaders(like emission and transparent) plugging alpha directly as a factor for the mix.
Or if what you need is to control the color of the sader only, use a color mix factor and have the alpha information work as a mask for the mix.
Further reading
Why does blender then use associated alpha and why shouldn't I use PNGs you might be asking...
Quoting from this very clever answer:
ONLY associated alpha (often called pre multiplied) allows the
compositing of pixels that are luminous and transparent (Like halos, blurs, fire, glares, etc) so choose a file format that supports associated alpha (the most common for visual effects is EXR)
It's impossible to produce that kind of alpha blending with
unassociated alpha, so formats like PNG are completely ruled out.
Image editing programs like Photoshop and GIMP work internally with
unassociated alpha, so they are not suitable for compositing that
kind of effects with alpha blending. You need to cheat and use an RGB
only image with an addition instead (those programs are also likely
to destroy the RGB information of every pixel with alpha=0, so be careful).
Suggested links:
https://wiki.blender.org/index.php/User:Sobotka/Alpha/Associated_and_Unassociated_Alpha
and
Compsiting Digital Images, by Porter & Duff
