Since a 3D application is a less controlled environment (thenthan a game-engine or 2D-paint-tool for example) - an image may be used for a modifier displacement map, composite node input, a background in the view-port or be painted onto - all at once. This means we have some constraints which make it harder to optimize for any single use-case.