We use normal maps, bump maps, and a lot of other special maps that contain non-color texture data. They are stored in image files but in linear color space (without gamma correction), and therefore we set the Color Space in the Image Texture node to Non-Color Data, as explained here: What exactly does the "Non-color data" option for image/environment textures do?
So far so good, but which is the best format? PNG seems to be popular, it can store 8 or 16 bits per channel, and unlike JPEG it does not suffer from compression artifacts since it uses lossless compression. I was satisfied with PNG, until in the comments of this question, color management expert troy_s said that he thinks EXR should be used for these files.
Since there was not enough space in the comments to go into details, I am asking this separately.
My biggest problem with EXR is that few programs can open it, and those programs start slowly, so I can't quickly check the contents outside Blender.