A Color Space, as Wikipedia says, is "a specific organization of colors." Color spaces are used in combination with color profiles as a way of making sure that the same numeric value is displayed as the same color on different media.
Color Management, as the Blender Manual says, " is used both to ensure all parts of the pipeline interpret colors correctly, and to make artistic changes like exposure and color grading."
Color management is the process of applying a color profile to a color space so that the resulting displayed color matches the intent of the color space designer.
Internally, Blender works in a linear color space, but image files may be stored in a different color space, such as sRGB. The Image Texture node converts the image from its stored color space to the linear color space according to various formulas that depend on how the image was created.
But image format files are also used to store data that's not actual color data, because the data needs a one-entry-per-pixel format. So Blender needs to be told that an image texture file isn't really an image so that Blender doesn't try to convert it.
The reason why you can sometimes get away with forgetting to set the space to Non-Color is that actual color spaces have regions that overlap each other, so converting from a color space to linear might give a result that is very close to what not converting would give. This works often for non-color data that's actually stored in a gray scale format. It doesn't work as well for non-color data, let normals, that is stored in a format that uses all three values.