Normals to understand - How can I know where the normal ring is convex and where it is concave?
This is a Tangent Space normal map.
That is to say, blue (Z) represents the interpolated normal of the underlying geometry, red (X) represents a tilt towards the tangent direction, which is positive X in the UV, (texture,) space, as mapped onto the surface, and green represents a tilt towards the bitangent direction, which is positive Y in the UV space, again, as mapped onto the surface of the geometry.
To quote directly from the Wikipedia page on normal mapping - 'How it Works':
A normal pointing directly towards the viewer (0,0,-1) is mapped to (128,128,255). Hence the parts of object directly facing the viewer are light blue. The most common color in a normal map.
A normal pointing to top right corner of the texture (1,1,0) is mapped to (255,255,128). Hence the top-right corner of an object is usually light yellow. The brightest part of a color map.
A normal pointing to right of the texture (1,0,0) is mapped to (255,128,128). Hence the right edge of an object is usually light red.
A normal pointing to top of the texture (0,1,0) is mapped to (128,255,128). Hence the top edge of an object is usually light green.
A normal pointing to left of the texture (-1,0,0) is mapped to (0,128,128). Hence the left edge of an object is usually dark cyan.
A normal pointing to bottom of the texture (0,-1,0) is mapped to (128,0,128). Hence the bottom edge of an object is usually dark magenta.
A normal pointing to bottom left corner of the texture (-1,-1,0) is mapped to (0,0,128). Hence the bottom-left corner of an object is usually dark blue. The darkest part of a color map.
By Julian Herzog, CC BY 4.0, https://commons.wikimedia.org/w/index.php?curid=25512888
From this, you can see the top disc is convex, and the bottom disc concave.
(So, before anything else: if you're going to listen to this answer, please use your own head as well :) I find 3D math difficult, even though I enjoy it, and I'm working this answer out as I go. The output is what I want, which suggests that I did it right, but I'm not sure, and I don't want to make myself out to be an expert-- I'm only offering what I think is right, in case it or the thought processes it involves are useful.)
Concavity and convexity are per-axis values. A sphere is convex in both axes. A cave is concave in both axes. A saddle is convex in one axis and concave in one axis.
Tangent space normal maps cannot be understood without knowing information about the mesh-- their normals are in tangent space, and tangent space depends on both the mesh normal and the UV tangent. So asking about the concavity or convexity of a tangent space normal map, without taking the mesh into consideration, isn't really something that makes any sense. Change the UV, and you might change the curvature.
But an object space normal map doesn't rely on the mesh, and per-axis concavity or convexity makes sense in that case-- and an object space normal map can be created from a mesh + tangent space normal map, just by baking an object space normal map (or by remapping object-space normals to the 0,1 space and baking raw emission.)
Now, how to evaluate whether our normals are convex or concave in some particular axis? By comparing offset normals-- that is, we want to see how our normals change as we travel across our UV:
So I compare our normal if we offset 1 pixel in U with our normal if we offset -1 pixel in U. Then I take the cross product of those two vectors. If our second vector is positively rotated relative to the first vector, the cross product is going to point in one direction, but if it's negatively rotated, then the cross product is going to point the other direction. So I can take the dot product of the cross product with any arbitrary vector to figure out which way it's pointing, and thus whether the cross product points this way or that way.
I'm outputting the convexity in U-- that is, convexity in the UV tangent. The full map of convexity would also output convexity in the V, by looking at the normals with offset V inputs instead of offset U inputs. If the convexity of U is different than the convexity of V, we have negative (saddle) curvature; if they're the same, we have positive (sphere or cave) curvature.
Also note that because I'm doing this by looking up actual values on an actual image, that seams impact this; and note that I'm not handling the special case of when my offset normals are identical (which would lead to a 0-vector cross product, which would lead to a 0 dot product, and would indicate no curvature, neither convexity nor concavity.)
Blender isn't really great for doing this. A decent game engine shader could get you delta(normal)/delta(UV) a lot more easily and more accurately; and really, this is the sort of work that would make more sense to do in compositing, except Blender just doesn't have the compositing tools developed enough to be able to do it without a lot of dumb busy work.