Is it possible to generate an image based on edges and corners?

Question

Prompted by my previous question, is it possible to generate a black and white map based on the sharpness of edges and corners of a mesh?

E.g. to use as a map to make procedural texture displacements more intense on edges and corners.

How can I achieve this?

Answer 1

Vertex Colors

Yes, you can use vertex colors.

• UV map the mesh.
• Enter vertex color paint
• Paint Menu -> Dirty Vertex Colors
  (adjust angle to suit preference)

Setup for baking: * Make the material to use vertex colors (so they render in the viewport). * bake the vertex colors into an image.

---

Note 2.70 - when its released, (or any recent build) has the ability to bake vertex colors directly so you don't have to worry about setting up the material.

Ambient Occlusion

You can configure AO to shadow sharp edges (with inverted normals) and bake that into an image too.

Answer 2

As Ideasman42 hinted, it is possible to invert the normals and bake AO to get what you want. However, this doesn't work at all times.

It works like this. Normally when baking AO you want holes and dents in your mesh to be darker. Not the same amount of light reaches those areas. This can also be used for dirt. If we instead want to mark the protruding edges we can invert the normals. A protruding edge is converted into a dent (sort of). It will render darker when baking AO.

One tweak must be done. If you bake a cube 2 units wide it will be baked totally black. Rays for the inside of the cube (because the normals are inverted) will hit the opposite side of the cube. The Ray length must be set very short. Here are the settings for this:

A downside of this method is that it works porly with self intersecting geometry. Rays from the inside will hit the self intersecting pieces and render dark where you don't want them.

Vertex colours strike me as the superior method in most cases.

Comparison

Made a simple test to compare them both:

For the vertex paint method I hade to subdivide some to get the desired effect.

With baking AO I hade to save the image and edit it externaly (inversion and contrast).

Answer 3

In Blender 2.8:

• Make object shade smooth
• Use Auto Smooth or Custom Normal to define hard edge normal
• Now switch to cycle
• Create a Bevel node, Geometry Node, Vector math (Dot Product), Invert node
• Connect Bevel>normal and Geometry>normal to Dot Product
• Connect invert node with Dot product output
• Use Viewer node (ctrl+Shift+LMB (Node Wrangler)) to see the mask
• If there are any normal map, then plug normal map node with the normal image (Non Color Data). Then connect normal map node output to the bevel normal input socket.

For clarity Video Tutorial: https://youtu.be/9ojVYZVqiU8

Image below:

Answer 4

Here is my adaptation of a technique I found in the b°wide node pack's fake AO node group. It works by subtracting the normal and true normal vectors, which on a smooth shaded mesh are the smoothed and original (non-smoothed) normals respectively. So closer to the corners of the mesh there will be a greater difference between the two and thus the subtraction will give a higher value.

The only downsides to this method are that it requires the mesh be smooth shaded and the mesh must be subdivided. You can get around the first one by plugging the true normal into the normal socket of the shader as I have done in the example.

Here is the render with the output of the subtraction plugged into a diffuse shader.

Here is the node setup:

And the mesh (as you can see it is smooth shaded):

This can then be baked using cycles baking to create the map you are looking for. I use this technique fairly regularly on materials for mainly old metal and rocks. As an aside, two slightly different gray diffuse shaders mixed based on this make an amazingly realistic material on a high poly, hand-sculpted, rock.