How to adjust normal map

Question

I am attempting to create a Normal Map in order to apply High Res detail to a Low Res object. No matter how I set the Ray Distance for the bake there are areas giving me problems. Especially tight areas like between the toes and mouth as well as the eye openings, nostrils and the tail base. Any suggestion on how to clean up these areas?

BLEND FILE HERE

Answer 1

Actually yes. It took me a while. Normal mapping is going to be one of the hardest things to do right. With a multi object, model you can explode the mesh and use displacement modifier cages. I don't think this model can do that though, not easily anyways, its a smooth organic and the cage might need a lot of tweaking..

I dont know if you could rig and pose the model in order to create a cage for the small areas like the toes armpits and groin. If you used a displace modifier to create a cage those sinchy areas would be toughest. You might have to fineagal the cage mesh for a better bake. I could be very wrong here, im still a novice. From what I've seen you may need to manually paint those problem areas in the normal map in GIMP. I wouldnt even know where to begin with that. Characters are kind of hard. If this was an object you could split it into parts and bake easily.

Please don't throw me in an oven if someone more experienced blows my suggestions out of the water.

On udemy theres a 3dcoat and blender tutorial on making a sword. It does an amazing job describing the normal map bake process.

https://www.udemy.com/intermediate-game-asset-creation-3d-modeling-in-blender/learn/v4/overview

Answer 2

As continuation of the above answer... i have to say that you have to make a cage, to make a cage use your low poly mesh, duplicate it, leave in the same place, then add a displace modifier with very low but enough distance, it has to look inflated and like shrink wrapping to it but not the like high and low poly meshes...then select "cage" option and select the cage model, before trying baking again, Some tips tip 1:

change your render (cycles) setting to 1 sample, you wont need more than that, if you dont change this, is gonna take longer longer to bake, oh yeah as well change the performance - tiles setting to the image size of the normal map output image i.e 2048 x 2048 px if applies or whatever resolution you are using...

tip 2: using UDIMS for your model is good if your model has several splitted uv islands,if its a single one, just keep the default setting instead, with this you will get more resolution which means more detail in the final normal map images.

tip 3: if your model has lots of hard edges, mark those edges with seams, hard edges are buggy when baking normal maps, you will need to split them to get clear results in shading...for example a cube that has 6 faces, which obviously these edges around each face are sharp, you will to mark them with seams, as result the uv map should look like each face from the cube are splitted, rathen than having the default uv map that looks like a cross uv island, so unfortunately we cant yet have normals "with good shading" with such kind of uv map projection, and this is overwhelming this baking normal techniques,there should be already have been replaced with a better and more useful technique without needing to care too much of the uv maps, because yeah things like bad uv stretching can also cause bad normals bakes, so if you try baking that with this type of uv map you would notice most of the edges have good shading but the rest dont...

tip 4: good smoothing groups in the low poly mesh matters too (as well goes for the HP mesh) if the low poly has bad smoothing, this normal data will be also be transferred, or maybe can be said "blended" to the high poly normal data, as result getting the two normal datas joined in the output normal map image, thus getting a good normal shading from the high poly, but with also bad shading because the low poly mesh wasn't taken in to account before baking... ah sorry if this is too long or maybe a resume i think so, of the flaws and issues of baking normals map, leave a comment if you think i missed something, or other else tip.