Circled in red you can see some of my UV islands are not taken into account by the AO baking process. This is due to their small size on the model (and therefore the UVs) and the small size of the AO texture itself. The problem is that those faces will be assigned a random AO color from whatever island lies next to them (because of pixel margin, see the image), which will result in weird breaks on the object surface (e.g a black polygon in the middle of a white surface).
I see a couple of solutions but most of them have annoying downsides:
Modify UVs manually and create seams by hands (avoiding Smart UV project), aiming for continuity when unwrapping. -> This take too long and is repetitive work, might not work on some models.
Selecting a small face on the object, and Select Similar by Area, then scale the UVs for all those faces. -> This is my current solution, it works, but require some manual tweaking for really thin islands.
Bake on a higher res texture and downscale it afterward to use them in a real-time engine. -> Baking time will be much longer.
Some solutions I'm thinking about theoretically but can't find a way to actually apply them in Blender:
- Select islands if size < 1 pixel automatically, and scale them until they are at least 1 pixel in x and y axis.
- Force the baker to acknowledge sub 1 pixel islands.