I'm trying to bake the textures from a sculpted high poly mesh onto a low poly one, to get a PBR material applied to the low poly mesh. So far, I've baked the normal map and the roughness map, both giving satisfying results, but the colour map is giving me a hard time. The high poly mesh is composed of several PBR materials that have been applied by using weight painting. I've also texture painted on top of a metallic texture to give it a gold plated look. But a picture is better than 1000 words I guess.
Before going further, one thing worth mentioning is that UVs are slightly different from one mesh to another: I didn't mark any seam before sculpting my mesh, but since it's a distorted plane, I assume a projection from the front view is doing a good enough job. The same thing has been done to the low poly version to have a similar UV map.
Now, into the baking process. I first tried to apply the same process as I did for the normal map and the roughness : I created a new image in the low poly's shader, no alpha canal for this one, I made sure it was selected without being connected to anything else, colour space set either on sRGB or Linear (I've tried both), selected the high poly mesh first and then the low poly, baked using "selected to active". I've tried this method with both "Diffuse" and "Combined"
I've also tried to bake the 2 materials into one colour map before adding it to the low poly mesh. To do that, I placed 1 image texture node in each material pannel and baked it. The result wasn't different from the previous attempt, except that this time, if I try to use this image on the low poly, it won't fit the mesh properly due to the difference of UVs.
I've played with the Extrusion and Ray distance values but it didn't change much and the actual settings are those which worked the best for my normal and roughness map. Any idea on how I can get better images than these?