I want to modify a 3D model in a particular way. Render images, modify these images (using a python program), and project these modifications back onto the texture. Is this possible or not?
Projection painting and UV project modifier
If the context of the question is the following:
- You have a model, a camera and a rendered image taken from this camera
- Then something (a Python program) changes the rendered image
- You now want to report the modification done on this image onto the object original texture
Then you can use the following method:
Important: during this process don't move the camera and don't move the 3D object.
The base setup is the following:
A model with some material, a UV map (named 'original') and a texture (named 'original' also).
Render the image, so you have for instance this:
And modify it by the way you want. For instance I've this:
Import the modified image into Blender.
Create a second UV map for the 3D object and name it 'projection'.
Select the 3D object, go to the modifier panel and add a 'UV project' modifier, and in this panel:
- Choose the 'projection' UV map
- Choose the camera as projector
- Indicate the aspect ratio which is the dimensions of the rendered image (I've not found a way to do that automatically or by an option)
You should have something like this:
Now, go to 'texture paint' mode. Open the tool panel and choose the clone brush.
- Set the mode to 'single image'
Choose your original image and the original UV map
Indicate 'clone from paint slot'
- Choose the 'modified' image as source
- And choose the 'projection' UV map for it
Now, in the 3D view, still in texture paint mode, you can draw to clone from your modified image onto the object original texture:
During this process you may want to avoid shader effects due to the shaders used when the image is originally rendered, because lights and shadows will get the rendered image altered from the object original texture. If so, you can set the object material like this, plugin directly the color input (here the original texture) to the material output node:
Yes it is possible and called projection painting. CGMatter has tutorials on this topic that should get you started. Blender 2.8 Texture extraction projection painting (part 1) and Blender 2.8 3D Object removal projection painting (part 3)