# Projecting multiple images onto an object with UV Project modifiers

I've just about got to grips with the UV Project modifier. I've unwrapped my object into a UV map with the image I want to project, and then applied it using a camera as a projector.

However, I want to project from multiple angles. So I created another UV map the same way with my second image and created another UV Project modifier. But this won't work, because I can only select 1 UV map at a time. I want the two projections to be blended together (or, even better, chosen programmatically according to which surface normal has the closer angle to the projector when I eventually move to automating this process in Python).

Here's what it looks like when projecting from the camera on the left. It's projecting the same image it would see if the house were textured properly. Likewise, the camera on the right has an image of the properly textured house from its angle, ready to "fill in the gaps" of what the left camera cannot see.

P.S. Please try not to include steps that could only be done manually/by eye. Because I am applying research to a large collection of objects, this will have to be done programmatically eventually.

• – lemon Mar 31 '17 at 7:13
• I tried stacking the two projection images into a single image and then used multiple projectors in the modifier like in the related question, but because the images and projectors have perspective, I can't just move the projectors until the geometry is aligned. I need to somehow keep the projectors in place while adding an offset to the image they project. – CJxD Apr 1 '17 at 15:28
• I've just gave a look to your file. And I'd like to be sure to understand the question, finally: you wanted to use UV project in order to reconstitute the UV maps from the already rendered images (as a test of the method)? – lemon Apr 4 '17 at 11:45
• You can calculate a perspective projection and define the UV maps from that, but some UV faces will overlap – lemon Apr 5 '17 at 4:41
• @lemon Could you elaborate further on this? Imagine this use case: I have a Blender model of a real-life object but it's untextured. I also have multiple images of the object taken with a real-life camera that I know the parameters of. What's the best way to project these images to recreate as much of the object's look as possible? – CJxD Apr 5 '17 at 9:26

Following the last comments, this is more some guidelines than a real answer.

What you can do/code inside Blender is an addon based on the same principle as UV project modifier. The principles are the following:

• Given a camera and its parameters, you can obtain its projection matrix. For instance have a look at this.

• You can also use/create an UV map (to be built by the code). In this addon, I image that you could parameter one UV map by camera (that may be needed or not, depending on the way you'll define the materials/textures correspondences).

• Now you can loop over your model polygons (faces), test their normals against the concerned cameras axis and so determinate which polygons are best concerned by which camera.

• Knowing each polygon/camera pairs, and the previous projection matrix, you can calculate the UVs of the polygons, then associate them to the corresponding UV map.

You are done...

Eventually you can also dynamically test camera's parameters changes (tuned by user to adjust the uv/image mapping) and auto update the UV map (performance to be taken into account).

That should be relatively simple.

But the problem is you will have partially hidden polygons due to the projection (it is true too for orthographic projection) and these polygons will overlap on the UV map.

edit:

A more complete answer in an addon that calculates the UV projections of the mesh, depending of the camera positions.

As indicated above, it uses the same principle as the UV project modifier (in the UI part).

(the blank parts in the gif are parts where no image is given).

For each camera, you need to indicate the corresponding UV map and material slot to use. Then the script calculates the polygons oriented to each camera and for each polygon the projected coordinates to be assigned to the UV map.

import bpy
import time
from mathutils import Vector
from operator import itemgetter

#Used to store information related to each camera setting
class UVPPCameraSetting:
def __init__( self, scene, camera, obj, uvMap, materialSlotIndex ):
self.camera = camera
self.uvMap = uvMap
self.materialSlotIndex = materialSlotIndex
#Z axis of the camera translate in world (Z axis is opposite to view)
self.zInWorld = Vector( (0, 0, 1) )
self.zInWorld.rotate( self.camera.matrix_world.to_euler() )
#To keep polygons to handle for this setting
self.polygons = set()
#Camera parameters
#Matrix to convert from object coordinate to camera coordinates
self.toCameraMatrix = camera.matrix_world.inverted() * obj.matrix_world
#The frame is composed of the coordinates in the camera view
frame = [v / v.z for v in camera.data.view_frame(scene=scene)]
#Get the X, Y corners
self.minX = min( v.x for v in frame )
self.maxX = max( v.x for v in frame )
self.minY = min( v.y for v in frame )
self.maxY = max( v.y for v in frame )
#Precalculations to avoid to repeat them when applied to the model
self.deltaX = self.maxX - self.minX
self.deltaY = self.maxY - self.minY
self.offsetX = self.minX / self.deltaX
self.offsetY = self.minY / self.deltaY

#Calculate the UV coordinateds from the object coordinates
def CalcUV( self, objCo ):
#Object in camera view
camCo = self.toCameraMatrix * objCo
#Z is "inverted" as camera view is pointing to -Z of the camera
z = -camCo.z
try:
#Translates x and y to UV coordinates
x = (camCo.x / (self.deltaX * z)) - self.offsetX
y = (camCo.y / (self.deltaY * z)) - self.offsetY
return x, y, z
except:
#In case Z is zero
return 0.5, 0.5, 0

def UVPerspectiveProject( scene, obj, camSettingList ):
print( '------------------------' )
startTime = time.time()

matrix_world = obj.matrix_world
rotation_world = matrix_world.to_euler()

#Assign polygon to its corresponding camera considering faces most aligned to camera view
for p in obj.data.polygons:
normalInWorld = Vector( p.normal )
normalInWorld.rotate( rotation_world )
camSetting, maxDot = max( ((c, normalInWorld.dot( c.zInWorld)) for c in camSettingList), key=itemgetter(1) )

print( 'precalculations. elapse in seconds', time.time() - startTime )

loops = obj.data.loops
vertices = obj.data.vertices
#For each setting,
for cs in camSettingList:
camera = cs.camera
uvMap = cs.uvMap
polygons = cs.polygons
projectedCo = {} #Storage to avoid multiple calculations of the same world_to_camera_view
#Go through all polygons
for p in obj.data.polygons:
#If the polygon corresponds to the setting
if p.index in polygons:
#Assign the material index
p.material_index = cs.materialSlotIndex
#Calculate each vertex uv projection
for i, vi in [(i, loops[i].vertex_index) for i in p.loop_indices]:
if vi not in projectedCo: #not already calculated for this cam
x, y, z = cs.CalcUV( vertices[vi].co )
projectedCo[vi] = (x,y)
uvMap.data[i].uv = projectedCo[vi]
#If not, uv are set to (0,0)
else:
for i in p.loop_indices:
uvMap.data[i].uv = (0, 0)

print( 'done. elapse in seconds', time.time() - startTime )

def TargetObjExists( context ):
try:
scn = context.scene
obj = scn.objects[scn.uvPerspectiveProject.object_name]
return obj.type == 'MESH'
except:
pass
return False

def IsCamera( context, camera_name ):
try:
camera = context.scene.objects[camera_name]
return camera.type == 'CAMERA'
except:
pass
return False

def SettingsAreOK( context ):
try:
scn = context.scene
settings = scn.uvPerspectiveProject
obj = scn.objects[settings.object_name]
assert obj.type == 'MESH'

for item in settings.cameras_settings:
camera = scn.objects[item.camera_name]
assert camera.type == 'CAMERA'
uv_map = obj.data.uv_layers[item.uv_map_name]
materialSlot = obj.material_slots[item.material_slot_name]

return True
except:
pass
return False

class UVPerspectiveProjectOperator( bpy.types.Operator ):
bl_idname = "lemon.uvperspectiveprojectoperator"
bl_label = "UV perspective project"
bl_options = {'REGISTER'}

@classmethod
def poll(self, context):
return context.mode == 'OBJECT' and SettingsAreOK( context )

def invoke(self, context, event):
scn = context.scene
settings = scn.uvPerspectiveProject

#try:
obj = scn.objects[settings.object_name]
cameraSettings = [UVPPCameraSetting( scn, scn.objects[item.camera_name], obj, obj.data.uv_layers[item.uv_map_name], obj.material_slots.find( item.material_slot_name ) ) for item in settings.cameras_settings]
UVPerspectiveProject( scn, obj, cameraSettings )
#except:
#pass

return { 'FINISHED' }

class UVPerspectiveProjectCamSettingsActions( bpy.types.Operator ):
bl_idname = "lemon.uvperspectiveprojectpanelcamlistactions"
bl_label = ""

action = bpy.props.EnumProperty( items = ( ('UP', "Up", ""), ('DOWN', "Down", ""), ('REMOVE', "Remove", ""), ('ADD', "Add", ""), ) )

def invoke(self, context, event):

scn = context.scene
settings = scn.uvPerspectiveProject
index = settings.camera_setting_index

if self.action == 'DOWN' and index < len(settings.cameras_settings) - 1:
settings.cameras_settings.move( index, index + 1 )
settings.camera_setting_index += 1

elif self.action == 'UP' and index > 0:
settings.cameras_settings.move( index, index - 1 )
settings.camera_setting_index -= 1

elif self.action == 'REMOVE':
settings.cameras_settings.remove( settings.camera_setting_index )
settings.camera_setting_index -= 1

settings.camera_setting_index = len(settings.cameras_settings) - 1

return {"FINISHED"}

class UVPerspectiveProjectCamSettingsPanel( bpy.types.UIList ):

def draw_item(self, context, layout, data, item, icon, active_data, active_propname, index):

scn = context.scene
settings = scn.uvPerspectiveProject

try:
obj = scn.objects[settings.object_name]
except:
pass

suffix = " (" + str( index + 1 ) + ")"

row = layout.row()
col = row.column( align = True )
col.prop_search( item, "camera_name", scn, "objects", text="Camera" + suffix  )
col.prop_search( item, "uv_map_name", obj.data, "uv_layers", text="UV map" + suffix )
col.prop_search( item, "material_slot_name", obj, "material_slots", text="Material" + suffix )

def invoke(self, context, event):
pass

class UVPerspectiveProjectPanel( bpy.types.Panel ):
bl_idname = 'lemon.uvperspectiveprojectpanel'
bl_space_type = "VIEW_3D"
bl_region_type = "UI"
bl_label = "UV perspective project"

@classmethod
def poll(self, context):
return context.mode == 'OBJECT'

def draw(self, context):
layout = self.layout
scn = context.scene

settings = scn.uvPerspectiveProject

layout.row().prop_search( settings, "object_name", scn, "objects", text="Object" )

if TargetObjExists( context ):
layout.row().label( text="Cameras settings:" )

row = layout.row()
row.template_list( "UVPerspectiveProjectCamSettingsPanel", "", settings, "cameras_settings", settings, "camera_setting_index", rows = 3 )

col = row.column( align = True )
col.operator( UVPerspectiveProjectCamSettingsActions.bl_idname, icon='ZOOMIN', text="" ).action = 'ADD'
col.operator( UVPerspectiveProjectCamSettingsActions.bl_idname, icon='ZOOMOUT', text="" ).action = 'REMOVE'
col.separator()
col.operator( UVPerspectiveProjectCamSettingsActions.bl_idname, icon='TRIA_UP', text="" ).action = 'UP'
col.operator( UVPerspectiveProjectCamSettingsActions.bl_idname, icon='TRIA_DOWN', text="" ).action = 'DOWN'

layout.row().operator( UVPerspectiveProjectOperator.bl_idname, text="Apply" )

class UVPerspectiveProjectCamSettingsProps( bpy.types.PropertyGroup ):
camera_name = bpy.props.StringProperty()
uv_map_name = bpy.props.StringProperty()
material_slot_name = bpy.props.StringProperty()

class UVPerspectiveProjectProps( bpy.types.PropertyGroup ):
object_name = bpy.props.StringProperty()
cameras_settings = bpy.props.CollectionProperty( type = UVPerspectiveProjectCamSettingsProps )
camera_setting_index = bpy.props.IntProperty()

def register():
bpy.utils.register_module(__name__)
bpy.types.Scene.uvPerspectiveProject = bpy.props.PointerProperty( type = UVPerspectiveProjectProps )

def unregister():
del bpy.types.Scene.uvPerspectiveProject
bpy.utils.unregister_module(__name__)

if __name__ == "__main__":
register()


• Thanks again for this script, it's just what I'm looking for. I'm encountering an issue though: in your .blend file, all of the projections work together, but when I tried to replicate what you did, I can only see one UV Map at a time. All other areas are blank until I select a different UV map. – CJxD May 1 '17 at 11:53
• And I don't suppose there's a way to use Blender Render rather than Cycles? I have hundreds of thousands of renders to make. – CJxD May 1 '17 at 11:54
• @CJxD, I think that can work for Blender Internal too. Concerning the projections: each UV map will have all projection (this is the way Blender is working) but unneeded faces are set to 0,0,0 (by choice/convention in the script). The principle globally (in both renderer): a face is assigned to a material and a material uses a UV map (via a texture/image in both renderer). – lemon May 1 '17 at 12:19
• I get you, but how then were you able to show both projections at the same time if half the faces are set to 0,0,0 for any particular UV map? – CJxD May 1 '17 at 13:40
• @CJxD, you can show both projection in 3D view due to the material setting (which can be done also for BI), because faces are assigned to the corresponding material (and that's why the UI of the script links cam/mat/uvmap). But you can't show multiple projections in the image/uv editor. UV set to (0,0) are set so for the non useful faces for this particular UV map. – lemon May 1 '17 at 13:45