I am a beginner in Blender and Blender scripting. Currently I am trying to create a script that automatically renders images of a single object from different camera viewpoints. I use the following setup:
I have a text file with the different camera poses (position and orientation) w.r.t the world frame (object).
I create the object (Monkey) using the python API
I created functions such as add_camera(), add_light, render_settings, which create these 'objects' with the desired settings. The code for the add_camera function is shown below.
def add_camera(name, focal_length, x,y,z,q0,q1,q2,q3): cam_data = bpy.data.cameras.new(name) cam = bpy.data.objects.new(name=name,object_data=cam_data) bpy.context.collection.objects.link(cam) cam.data.lens = focal_length cam.data.type = 'PERSP' cam.rotation_mode = 'QUATERNION' cam.location = mathutils.Vector((x,y,z)) cam.rotation_quaternion = mathutils.Quaternion((q0,q1,q2,q3)) bpy.context.scene.camera = cam
I then use a for loop to loop over the different camera poses extracted from the textfile, creating a new camera for each pose and setting that one to be active. This works and creates the expected rendered images from the different camera views.
for pose in range(len(verts)): scene = bpy.context.scene cam = add_camera('Camera',50,verts[pose],verts[pose],verts[pose],verts[pose],verts[pose],verts[pose],verts[pose]) light = add_light('Light',verts[pose],verts[pose],verts[pose], 500, 'POINT') file = os.path.join('D:/Images/', 'image_' + str(pose)) bpy.context.scene.render.filepath = file bpy.ops.render.render(write_still=True)
Furthermore, I want to generate and export the 2D bounding box around that object. For the bounding box generation I am using this script which works perfectly for a single scene/instance (using e.g. last created camera). However, when I try to incorporate this in the loop, it fails and outputs different coordinates compared to simply running the aforementioned bounding box script on a single scene (camera, object and light). It often outputs the entire image size as the bounding box.
I adapted the for loop as follows:
mesh_object = bpy.data.objects['Monkey'] for pose in range(len(verts)): scene = bpy.context.scene cam = add_camera('Camera',50,verts[pose],verts[pose],verts[pose],verts[pose],verts[pose],verts[pose],verts[pose]) cam = bpy.context.scene.camera light = add_light('Light',verts[pose],verts[pose],verts[pose], 500, 'POINT') file = os.path.join('D:/Images/', 'image_' + str(pose)) print(camera_view_bounds_2d(scene, cam, mesh_object)) bpy.context.scene.render.filepath = file bpy.ops.render.render(write_still=True)
I have no idea what is going wrong, so any help would be greatly appreciated! Furthermore, the end-goal would be to automatically write the bounding-boxes for each rendered image to a separate text file as well. The idea was to use the implementation shown here:
def write_bounds_2d(filepath, scene, cam_ob, mesh_object): with open(filepath, "w") as file: file.write("%i %i %i %i\n" % camera_view_bounds_2d(scene, cam_ob, mesh_object))
EDIT: The BoundingBox script I used, followed by a script that loops over the poses. Furthermore, the datafile of the different poses is also included.
import bpy import os def clamp(x, minimum, maximum): return max(minimum, min(x, maximum)) def camera_view_bounds_2d(scene, cam_object, mesh_object): """ Returns camera space bounding box of mesh object. Negative 'z' value means the point is behind the camera. Takes shift-x/y, lens angle and sensor size into account as well as perspective/ortho projections. :arg scene: Scene to use for frame size. :type scene: :class:`bpy.types.Scene` :arg obj: Camera object. :type obj: :class:`bpy.types.Object` :arg me: Untransformed Mesh. :type me: :class:`bpy.types.Mesh´ :return: a Box object (call its to_tuple() method to get x, y, width and height) :rtype: :class:`Box` """ """ Gets the camera frame bounding box, which by default is returned without any transformations applied. Create a new mesh object based on mesh_object and undo any transformations so that it is in the same space as the camera frame. Find the min/max vertex coordinates of the mesh visible in the frame, or None if the mesh is not in view. """ matrix = cam_object.matrix_world.normalized().inverted() depsgraph = bpy.context.evaluated_depsgraph_get() #Looks like this is only if modifiers and animations have been applied, it then updates this, based on the frame it is at. The scene will be updated based on the dependency graph mesh_eval = mesh_object.evaluated_get(depsgraph) # A new mesh data block is created, using the inverse transform matrix to undo any transformations mesh = mesh_eval.to_mesh() #Simply keep it like this, you do not need to specify arguments mesh.transform(mesh_object.matrix_world) mesh.transform(matrix) camera = cam_object.data # Get the world coordinates for the camera frame bounding box, before any transformations frame = [-v for v in camera.view_frame(scene=scene)[:3]] #What this does, is that it has the four corners of the camera box in world coordinates before object transformation. However, for this purpose we only need 3, top left, top right and bottom left. #For perspective camera you need to do a transformation camera_persp = camera.type != 'ORTHO' lx =  ly =  for v in mesh.vertices: #so you loop over all the vertices of the object (for cube 8) co_local = v.co #Extract locations of the object in the object reference frame z = -co_local.z #Object in front of the camera has a negative z axis officially, so you now make it positive, to follow the convention that negative z means behind the camera. if camera_persp: if z == 0.0: lx.append(0.5) ly.append(0.5) if z <= 0.0: """ Vertex is behind the camera; ignore it. """ continue else: # Perspective division - I think this makes it into homogeneous coordinates (divide by homogeneous w component) frame = [(v / (v.z / z)) for v in frame] #v.z is the camera frame coordinates world, so you loop through frame (line 46) and then #I think this decides on the size of the camera frame in world coordinates min_x, max_x = frame.x, frame.x min_y, max_y = frame.y, frame.y # max_x - min_x is the width, so you find the vertex location, minus the min location (the size of image), therefore x always positive in image reference frame. # max_y - min_y is the height of the camera frame (image) and then with origin of system in top left corner you will always have a positive y coordinate if in the image. This process is done for each vertex. This is normalized (dividing by width or height) x = (co_local.x - min_x) / (max_x - min_x) y = (co_local.y - min_y) / (max_y - min_y) #So you append the x,y location of the vertex in the image reference frame lx.append(x) ly.append(y) mesh_eval.to_mesh_clear() """ Image is not in view if all the mesh verts were ignored """ if not lx or not ly: return None #What this does is you check all the vertices and then you calculate the image frame coordinate corresponding to it. min_x = clamp(min(lx), 0.0, 1.0) max_x = clamp(max(lx), 0.0, 1.0) min_y = clamp(min(ly), 0.0, 1.0) max_y = clamp(max(ly), 0.0, 1.0) # """ Image is not in view if both bounding points exist on the same side """ # if min_x == max_x or min_y == max_y: # return None # You need this to transform the normalized coordinates to the render output size """ Figure out the rendered image size """ r = scene.render fac = r.resolution_percentage * 0.01 dim_x = r.resolution_x * fac dim_y = r.resolution_y * fac """ Image is not in view if both bounding points exist on the same side """ if round((max_x - min_x) * dim_x) == 0 or round((max_y - min_y) * dim_y) == 0: return None #You scale the image coordinates based on the size of the image.. return ( round(min_x * dim_x), # X round(dim_y - max_y * dim_y), # Y round((max_x - min_x) * dim_x), # Width round((max_y - min_y) * dim_y) # Height )
The script I use to loop over the different poses:
import bpy import os import mathutils import csv import importlib import sys dir = os.path.dirname(bpy.data.filepath) if not dir in sys.path: sys.path.append(dir) import BoundingBox import pipeline_arch importlib.reload(BoundingBox) importlib.reload(pipeline_arch) from BoundingBox import * config_poses = 'D:/cameraPoses.csv' def add_light(name,x,y,z, energy = 500, light_type = 'POINT'): # Create light datablock light_data = bpy.data.lights.new(name=name, type='POINT') light_data.energy = energy # Create new object, pass the light data light_object = bpy.data.objects.new(name=name, object_data=light_data) # Link object to collection in context bpy.context.collection.objects.link(light_object) light_object.location = mathutils.Vector((x,y,z)) def add_camera(name, focal_length, x,y,z, q0 = 0, q1 = 0, q2 = 0, q3 = 0): # Create camera block cam_data = bpy.data.cameras.new(name) cam = bpy.data.objects.new(name=name,object_data=cam_data) # Link object to collection in context bpy.context.collection.objects.link(cam) #Setting the camera parameters cam.data.lens = focal_length cam.data.type = 'PERSP' #Place the camera cam.rotation_mode = 'QUATERNION' cam.location = mathutils.Vector((x,y,z)) cam.rotation_quaternion = mathutils.Quaternion((q0,q1,q2,q3)) bpy.context.scene.camera = cam def render_settings(resolution_x = 1024, resolution_y = 1024, image_type = 'PNG', color_mode = 'RGB'): bpy.context.scene.render.resolution_x = resolution_x bpy.context.scene.render.resolution_y = resolution_y bpy.context.scene.render.image_settings.file_format = image_type bpy.context.scene.render.image_settings.color_mode = color_mode #Load the camera poses with open(config_poses, 'r', newline = '') as csvfile: ofile = csv.reader(csvfile, delimiter = ',') next(ofile) rows = (r for r in ofile if r) verts = [[float(i) for i in r] for r in rows] #Create the objects bpy.ops.mesh.primitive_monkey_add(location = (0.0,0.0,0.0)) bpy.context.object.name = "Monkey" bpy.context.object.scale = (2,2,2) mesh_object = bpy.data.objects['Monkey'] #Running the script over the different poses with the desired settings render_settings(512,512,'PNG','RGB') #light = add_light('Light',20,1,1,5000, 'POINT') file_boundingbox = os.path.join('D:/', 'bounding_box.txt') for pose in range(len(verts)): scene = bpy.context.scene cam = add_camera('Camera',50,verts[pose],verts[pose],verts[pose],verts[pose],verts[pose],verts[pose],verts[pose]) cam = bpy.context.scene.camera light = add_light('Light',verts[pose],verts[pose],verts[pose], 2000, 'POINT') #mesh_object = bpy.data.objects['Monkey'] file = os.path.join('D:/', 'imageMonkey_' + str(pose)) print(camera_view_bounds_2d(scene, cam, mesh_object)) #write_bounds_2d(file_boundingbox, scene, cam, mesh_object) bpy.context.scene.render.filepath = file bpy.ops.render.render(write_still=True) #Clean the scene bpy.ops.object.select_all(action = 'SELECT') bpy.ops.object.delete() #Clean the data data = bpy.data for camera in data.cameras: data.cameras.remove(camera, do_unlink = True)
x,y,z,q0,q1,q2,q3 20,0,0,0.5,0.5,0.5,0.5 0,20,0,0.0,0.0,-0.707107,-0.707107 -20,0,0,0.5,0.5,-0.5,-0.5 0,-20,0,0.707107,0.707107,0.0,0.0 0,0,20,0.707107,0.0,0.0,0.707107 0,0,20,1,0,0,0 0,0,-20,0,0.707107,-0.707107,0.0 0,0,-20,0,0.382683,-0.923880,0.0 10,-20,0,0.707107,0.707107,0.0,0.0