Is there a way to know (programmatically) how many objects are visibile in a rendering?

Question

i'm new here, i browsed a bit before posting but i haven't found any solution yet. I'm writing a small script to create several random images with spheres of different sizes and i'm looking for a way to know how many of them are visible in a render. The final idea is to have a dataset to test some computer vision algorithms such as that i have the number of objects present in each image. I'm attaching the script so far.

#!/usr/bin/python2.7
import bpy
import bpy_extras
import random
import math

def makeMaterial(name, diffuse, specular, alpha):
    mat = bpy.data.materials.new(name)
    mat.diffuse_color = diffuse
    mat.diffuse_shader = 'LAMBERT'
    mat.diffuse_intensity = 1.0
    mat.specular_color = specular
    mat.specular_shader = 'COOKTORR'
    mat.specular_intensity = 0.5
    mat.alpha = alpha
    mat.ambient = 1
    return mat

def setMaterial(ob, mat):
    me = ob.data
    me.materials.append(mat)

def createSphere(loc, siz, material):
    origin = (0,0,0)
    bpy.ops.mesh.primitive_uv_sphere_add(location=loc, size=siz)
    #bpy.ops.transform.translate(value=(1,0,0))
    #bpy.context.object.location = location
    setMaterial(bpy.context.object, material)



def createScene():
    #Cleaning the scene
    bpy.ops.object.select_by_type(type='MESH')
    bpy.ops.object.delete()

    #Create a material
    materials = (makeMaterial('Red',(1,0,0),(1,1,1),1),
                 makeMaterial('Green',(0,1,0),(1,1,1),1),
                 makeMaterial('Blu',(0,0,1),(1,1,1),1))

    rand = random.randrange(50)
    print ("Number of spheres %s" % rand)
    for i in range(rand):
        location = (random.random()*10-5, random.random()*10-5, 0)
        size = random.random()
        createSphere(location, size, materials[int(random.randrange(3))])

def render():
    step_count = 32

    for step in range(0, step_count):
        cam.rotation_euler[2] = math.radians(step * (360.0 / step_count))

        bpy.data.scenes["Scene"].render.filepath = '/home/daniele/VR/vr_shot_%d.jpg' % step
        bpy.ops.render.render( write_still=True )

#Set a seed to make reproducible
random.seed(32)
cam = bpy.data.objects['Camera']
createScene()

render()

Answer 1

You can give each object a unique object ID ("pass index" in the object properties).These can be output as render pass; each pixel will have the ID of the object that's visible there. This will not only give you a count of the number of objects (just count the number of unique values) but also a ground truth for segmentation & shape recognition.

This Python script will give each selected object another ID:

import bpy

for nr, obj in enumerate(bpy.context.selected_objects):
    obj.pass_index = nr + 1  # reserve pass index 0 for "no object".

For more info, see the ID Mask Node section of the Blender manual.

Answer 2

I found a way to figure out which objects are seen by the camera. It might be a clumsy one, as I do something manually which maybe Blender hast a better function for already, but it is unknown to me at least. My way uses the following approach:

first, get the location of the camera in world space. Then, get reference points in 3D which describe the boundaries of the render border. Then, segment the view in slices of a length of n pixels (in regards to render resolution), and cast a ray from the camera in this direction. If there is an Object hit, add it to a list. Later on, I can iterate over this list and select all of the objects which were hit by the ray.

This will only collect objects which are not occluded by other objects.

The following script has a few parameters to play with: n is the step size in pixels, if you set it to 1, every pixel receives a ray, setting it to 50 fires a ray every 50 pixels, etc. maxDist is the maximum length of the ray in Blender units. showEmpties is a debugging parameter. Only set this if n is bigger than 50, otherwise you'll wait forever. If you look through the camera, an empty will be placed on every spot where a ray is fired.

My script uses the current renderable camera to evaluate the scene, so make sure there is one set. Copy the following script to your script editor and run it:

import bpy

C = bpy.context

# fire a ray every n pixels
n = 50
maxDist = 100

# for debugging, the ray direction can be visualized as empties
showEmpties = False

# get the origin to fire a ray
startingPoint = C.scene.camera.location

# get the direction to fire a ray to
# the corners of the render border are:
#
#  y1    x1    
#  y2    x2    

x1, x2, y1, y2 = [C.scene.camera.matrix_world * x - C.scene.camera.location for x in C.scene.camera.data.view_frame(C.scene)]

print(x1, x2, y1, y2)
bpy.ops.object.select_all(action='DESELECT')

# iterate over the render resolution in x and y, using every nth pixel
hitObjects=[]
for x in range(int(C.scene.render.resolution_x / n) + 1):
    newVecx = (y2 - x1) * ( x * n / C.scene.render.resolution_x) 
    # print (newVecx)

    for y in range(int(C.scene.render.resolution_y / n) + 1):
        newVecy = (x2 - x1) * ( y * n / C.scene.render.resolution_y) 

        # fire the ray and see what is hit
        res = C.scene.ray_cast(C.scene.camera.matrix_world.to_translation(), x1 + newVecx + newVecy, distance=maxDist)
        if res[0]:

            # an object is hit, add it to the list if it is not present yet
            if res[4] not in hitObjects:
                hitObjects.append(res[4])

        if showEmpties:
            bpy.ops.object.empty_add(radius=0.001, location=C.scene.camera.matrix_world.to_translation() + x1 + newVecx + newVecy)

for x in hitObjects:
    x.select=True