I have a scene with a many objects and one camera.

I would like to select the objects that the camera sees, even partially.

Is there a way to do that in Python that accounts for the objects bounds?

Note, I know that KD trees and Octrees are used for the lookup. There is mathutils.kdtree but the example only suggests how to query objects close to a point. I'm not familiar with the math for combining camera frustrum and binary trees lookup.

  • $\begingroup$ You might render the scene with shadeless materials and identify the visible objects by their color. In the next blender version there will be a new module which allows offscreen rendering. $\endgroup$ Commented Jan 19, 2016 at 14:05
  • $\begingroup$ yes there may be a way and some decent workarounds, but it's not clear how you produce the mesh for splines: bevel / extrude? by another spline as taper? by mesh deformed by spline? - Here I see an issue, cause you may need to convert to mesh temporary anyway. After all you wanna see if the Resulting Mesh is in view and not the Spline line. So how are the splines/meshes or lod defined? (for mesh I have sort of a solution, but I'm not sure how to deal with splines) $\endgroup$
    – o.g.
    Commented Jan 20, 2016 at 12:57
  • $\begingroup$ I'm thinking about doing multiple kdtree searches in space along the vector that follows where the camera points to. First I could have a small hidden sphere (not an object, a math one) that follows the camera and everything colliding with it is meshed. That way nearby objects are meshed and then with several lookups in space a proper intervals, I find other objects in the camera field of view. $\endgroup$
    – nantille
    Commented Jan 20, 2016 at 15:12
  • $\begingroup$ You could use a kdtree if you have many many objects and so you filter a big part of them at first. Then perform the refined check only on a smaller list (as in example I gave). - For large amounts vs speed it makes sense $\endgroup$
    – o.g.
    Commented Jan 21, 2016 at 10:56
  • $\begingroup$ The only advantage of using a tree structure here would be if you have a static scene and want to move the camera around it. - In that case you could build a tree and re-use it for lookups. $\endgroup$
    – ideasman42
    Commented Jan 21, 2016 at 21:43

3 Answers 3


There are a few different ways to handle this, you could...

  • project into 2d space, then detect which objects are in the 2d frame.
  • calculate the camera bounds in 3d and detect which objects are inside it.

Here is some sample code that demonstrates the second method. It uses a set of planes (4 or 5 for orthographic cameras) and finds all objects that have any part of their bounding boxes within the planes:

Note, this isn't all that elegant, we could for example have a single function that intersects 2 sets of planes - one for the camera - another for the bound-box. However this is at least working and can give you some starting point - others may like to improve or post a method that uses projection.

def camera_as_planes(scene, obj):
    Return planes in world-space which represent the camera view bounds.
    from mathutils.geometry import normal

    camera = obj.data
    # normalize to ignore camera scale
    matrix = obj.matrix_world.normalized()
    frame = [matrix @ v for v in camera.view_frame(scene=scene)]
    origin = matrix.to_translation()

    planes = []
    from mathutils import Vector
    is_persp = (camera.type != 'ORTHO')
    for i in range(4):
        # find the 3rd point to define the planes direction
        if is_persp:
            frame_other = origin
            frame_other = frame[i] + matrix.col[2].xyz

        n = normal(frame_other, frame[i - 1], frame[i])
        d = -n.dot(frame_other)
        planes.append((n, d))

    if not is_persp:
        # add a 5th plane to ignore objects behind the view
        n = normal(frame[0], frame[1], frame[2])
        d = -n.dot(origin)
        planes.append((n, d))

    return planes

def side_of_plane(p, v):
    return p[0].dot(v) + p[1]

def is_segment_in_planes(p1, p2, planes):
    dp = p2 - p1

    p1_fac = 0.0
    p2_fac = 1.0

    for p in planes:
        div = dp.dot(p[0])
        if div != 0.0:
            t = -side_of_plane(p, p1)
            if div > 0.0:
                # clip p1 lower bounds
                if t >= div:
                    return False
                if t > 0.0:
                    fac = (t / div)
                    p1_fac = max(fac, p1_fac)
                    if p1_fac > p2_fac:
                        return False
            elif div < 0.0:
                # clip p2 upper bounds
                if t > 0.0:
                    return False
                if t > div:
                    fac = (t / div)
                    p2_fac = min(fac, p2_fac)
                    if p1_fac > p2_fac:
                        return False

    ## If we want the points
    # p1_clip = p1.lerp(p2, p1_fac)
    # p2_clip = p1.lerp(p2, p2_fac)        
    return True

def point_in_object(obj, pt):
    xs = [v[0] for v in obj.bound_box]
    ys = [v[1] for v in obj.bound_box]
    zs = [v[2] for v in obj.bound_box]
    pt = obj.matrix_world.inverted() @ pt
    return (min(xs) <= pt.x <= max(xs) and
            min(ys) <= pt.y <= max(ys) and
            min(zs) <= pt.z <= max(zs))

def object_in_planes(obj, planes):
    from mathutils import Vector

    matrix = obj.matrix_world
    box = [matrix @ Vector(v) for v in obj.bound_box]
    for v in box:
        if all(side_of_plane(p, v) > 0.0 for p in planes):
            # one point was in all planes
            return True

    # possible one of our edges intersects
    edges = ((0, 1), (0, 3), (0, 4), (1, 2),
             (1, 5), (2, 3), (2, 6), (3, 7),
             (4, 5), (4, 7), (5, 6), (6, 7))
    if any(is_segment_in_planes(box[e[0]], box[e[1]], planes)
           for e in edges):
        return True

    return False

def objects_in_planes(objects, planes, origin):
    Return all objects which are inside (even partially) all planes.
    return [obj for obj in objects
            if point_in_object(obj, origin) or
               object_in_planes(obj, planes)]

def select_objects_in_camera():
    from bpy import context
    scene = context.scene
    origin = scene.camera.matrix_world.to_translation()
    planes = camera_as_planes(scene, scene.camera)
    objects_in_view = objects_in_planes(scene.objects, planes, origin)

    for obj in objects_in_view:

if __name__ == "__main__":
  • $\begingroup$ 2 questions: 1. isn't this what basically bpy_extras.object_utils.world_to_camera_view is giving as a factor (0-1 inside screen)? and 2. the bbox should be a bit larger (with a tolerance or so) so that it considers the result with bevel radius? or is that already there ? $\endgroup$
    – o.g.
    Commented Jan 21, 2016 at 11:04
  • $\begingroup$ 1) its just a different way to test - using planes instead of projection, wouldn't say its the same though. 2) As for padding the bound-box, curve bevel is taken into account, but perhaps render subdivision levels change the bound-box slightly - so you may want to do this, it depends on your exact use case - YMMV. $\endgroup$
    – ideasman42
    Commented Jan 21, 2016 at 11:43
  • $\begingroup$ ok, they are not the same and it may be better to clearly check by planes. As for padding, op stated that "he will not mesh the curves" unless in view, so we can assume the bbox goes only for the naked spline. - Thus the need for padding bbox or camera planes (probably simpler) to compensate for a "future" mesh, slightly larger than the initila bbox. $\endgroup$
    – o.g.
    Commented Jan 21, 2016 at 12:06
  • $\begingroup$ very cool answers both of you, I'll try all this between today and tomorrow, thank you ! $\endgroup$
    – nantille
    Commented Jan 21, 2016 at 13:24
  • $\begingroup$ There was an error in clip_segment_in_planes, now fixed. $\endgroup$
    – ideasman42
    Commented Jan 23, 2016 at 1:27

I want to add how I solved this problem for Blender 3.4.

My solution is based on: ​List all ​occluded objects from camera.

In addition to the solution above, I check how many pixels are dedicated to each object. So you could decide afterwards if it occupies sufficient space on the rendered image or not.

def occlusion_test(scene, depsgraph, camera, resolution_x, resolution_y):
    # get vectors which define view frustum of camera
    top_right, _, bottom_left, top_left = camera.data.view_frame(scene=scene)

    camera_quaternion = camera.matrix_world.to_quaternion()
    camera_translation = camera.matrix_world.translation

    # get iteration range for both the x and y axes, sampled based on the resolution
    x_range = np.linspace(top_left[0], top_right[0], resolution_x)
    y_range = np.linspace(top_left[1], bottom_left[1], resolution_y)

    z_dir = top_left[2]

    hit_data = set()

    # iterate over all X/Y coordinates
    pixel_count_dict = {}
    for x in x_range:
        for y in y_range:
            # get current pixel vector from camera center to pixel
            pixel_vector = Vector((x, y, z_dir))
            # rotate that vector according to camera rotation

            is_hit, _, _, _, hit_obj, _ = scene.ray_cast(depsgraph, camera_translation, pixel_vector)

            if is_hit:
                if hit_obj.name not in pixel_count_dict:
                    pixel_count_dict[hit_obj.name] = 0
                pixel_count_dict[hit_obj.name] += 1

    return hit_data, pixel_count_dict

context = bpy.context
res_ratio = 0.25
res_x = int(context.scene.render.resolution_x * res_ratio)
res_y = int(context.scene.render.resolution_y * res_ratio)

visible_objs, pixel_count_dict = occlusion_test(context.scene, context.evaluated_depsgraph_get(), context.scene.objects['Camera'],res_x, res_y)

First I will consider here:

  • A list of objects, a scene, a camera, assuming you already defined these
  • we only care about mesh or bezier curves (I include mesh for testing, you can skip it)
  • we consider all splines inside a curve object (you can count only for first if the case)

  • a tolerance: this helps with bevel. so we consider a bit before the spline point is in view

  • a LOD, just for example (make your rules there)

Note that the tolerance here has to be related to a 0 to 1 factor and I just give a 0.03 out of the blue. this can be computed in relation to distance and a radius if u need more precision.

The steps are:

  • for each object
    • create a list of points, mesh points or by subdividing curves
    • test if the point is inside rendered area, by screen factor 0 to 1
    • create 0 if off screen or LOD (1, 2, 3 ...)
  • produce a list of integers that store the state of each object

Key moments:

  • for splines use mathutils.geometry.interpolate_bezier to subdivide it (we use points, so we need segments to be little here)
  • for on screen use bpy_extras.object_utils.world_to_camera_view that gives a factor related to screen (0-1 for on screen, rest is off)

The code example would then be something like this:

# check if mesh or spline points are in view, (o.g. variant 1, 2016)

from mathutils.geometry import interpolate_bezier
from bpy_extras.object_utils import world_to_camera_view

def pointsOfMesh_or_subdivideSpline(object):
    points = []

    if getattr(object, 'type', '') == 'MESH':
        points = [v.co for v in object.data.vertices]

    elif getattr(object, 'type', '') ==  "CURVE":
        splines = object.data.splines
        for spline in splines:

            if getattr(spline, 'type', '') == 'BEZIER':
                resolution = spline.resolution_u + 1 
                last = 0 if spline.use_cyclic_u else 1

                lenP = len(spline.bezier_points)
                for i, p1 in enumerate(spline.bezier_points):
                    if i < lenP - last:
                        p2 = spline.bezier_points[(i + 1) % lenP]

                        knot1 = p1.co
                        handle1 = p1.handle_right
                        handle2 = p2.handle_left
                        knot2 = p2.co
                        _points = interpolate_bezier(knot1, handle1, handle2, knot2, resolution)
    return points

def testInView(coord, tolerance):
    pointInView = False
    z = -1

    if scene is not None and camera is not None and coord is not None:
        xFactor, yFactor, z = world_to_camera_view(scene, camera, coord)

        # add this if you use ortho !!!:
        #if camera.data.type != "PERSPECTIVE":
        #    sx, sy = camera.data.shift_x, camera.data.shift_y
        #    xFactor, yFactor = xFactor - 2 * sx, yFactor - 2 * sy

        # !! tolerance can be computed with above z and radius or so
        if -tolerance < xFactor < 1 + tolerance and  -tolerance < yFactor < 1 + tolerance and z > 0:
            pointInView = True

    return pointInView, z

objectsInViewList = []
LOD2limit = max(LOD2limit, LOD3limit)   #clamp lod 2 to lod 3

# mesh points or spline subdivision points with a tolerance
for object in ObjectList:
    if object is None or getattr(object, 'type', '') not in ['MESH', 'CURVE']:

        matrix = object.matrix_world
        zlist = []

        # see if any point is in view
        points = pointsOfMesh_or_subdivideSpline(object)
        for point in points:
            pointInView, z = testInView(matrix * point, tolerance)
            if pointInView: zlist.append(z)

        # these LODs are just an example
        if zlist == []: LOD = 0
        elif min(zlist) < LOD3limit: LOD = 3
        elif LOD3limit <= min(zlist) < LOD2limit: LOD = 2
        else: LOD = 1


so you get a list of 0 (hide) or LOD for each object in list

Again, note that tolerance is related to screen factor, so I just use a 0.03 here, but you can further calculate it related to z

Testing, just to see

All inputs there, I assume you have defined somehow. (I intentionally use None and lamp to see..)

Compare the result 0,1,2, list with the obvious state of objects Note that bezier 001 is slightly off screen, but I still get a 1, cause of the tolerance.

enter image description here

Note that I use Animation Nodes/ script node for convenience here, skips a lot of formalities, registering etc and I can use bogus obj list etc, plus it updates real time.

However, just to illustrate how the code works

  • $\begingroup$ Checking if the points are in the view isn't totally reliable, since an edge may span the view but not have any vertices in it, or a plane may fill the camera completely (where no edges or vertices intersect). $\endgroup$
    – ideasman42
    Commented Jan 21, 2016 at 10:56
  • $\begingroup$ I know, that is why I have asked for clarification on how the splines mesh is done. Being done with bevel, I can use enough subdivisions and with a certain tolerance, that covers for bevel radius, it should work for the specific case. Also, the LOD is there so should cover for too close objects (I did not do that in the example) - Otherwise, I would have build the 4 planes for camera view sides and check the normals for i/o. That is basically what you did :), so no point in redoing $\endgroup$
    – o.g.
    Commented Jan 21, 2016 at 11:59
  • $\begingroup$ I would see the problem of no-vertice but edge intersection appear only at close-ups. To deal with this I could have the curves close to the camera meshed (as in my comment above). In this case, when in doubt, it's better to have more meshes than too little. $\endgroup$
    – nantille
    Commented Jan 21, 2016 at 13:44
  • $\begingroup$ @ideasman42 I tried to use your function but I caped getting error when multiplying the matrix_world and bound_box vector. first I tried to wrap it up with numpy array since the operator @ wasn't supported and after that I received a dimensionality error since the matrix holds shape of 4x4 and the vector shape of 3. I'm using blender 2.79b $\endgroup$
    – AvivSham
    Commented Jan 28, 2020 at 6:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .