What does `world_to_camera_view` depend on?

Question

In a small part of a much larger repository of scripts and sequence of related .blend files, I use bpy_extras.object_utils.world_to_camera_view repeatedly, invoked like:

cam_coord_vec = world_to_camera_view(bpy.context.scene, cam, obj.matrix_world @ final_co)

Unfortunately, I'm seeing some non-determinism. In a print statement like the following (where v is a mathutils.Vector()), everything (at least, all non-callables; I have not yet called and checked the output of each callable in the final dir) is identical except cam_coord_vec:

print(f">>>> {v.co} # {v.index} -- {final_co} -- {obj.matrix_world @ final_co} -- {cam_coord_vec}")                                    
print(f"#2>>>> \t {cam.location} {cam.data.shift_x} {cam.data.shift_y} {cam.data.ortho_scale} {cam.rotation_euler}")                   
print(f"#3>>>> \t {cam.data.sensor_height} {cam.data.sensor_width} {cam.data.clip_start} {cam.data.clip_end}")                         
print(f"#4>>>> \t {cam.data.display_size} {cam.data.lens}")                                                                            
print(f"#5>>>  \t {[getattr(cam.data, d) for d in dir(cam.data)]}") 
print(f"#6>>>  \t {bpy.context.scene.name}")

There is quite a lot going on in both the code and in the .blend files -- e.g., many places where the code might trigger a scene update, liberal use of shapekeys driven by custom property drivers, and I'm developing in an old Blender version, v2.83.2. Maybe as a side effect of this, I'm struggling to isolate the cause, or to build a minimal working example of the problem.

The world_to_camera_view_docs say it "Takes shift-x/y, lens angle and sensor size into account as well as perspective/ortho projections.", but it's a bit unclear to me how to check that all of this data is identical between the two runs, or if I've covered all relevant items above.

Here's an example of the output of the above, from a first run:

#~ >>>> <Vector (1.5000, 0.5000, 0.0500)> # 0 -- <Vector (1.5000, 0.5000, 0.9775)> -- <Vector (0.5000, 0.5000, 0.9766)> -- <Vector (8.2782, -4.0987, 19.8199)>
#~ #2>>>>      <Vector (2.6106, -12.6106, 18.2431)> 0.0 0.0 1.0 <Euler (x=0.7854, y=0.0000, z=0.7854), order='XYZ'>
#~ #3>>>>      24.0 36.0 0.10000000149011612 1000.0
#~ #4>>>>      1.0 50.0
#~ #5>>>       [None, 'bpy.types', (), 0.6911112070083618, 0.6911112070083618, 0.4710899591445923, None, <bpy_func Camera.animation_data_clear()>, <bpy_func Camera.animation_data_create()>, bpy.data.cameras['Camera'].background_images, <bpy_struct, Struct("Camera")>, 1000.0, 0.10000000149011612, <bpy_func Camera.copy()>, bpy.data.cameras['Camera'].cycles, 1.0, bpy.data.cameras['Camera'].dof, <bpy_func Camera.evaluated_get()>, False, False, False, 50.0, 'MILLIMETERS', None, <bpy_func Camera.make_local()>, 'Camera', 'Camera', bpy.data.cameras['Camera'], 1.0, <bpy_func Camera.override_create()>, None, 0.5, None, bpy.data.cameras['Camera']...Struct, 'AUTO', 24.0, 36.0, 0.0, 0.0, False, False, False, False, False, False, False, False, False, False, False, False, True, False, False, False, bpy.data.cameras['Camera'].stereo, False, 'ORTHO', <bpy_func Camera.update_tag()>, False, <bpy_func Camera.user_clear()>, <bpy_func Camera.user_of_id()>, <bpy_func Camera.user_remap()>, 1, <bpy_func Camera.view_frame()>]
#~ #6>>>       Scene

And from a second:

#~ >>>> <Vector (1.5000, 0.5000, 0.0500)> # 0 -- <Vector (1.5000, 0.5000, 0.9775)> -- <Vector (0.5000, 0.5000, 0.9766)> -- <Vector (-2.3284, 1.3752, 19.8199)>
#~ #2>>>>      <Vector (2.6106, -12.6106, 18.2431)> 0.0 0.0 1.0 <Euler (x=0.7854, y=0.0000, z=0.7854), order='XYZ'>
#~ #3>>>>      24.0 36.0 0.10000000149011612 1000.0
#~ #4>>>>      1.0 50.0
#~ #5>>>       [None, 'bpy.types', (), 0.6911112070083618, 0.6911112070083618, 0.4710899591445923, None, <bpy_func Camera.animation_data_clear()>, <bpy_func Camera.animation_data_create()>, bpy.data.cameras['Camera'].background_images, <bpy_struct, Struct("Camera")>, 1000.0, 0.10000000149011612, <bpy_func Camera.copy()>, bpy.data.cameras['Camera'].cycles, 1.0, bpy.data.cameras['Camera'].dof, <bpy_func Camera.evaluated_get()>, False, False, False, 50.0, 'MILLIMETERS', None, <bpy_func Camera.make_local()>, 'Camera', 'Camera', bpy.data.cameras['Camera'], 1.0, <bpy_func Camera.override_create()>, None, 0.5, None, bpy.data.cameras['Camera']...Struct, 'AUTO', 24.0, 36.0, 0.0, 0.0, False, False, False, False, False, False, False, False, False, False, False, False, True, False, False, False, bpy.data.cameras['Camera'].stereo, False, 'ORTHO', <bpy_func Camera.update_tag()>, False, <bpy_func Camera.user_clear()>, <bpy_func Camera.user_of_id()>, <bpy_func Camera.user_remap()>, 1, <bpy_func Camera.view_frame()>]
#~ #6>>>       Scene

The two appear to be identical, except that the returned world_to_camera_view vector is <Vector (8.2782, -4.0987, 19.8199)> in the first case, and <Vector (-2.3284, 1.3752, 19.8199)> in the second.

Are there other inputs world_to_camera_view depends on that I could print, to look for the cause of this inconsistency?

Answer 1

The source code for world_to_camera turns out to be in Python and very short. Since it is implemented in Python, I directly I inserted some extra debugging statements inside its source code and restarted Blender to induce it to re-load the updated object_utils.py, then re-invoked my code twice. The very first co_local calculation in line 255 returned <Vector (7.7782, -4.5987, -19.8199)> in a behavior-as-expected run, but <Vector (-2.8284, 0.8752, -19.8199)> in an inconsistent-and-unexpected-behavior run.

It turns out that world_to_camera_view depends on the camera's matrix_world (which should have occurred to me sooner!). A common problem with complex Python code in Blender is that matrix_world does not properly update unless a scene update is triggered, which can in simple cases be resolved by explicitly re-computing matrix_world from matrix_basis, and this seems sufficient to correct the issue in the case of my code (I inserted this modification directly in the Blender source, inside the world_to_camera_view function, but bpy_types.Object.matrix_world is writable, so it may be safer to explicitly overwrite the object attribute before invoking world_to_camera_view). In my case, the camera's matrix_world not being up-to-date generated apparent non-determinism, owing to Blender updates happening at inconsistent points (or, at least, at points that are difficult to track, given the complexity of the code and that it can be hard to remember which Blender Python operations do/don't induce a scene update). (Based on the world_to_camera_view code, the camera.view_frame calculation could in principle be another place that non-obvious side effects creep in like this, but I didn't observe this in my case.)