# How to find pixels which are getting occluded in the next frame?

I have 3D model of a scene with several objects that are transformed from one keyframe to another one (translation, rotation, scaling). This scene will be rendered as animation with write_still=True so the final results are images per each frame. I am interested in finding pixels that are visible in one frame but are occluded in the next frame. A simple example could be something like this:

First frame ----------- > Second frame

In this example, the colorful plane is fully visible in the first frame but is getting behind the gray square in the second frame, so not only the bottom left corner of it is occluded now, but part of the background near the gray square is also occluded by the colorful square. In the next image, the occluded areas are highlighted white. I would like to have a binary mask that shows which pixels from the first frame will be occluded in the next frame. The final goal is to do this by Python interface, but any suggestion to solve this is very appreciated.

Please note that objects are 3D, they might rotate or scale too. Lighting will change and the camera motion is also involved. A more complete example is as follows:

First frame

Second frame

Occlusion map

• The difference of two images (image1 - image2) is supported in the Compositor of Blender. Images can be compared by the standard math operations less >, greater <, equal == et cetera.Then you can threshold the result by various means including a color ramp (mapping color) . Perhaps this is a starting point. You may want to search the Blender API for image functions. Apr 22 '20 at 4:54
• What is the larger context of you work? Is the work post production, part of a pipeline, ...? Realtime? If you are doing something very elaborate please state so if possible. Apr 22 '20 at 5:00
• Thank you @atomicbezierslinger for this point, but actually, in a more realistic example, differencing images won't work. The camera will move around, lighting will change, etc. I need to kind of keep track of visible vertices to the camera or something! Apr 22 '20 at 5:03
• Using the speed vectors, I am finding the motion vector for each pixel between frames, now I need to determine which pixels are not visible in the second frame. Apr 22 '20 at 5:08
• Again the compositor may help. To a large degree you can use Object ID ,a number say, for each object, lets say N1 and N2. Then the pixel itself can be examined for those IDs, N1 and N2. The object ID is a feature of Blender. You may want to add your additional variations to the questions itself. Apr 22 '20 at 5:17