How to mask animated objects so they aren't visible past a certain area?

Question

I am animating objects that will be rendered with video footage in the compositor. My difficulty is how to apply a mask so that the objects disappear when they expand past a certain point.

Objects within circle mesh should be visible

Objects outside of circle mesh should be invisible

Current nodes set-up for object materials:

What masking options are at my disposal to achieve this effect, and how can I apply them?

Thanks for any insights.

Answer 1

There are a good many ways of doing this, but here are a few:

Mask editor

1. In the UV image editor, set the mode to Mask and press New:

   

2. Press ⇧ ShiftA>Circle to add a circle mask, the position it with G:

   

3. Use the mask to control the alpha over factor in the compositor:

   

   You could even feather or blur the mask to make the objects fade out instead of disappearing abruptly.

Object/material id mask

You could also use the circle object in the 3D view to do this, using ID masks (see my answer here).

Note that this will only work if you have the circle object visible in the render and filled with a face, or use multiple renderlayers.

Holdout material:

If you want to replace part of a scene with transparency, you can use an object with a holdout shader to make that object render as alpha transparency:

Boolean modifier

This is probably not a good option in this case, but this can be useful when you need to "mask" whole objects:

Collision object

If the object is in a particle system, you can use a sphere with collision physics, set to kill particles:

No particles will be able to escape the sphere.