Compositing: How the resize the scene to fit the matte?

Question

I've a scene and an image matte built from an image with a green rectangle depicting a hole. I'd like to have the camera view resized to fit the hole.

The image with the hole may change, the hole may be sized and located differently (no input about that, only the green color of the pixels).

I have used a Transform node at the location indicated with ?. I have to adjust the offsets and the scale factor visually to position the camera view exactly in the hole. I'm looking for something automatic if this exists, or the most effective solution in any case.

I'm a true beginner.

Assumptions:

• This final result is an image, not an animation.
• The green rectangle and the camera have the same W/H ratio.
• The image appears in the foreground, normal to the camera view axis, in the composited result. No perspective is involved.

Answer 1

Here are some mathematics governing the transform:

• W = pixel width of final scene
• H = pixel height of final scene
• x1,y1 = min coordinate of green screen
• x2,y2 = max coordinate of green screen

You will feed the inside image into a scale node and then a translate node.

The values for the scale node will be

• scale_x = (x2-x1)/W
• scale_y = (y2-y1)/H

The values for the translate node will be

• translate_x = (x2+x1-W)/2
• translate_y = (y2+y1-H)/2

If you want to use the Relative checkbox, then divide those values by W and H respectively.

Keep in mind that blender's coordinate system is mathematical, not page coordinates, so max_y is at the top of the screen. (page coordinates usually put y=0 at the top instead)

This math applies after you have determined the coordinates of the green rectangle.

Here is an operator that will let you pick a file and a color and the operator will rewrite your compositor node tree to mix the current 3D scene with the matte image. Be warned, this can be misled by stray matching pixels, so if your source images aren't properly constructed, you are boned:

http://web.purplefrog.com/~thoth/blender/python-cookbook/find-green-rectangle.html

bl_info = {
    "name" : "Composite into Matte image",
    "description": "An operator that configures the compositor to substitute the 3D scene into a chroma-keyed subset of the matte image chosen by the operator",
    "author" : "Robert Forsman <[email protected]>",
    "version": (0,1),
    "blender": (2,74,0),
    #"location": "????",
    "warning": "",
    #"wiki_url": "http://wiki.blender.org/index.php/Extensions:2.6/Py/Scripts/???",
    "category": "Node",
}
import bpy
from math import *

def get_image(path):
    matches = [ img for img in bpy.data.images if img.filepath==path]
    if len(matches)>0:
        return matches[0]
    rval = bpy.data.images.load(path)
    return rval


def matches( c1, c2, epsilon = 0.02):

    for i in range(len(c1)):
        if i<len(c2):
            b = c2[i]
        else:
            b=1
        if epsilon < abs(c1[i]- b):
            return False
    return True

def find_green_rectangle(img, chroma_key):
    # hold down the left mouse button in the UV/Image editor window to get the values for chroma_key

    width = img.size[0]
    #print (img.pixels[:4])
    pix = img.pixels[:]
    min_x = None
    max_x = None
    min_y = None
    max_y = None
    for i in range(0,len(pix), 4):
        one_pixel = pix[i:(i + 4)]
        if matches(one_pixel, chroma_key):
            x = floor(i/4)% width
            y = floor(i/4/ width)
            if min_x is None or x<min_x:
                min_x = x
                print([ i,x,y , one_pixel])
            if max_x is None or x>max_x:
                max_x = x
            if min_y is None or y<min_y:
                min_y = y
            if max_y is None or y > max_y:
                max_y = y
        #if i%100000 ==0:
        #    print(i)
    return [ min_x, min_y, max_x, max_y, width, img.size[1]]


def convert_bounds_to_node_parameters(scn, x1, y1, x2, y2, W, H):
    translate_x = (x2+x1-W)/2 /W
    translate_y = (y2+y1-H)/2 /H
    scale_x = (x2-x1)/W
    scale_y = (y2-y1)/H

    #scale_nodes = [ n for n in scn.node_tree.nodes if n.type=='SCALE']
    # use the N-panel in the compositor nodes to give your scale node this name
    scale_nodes = [ n for n in scn.node_tree.nodes if n.name=='matte scale']
    translate_nodes = [ n for n in scn.node_tree.nodes if n.name == 'matte translate']

    scale_nodes[0].inputs[1].default_value = scale_x
    scale_nodes[0].inputs[2].default_value = scale_y

    translate_nodes[0].inputs[1].default_value = translate_x
    translate_nodes[0].inputs[2].default_value = translate_y
    translate_nodes[0].use_relative = True

def wipe_and_rebuild_compositor_nodes(scn, img, wipe=True):

    scn.use_nodes = True
    while wipe and len(scn.node_tree.nodes)>0:
        scn.node_tree.nodes.remove(scn.node_tree.nodes[0])

    nodes = scn.node_tree.nodes
    links = scn.node_tree.links

    dx = 200
    image_n = nodes.new("CompositorNodeImage")
    image_n.location = (0,0)
    image_n.image = img

    #matte_n = nodes.new("CompositorNodeColorMatte")
    #matte_n.location = (dx,0)
    #links.new(matte_n.inputs[0], image_n.outputs[0])

    scale1_n = nodes.new("CompositorNodeScale")
    scale1_n.location = (2*dx,0)
    links.new(scale1_n.inputs[0], image_n.outputs[0])
    #links.new(scale1_n.inputs[0], matte_n.outputs[0])
    if True:
        scale1_n.space = 'RENDER_SIZE'
        scale1_n.frame_method = 'STRETCH'
    else:
        scale1_n.inputs[1].default_value = scn.render.resolution_x / img.size[0]
        scale1_n.inputs[2].default_value = scn.render.resolution_y / img.size[1]

    y2=-250
    layer_n = nodes.new('CompositorNodeRLayers')
    layer_n.location = (0,y2)

    scale2_n = nodes.new("CompositorNodeScale")
    scale2_n.location = (dx,y2)
    scale2_n.name = "matte scale"
    links.new(scale2_n.inputs[0], layer_n.outputs[0])

    translate_n = nodes.new("CompositorNodeTranslate")
    translate_n.location = (2*dx, y2)
    translate_n.name = "matte translate"
    links.new(translate_n.inputs[0], scale2_n.outputs[0])

    mix_n = nodes.new("CompositorNodeAlphaOver")
    mix_n.location = (3*dx, 0)
    links.new(mix_n.inputs[1], scale1_n.outputs[0])
    links.new(mix_n.inputs[2], translate_n.outputs[0])

    output = nodes.new('CompositorNodeComposite')
    output.location = (4*dx,0)
    links.new(output.inputs[0], mix_n.outputs[0])




#
#

class CompositeSceneIntoMatte(bpy.types.Operator):
    bl_idname = "compositor.scene_into_matte"
    bl_label = "Composite Scene Into Matte"
    bl_options = {'REGISTER', 'UNDO'}

    matte_filename = bpy.props.StringProperty(name="Matte file path",
                                              description="path to the image that provides the matte image",
                                              #default="/var/tmp/matte.png",
                                              subtype='FILE_PATH')
    matte_color = bpy.props.FloatVectorProperty(
            name="Matte chroma key",
            subtype='COLOR',
            default=(0, 1, 0),
            min=0.0, max=1.0,
            description="color from the matte image that marks where the 3D scene should be overlaid"
           )

    def execute(self, ctx):
        try:
            try :
                img = get_image(self.matte_filename)
            except :
                self.report({'WARNING'}, "failed to load image from filename %r"%self.matte_filename)
                return {'FINISHED'}

            bounds = find_green_rectangle(img, self.matte_color)
            if bounds[2] is None:
                self.report({'WARNING'}, "matte color %r does not appear in image"%self.matte_color)
                return {'FINISHED'}
            wipe_and_rebuild_compositor_nodes(ctx.scene, img)
            convert_bounds_to_node_parameters(ctx.scene, bounds[0], bounds[1], bounds[2]+1, bounds[3]+1, bounds[4], bounds[5])
            return {'FINISHED'}
        except BaseException as e:
            self.report({'ERROR'}, e.args[0])
            return {'CANCELLED'}

#
#

def register():
    bpy.utils.register_module(__name__)

def unregister():
    bpy.utils.unregister_module(__name__)

if __name__ == "__main__":
    unregister()
    register()

if False:
    bounds = find_green_rectangle(get_image("/var/tmp/matte.png"))
    print(bounds)

    scn = bpy.context.scene
    convert_bounds_to_node_parameters(scn, bounds[0], bounds[1], bounds[2]+1, bounds[3]+1, bounds[4], bounds[5])

To use this, follow the instructions in the Install the Addon section of the Addon Tutorial . Then use the Spacebar pop-up menu to invoke the Composite Scene Into Matte operator.

Answer 2

You can use a Plane Track Deform: in doing so, you'll be able to apply the scene both on a rectangle or a deformed plane.

1. Open the "matte image" in the Movie Clip Editor on "Tracling" mode
2. Add 4 markers near the corners

3. Select all the markers and add a Plane Track

4. Drag the corners to fit the "hole" you want to fill with your scene
5. Open the Node Editor and the Plane Track Deform (add > distort > plane track deform)

Answer 3

Consider two planes in the 3d View, Orthogonal Camera. Both planes have image textures, are shadeless, have transparent portions, and thus illuminated at all times.

The further view is the background. The closer view is the foreground.

You can animate location on the X, Z axis, and size of the foreground. We assume your permanent view is View 1 Front Orthogonal.