Python: Obtaining Object Contour Pixels

Question

I would like to obtain the pixel coordinates of an object's outline in the currently-rendered scene, using the bpy module.

I found this, which is related, but I'm still not sure how to do it.

Has anyone had experience with something similar?

Answer 1

Edited - refined answer that only requires numpy

To find the outline pixels of an object:

1. Make sure it has a separate object ID than other objects.

2. Check the Object ID pass in the scene tab of the properties panel.

3. Replicate this node setup in your compositor and render your scene.

4. Run this script:

   import bpy
   import numpy as np
   
   S = bpy.context.scene
   
   # Read render settings - image size
   width  = int( S.render.resolution_x * S.render.resolution_percentage / 100 )
   height = int( S.render.resolution_y * S.render.resolution_percentage / 100 )
   depth  = 4 # RGBA
   
   img = list( bpy.data.images['Viewer Node'].pixels )      # Read viewer node
   
   # Reshape into 2D RGBA matrix, then Keep only R value for each pixel to convert to grayscale
   img = np.array( img ).reshape( [height, width, depth] )[:,:,0]
   
   # Find edge pixel coordinates. Each row has the X and Y values of an edge pixel
   edgePixelCoordinates = np.array( img.nonzero() ).T
   
   print( edgePixelCoordinates )
   

The print statement above will spew out a preview of the full array. Looks like this with the outline above:

[[128 511]
 [128 512]
 [128 513]
 ...,
 [379 531]
 [379 532]
 [379 533]]

Original answer with OpenCV

The code below will do most of the heavy lifting, though it isn't trivial to implement, and uses 3rd party python modules (mainly numpy and opencv). It also assumes you have a different object index for each object in your scene, and a node setup that includes an object ID mask plugged into the viewer node in your compositor.

What Isn't covered here:

• How to install 3rd party modules in blender's python. See this answer to solve that.
• How to automatically set the object index on the object ID mask node to that active object (that's a fairly easy step to add).

What is covered here:

• How to find the outline of a mask using openCV.
• How to find the pixel coordinates of the outline.
• How to generate a blender image containing the outline (wasn't required but is useful for debugging).

To use this script, set a different pass index to each object, or at least a different pass to the object of interest compared to other objects. Also, in the scene tab of the properties panel, add the Object Index render pass. In the compositor, Add an ID Mask node and connect it to the IndexOB render layers output, then connect the ID Mask's output to a viewer node. Render the scene once, and you're good to go! Run the script and you'll be able to see the outline in a new (or existing) image called 'edges' by default.

The script stores pixel coordinates as an array of XY pairs. The first 10 pixels from the contour above look like this:

>>> edgePixelCoordinates[0:10]
array([[128, 527],
       [128, 528],
       [128, 529],
       [129, 527],
       [129, 529],
       [129, 538],
       [129, 539],
       [129, 540],
       [130, 527],
       [130, 528]], dtype=int64)

Here's the code:

import bpy
import numpy as np
import cv2

S = bpy.context.scene

# Read render settings - image size
width  = int( S.render.resolution_x * S.render.resolution_percentage / 100 )
height = int( S.render.resolution_y * S.render.resolution_percentage / 100 )
depth  = 4 # RGBA

img = bpy.data.images['Viewer Node'].pixels             # Read viewer node
img = np.array( img ).reshape( [height, width, depth] ) # Reshape to 2D RGBA matrix

# use 1 color channel (=grayscale) and convert from float to 8bit int pixel values
img8bit = np.array( img[:,:,0] * 255, dtype = 'uint8' ) 

# Use the canny filter to find edges
edges = cv2.Canny( img8bit, 10, 20 )

# Find edge pixel coordinates. Each row has the X and Y values of an edge pixel
edgePixelCoordinates = np.array( edges.nonzero() ).T

## Add outline as image in blender    
# Generate an RGBA image from the 8bit outline image
edgesRGB = np.zeros((height,width,4), 'uint8')
for i in range(3):
    edgesRGB[..., i] = edges
edgesRGB[..., 3] = np.ones((height,width))

# Flatten image matrix to a 1D vector - the way images are stored in blender
blEdges = edgesRGB.ravel()

# Create new blender image to store outline, or assign to existing one
outlineImageName = 'edges'
edgesBlImage     = None
if 'edges' in bpy.data.images:
    edgesBlImage = bpy.data.images[ outlineImageName]
else:
    edgesBlImage = bpy.data.images.new( outlineImageName, width, height )

imNew.pixels = blEdges

Answer 2

First, I can obtain all of the object's vertex pixel locations as done here.

Next—though this may not be the most efficient method—I can determine contour pixels merely by processing the exported image data. I set the first and last vertex pixels in a row to 1, as well as any vertex pixels that are followed or preceded by non-vertex pixels. I set the rest of the pixels in the row to 0.

By repeating this for all pixel-rows in the exported image data, I should be able to obtain an outline.