I would like to obtain the pixel coordinates of an object's outline in the currently-rendered scene, using the bpy module.
I found this, which is related, but I'm still not sure how to do it.
Has anyone had experience with something similar?
To find the outline pixels of an object:
Run this script:
import bpy import numpy as np S = bpy.context.scene # Read render settings - image size width = int( S.render.resolution_x * S.render.resolution_percentage / 100 ) height = int( S.render.resolution_y * S.render.resolution_percentage / 100 ) depth = 4 # RGBA img = list( bpy.data.images['Viewer Node'].pixels ) # Read viewer node # Reshape into 2D RGBA matrix, then Keep only R value for each pixel to convert to grayscale img = np.array( img ).reshape( [height, width, depth] )[:,:,0] # Find edge pixel coordinates. Each row has the X and Y values of an edge pixel edgePixelCoordinates = np.array( img.nonzero() ).T print( edgePixelCoordinates )
The print statement above will spew out a preview of the full array. Looks like this with the outline above:
[[128 511] [128 512] [128 513] ..., [379 531] [379 532] [379 533]]
The code below will do most of the heavy lifting, though it isn't trivial to implement, and uses 3rd party python modules (mainly numpy and opencv). It also assumes you have a different object index for each object in your scene, and a node setup that includes an object ID mask plugged into the viewer node in your compositor.
What Isn't covered here:
What is covered here:
To use this script, set a different pass index to each object, or at least a different pass to the object of interest compared to other objects. Also, in the scene tab of the properties panel, add the Object Index render pass. In the compositor, Add an ID Mask node and connect it to the IndexOB render layers output, then connect the ID Mask's output to a viewer node. Render the scene once, and you're good to go! Run the script and you'll be able to see the outline in a new (or existing) image called 'edges' by default.
The script stores pixel coordinates as an array of XY pairs. The first 10 pixels from the contour above look like this:
>>> edgePixelCoordinates[0:10] array([[128, 527], [128, 528], [128, 529], [129, 527], [129, 529], [129, 538], [129, 539], [129, 540], [130, 527], [130, 528]], dtype=int64)
Here's the code:
import bpy import numpy as np import cv2 S = bpy.context.scene # Read render settings - image size width = int( S.render.resolution_x * S.render.resolution_percentage / 100 ) height = int( S.render.resolution_y * S.render.resolution_percentage / 100 ) depth = 4 # RGBA img = bpy.data.images['Viewer Node'].pixels # Read viewer node img = np.array( img ).reshape( [height, width, depth] ) # Reshape to 2D RGBA matrix # use 1 color channel (=grayscale) and convert from float to 8bit int pixel values img8bit = np.array( img[:,:,0] * 255, dtype = 'uint8' ) # Use the canny filter to find edges edges = cv2.Canny( img8bit, 10, 20 ) # Find edge pixel coordinates. Each row has the X and Y values of an edge pixel edgePixelCoordinates = np.array( edges.nonzero() ).T ## Add outline as image in blender # Generate an RGBA image from the 8bit outline image edgesRGB = np.zeros((height,width,4), 'uint8') for i in range(3): edgesRGB[..., i] = edges edgesRGB[..., 3] = np.ones((height,width)) # Flatten image matrix to a 1D vector - the way images are stored in blender blEdges = edgesRGB.ravel() # Create new blender image to store outline, or assign to existing one outlineImageName = 'edges' edgesBlImage = None if 'edges' in bpy.data.images: edgesBlImage = bpy.data.images[ outlineImageName] else: edgesBlImage = bpy.data.images.new( outlineImageName, width, height ) imNew.pixels = blEdges
First, I can obtain all of the object's vertex pixel locations as done here.
Next—though this may not be the most efficient method—I can determine contour pixels merely by processing the exported image data. I set the first and last vertex pixels in a row to 1, as well as any vertex pixels that are followed or preceded by non-vertex pixels. I set the rest of the pixels in the row to 0.
By repeating this for all pixel-rows in the exported image data, I should be able to obtain an outline.