2
$\begingroup$

Hello I am trying to get bounding box of items in my image which is generated automatically using a script by running blender from background. I am using the object index to get the boxes.

This is my current compositor setup, enter image description here

This is the code I am using to find the boxes

import bpy
import numpy as np

def getVisibleBoundingBox(objectPassIndex,pixels,height,width):
    bbox = np.argwhere( pixels == objectPassIndex )
    (ystart, xstart), (ystop, xstop) = bbox.min(0), bbox.max(0) + 1
    return xstart, xstop, height - ystart, height - ystop 

def getAllBoxes():

    S = bpy.context.scene
    width  = int( S.render.resolution_x * S.render.resolution_percentage / 100 )
    height = int( S.render.resolution_y * S.render.resolution_percentage / 100 )
    depth  = 4

    pixels = np.array( bpy.data.images['Viewer Node'].pixels[:] ).reshape( [height, width, depth] )
    print(max(pixels[540][1]))
    pixels = np.array( [ [ pixel[0] for pixel in row ] for row in pixels ] )
    boxes = []
    for i in  bpy.data.collections['CurrentItems'].objects:
        boxes.append([getVisibleBoundingBox(i.pass_index,pixels,height,width),i.name.split('.')[0]])
    return boxes

print(getAllBoxes())

This all worked fine when tested it with the gui but when running blender as background from command line it just gives this error ValueError: cannot reshape array of size 262144 into shape (1080,1063,4) I believe that is the empty viewer node image thats causing this.

Any ideas on how to get the image or some other method to get the bounding box that would work from background would be most helpful.

I have also tried saving the image as PNG and tried using that but it seems the objectPassIndex data is lost when saving as png.

P.S. I dont want to use world_to_camera_view as there is no reliable way to check for occlusion with that method.

EDIT: The bounding box needs to be a rectangle with the whole object inside it. If a object is occluded by some other object then only the visible part be included in the rectangle.

$\endgroup$
5
  • $\begingroup$ Can you provide more information about how the IndexOB is being used? And why don't you use bpy.data.images.size[:] for your height and width? What did you mean empty viewer? 262144 == 256*256*4 it is not empty, is it caused by running Blender in Background? $\endgroup$
    – HikariTW
    May 11 '20 at 1:49
  • $\begingroup$ Hi, I hope you don't mind me editing the title. I think with it being more specific this good question has a higher chance of catching the eyes of people who might be able to help. $\endgroup$ May 11 '20 at 7:05
  • $\begingroup$ Might be related: Python render script different outcome when run in background $\endgroup$
    – HikariTW
    May 11 '20 at 8:15
  • $\begingroup$ @HikariTW When you disconnect the viewer node there is a blank image in view node image and that is of the same size as this.When testing some stuff I accidentally left the node disconnected once and got this same error with the same number. And yes this is caused by running blender in background $\endgroup$ May 11 '20 at 13:31
  • 1
    $\begingroup$ Please exactly define how the bounding box needs to be. If there are 2 Objects and one partially occludes the other one, is the the bounding box of the occluded object what is still visible or the theoretical outline of the object? Is the bounding box the outermost rectangle in pixels, that doesn't touchg the pixels of the object, or does the bounding box touch the othermost pixels in x and -y dimension. Also is the bounding box retangular and the axis parallel to the edges of the image or can it also turn to give a best possible fit? $\endgroup$
    – WhatAMesh
    May 14 '20 at 14:25
1
$\begingroup$

A solution that worked for me was to use an File Output node instead of viewer node and save the image in openEXR format with full float color depth using this node setup

enter image description here

And then read that image via CV2 library instead of reading it from viewer node image.

Simply replace the following line in the code snippet in question

pixels = np.array( bpy.data.images['Viewer Node'].pixels[:] ).reshape( [height, width, depth] )

with

pixels = cv2.imread(r"filepathOf.exr",  cv2.IMREAD_ANYCOLOR | cv2.IMREAD_ANYDEPTH)
pixels = cv2.flip(pixels,0)

and it should work fine giving you coordinates of the bounding boxes

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.