# How do I can grab the current image in blender game engine from a python controller?

I am trying to create a python controller that executes game commands based on the image pixels of one of the camera viewports. This is for a robot simulation using computer vision.

I've tried two different approaches to no avail:

APPROACH #1: use bge.render.makeScreenshot() to save the current frame to a png file. The PROBLEM is that when I process that file later in the script, the PNG file is invalid/incomplete. I think the image file writing buffer is not flushed and closed yet. The files are there and look fine after the I exit the game engine.

APPROACH #2: try to render and grab pixel values as outlined in this thread. The PROBLEM is that image array is None.

Is there a way to fix either of these approaches or another better approach for getting an image frame from the blender game engine?

import bge

def main():
cont = bge.logic.getCurrentController()
own = cont.owner
scene = bge.logic.getCurrentScene()

if not 'init' in own:
own['init'] = 1
own['counter'] = 0

own['counter'] += 1

######### APPROACH #1
# get frame using makeScreenshot()
frame_filename = "//frame" + str(own['counter']) + ".png"
bge.render.makeScreenshot(frame_filename)

# do image processing on frameXXX.png ...

# ***PROBLEM: frameXXX.png is empty/invalid/incomplete

######### APPROACH #2
# render a camera scene and get image pixels
cam = scene.active_camera
source = bge.texture.ImageRender(scene,cam)
source.capsize = [640,480]
imageArray = bge.texture.imageToArray(source, 'RGB')
charArray = imageArray.to_list()

# do image proccessing on charArray ...

# ***PROBLEM: source.image and imageArray are type 'None'

• You've tried use an image from a non-active camera? – Strapicarus May 30 '17 at 1:06
• @Strapicarus I did try another camera that was not scene.active_camera and it also gave None. I'll try some other cameras. – j314erre May 30 '17 at 1:19
• Side notes: - A meaningless function name adds confusion to the code rather then clarification. I suggest to rename the function to something that expresses what you expect it to do. - You do not need the property "init" you can work with property "counter" directly e.g. counter = owner.get("counter", 0) – Monster May 30 '17 at 4:55
• @Stapicarus thank you. I created a new camera, and parented it to the camera I wanted to capture from before and it is giving output now. Not sure why the other cameras in the scene were not giving output but I guess they were in some sense active – j314erre May 30 '17 at 14:20
• I suggest to replace the term "frame" in the title with "render", "screen", "output" or "image". As it is now - it reads like you are talking about the timing rather than the image. – Monster May 31 '17 at 6:18

The second approach is the closest. But here's some code that may help (untested, but pulled from a working setup)

    obj = cont.owner
camera = obj.scene.objects['NOT ACTIVE CAMERA']
tex = bge.texture.Texture(obj, 0, 0)
tex.source = bge.texture.ImageRender(
camera_obj.scene,
camera_obj,
)
tex.capsize(640*480)
tex.source.background = [0, 0, 0, 0]
source = self.tex.source
data_array = bytes(640*480*4)
source.refresh(data_array, 'RGBA') # Take the picture....

print(data_array)  # Hopefully lots of data - in a 1D array.


You shouldn't recreate the data_array and texture objects each frame. Only do it on the first frame, then just keep running the source.refresh() line.

If you run into issues, let me know and I'll make a blend file.