How to use blender for reinforcement learning? How to control camera movement and save the rendered images that the camera captured?

Question

I am trying to follow this paper and reproduce their works. But I get stuck with some problems and I really hope that some of you can give me some ideas.

So, basically what the paper has done is firstly building hallway models as environments with randomized texture and furniture placement, then control the drone(with a camera) to move around the environments to avoid obstacles. So the trained neural network receives current camera image and should output the next target point in the image area.

My questions are listed below, I would like to run the simulation (training and testing) all by python scripts with tensorflow to use the neural network. 1) I think for this problem I better use Cycles Baking to save the rendered images to the environments. Am I correct? 2) How to control the camera(installed on the drone) movement considering that the command would be a numerical output of a neural network? Using Blender Game Engine (motion actuator) or normal object move command (acts same as keyboard -G). I need to get the distance from the nearest obstacle and judging if a collision happened to update my Q-value. 3) How to save the camera images? I know that normally it's ctrl-F3 but how to do that in python code? Actually, what I am trying to do is kind of like using the flying navigation mode and save the images(I don't know how to save images in that mode I tried ctrl-F3 but didn't work).

Lastly, are there some tutorials about using reinforcement learning (based on camera images) with blender? Because I've tried finding these on Google many times, but seems there is no good related tutorials.

Thanks for you guys' help. Looking forward to the response!

Answer 1

^{Welcome to Blender.SE Tianqi Wang, please take the tour to find out how to use this site. Your question may get closed because it contains multiple questions, is partially opinion based, very broad and not quite limited to Blender. In future you can post as many questions as you want, but please limit yourself to a single question per post.}

Cycles baking

Yes, for a more naturalistic look of your images, you could bake diffuse lighting onto textures which are mapped to the objects of the scene. This won't work with transparent or reflective material components, but I guess may be optional in your setup. You will probably want to match the surface properties of your realworld environment; for an easier start, you could choose a diffuse realworld environment for testing with little reflections (this will likely be easier to the NN as well). Note, that you have to rebake your lighting/textures if you reconfigure your set as the shadows cast by objects will no longer be correct.

Even though not physically based the renderers Blender Internal (2.79) or EEVEE (2.8) could also be used to create the texture maps, using tricks like environment lighting/ambient occlusion. The images in the paper look as if they were realtime renders with none of the enhancement (ao, light baking), so maybe this step adds unnecessary complexity.

^{No visual enhancements here. Looks like a 2000's game.}

Camera Movement

Your image feed doesn't have to be realtime since the calculations can be split into discreet steps. I'd advise you not to use Blender's game engine. (If you think a game engine is necessary, I'd suggest Unity. It is free for personal use, has a good connection with Blender and support automatic light baking as well.)

I think, setting the camera's properties directly in code should be fine.

import bpy
import mathutils

camera = bpy.context.scene.camera
# set the camera's rotation mode to quaternion once (or do it through the GUI)
camera.rotation_mode = 'QUATERNION'

# get the camera's position and orientation
camera_location = tuple(camera.location)
camera_orientation = tuple(camera.rotation_quaternion)

# set the camera's position
camera.location = mathutils.Vector((0, 0, 0))
camera.rotation_quaternion = mathutils.Quaternion((0, 0, 0, 0))

If you are using constraints or other dependencies in your scene, you have to call bpy.context.scene.update() after setting new coordinates for your camera.

Save images from the viewport

The functionality of saving images with CtrlF3 is not exposed to the API (in 2.79).

I have documented two workarounds here.

The 2nd method uses an OpenGl render, but you will have to save and read the image to and from disk on each frame.