2
$\begingroup$

I've been considering how to use Blender to extract depth information from a pair of images of the same subject (Whitby Abbey) but would appreciate some help in getting started. I've found a couple of previous questions that didn't seem to have a suitable answer (eg, Generate Z-Depth map from Stereo Image-Pair (just like nuke occula)).

My thinking is to use the pattern matching of the Motion Tracker to match up points from one image to the other and measure the parallax and calculate the depth to eventually generate a mesh.

I'm thinking what I need to know is can the motion tracker be used to match sections from one image to another and how could I go about this? What I want to do is to use Python to pick a point on one image and create a marker with specific dimensions and search area and get it to match it up to a point on the second image and return the coordinates of each.

Has anyone done anything similar? Any ideas, hints, tips, good starting points?

EDIT : The goal is to be able to sample hundreds (perhaps thousands) of points and to use the resultant measurements to build a mesh which could then have the original images projected onto it to produce a 3d representation of the original scene.

EDIT2 : What I really want to know is the following :

1) How can I use Python to create a Motion Tracking Marker - including setting its parameters such as size, search area, etc. 2) How can I use Python to position the marker on a specific frame 3) How can I use Python to trigger the marker to search for a matching pattern in the subsequent frame. 4) How do I get back the resultant offset between its location in one frame and the next.

I do have some limited experience of running Python within Blender and using bpy - and the maths involved (I'll be ignoring any complications such as camera distortion) - I'm just a bit lost in how to get started with interacting with the motion tracker.

Obviously, once I've got this part working I'll then need to start looking at how to use that information to programmatically create a mesh and potentially look to create an add-on - but that's another question.

Improve this question
$\endgroup$

3 Answers 3

Reset to default
4
$\begingroup$

Depth estimation from images is a well established field and Blender is not the software to go for. Look for keywords like 3D reconstruction, structure-from-motion, multiview stereo, stereo reconstruction, stereo depth estimation. Typical approach will be to detect SIFT (or some other) features, match them, compute relative orientation of the cameras and then use something like semi-global global matching. The code you can find, for example, in OpenCV. Look at this tutorial, which creates a disparity map. There is also stereo_match.py in OpenCV-Python samples. For feature matching look here.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks - that OpenCV looks extremely interesting and useful and I'll definitely be giving that a look so upvoting this answer. I would like to try and do something in Blender if possible though. $\endgroup$
    – Rich Sedman
    Commented Sep 16, 2016 at 21:58
2
$\begingroup$

Maybe this is something for you: LINK to PDF

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks - that is certainly interesting. It's a different way of inferring depth - one that's probably way beyond what I'm trying to achieve. $\endgroup$
    – Rich Sedman
    Commented Sep 16, 2016 at 8:15
  • $\begingroup$ Have you looked into PCL: link $\endgroup$
    – chris
    Commented Sep 17, 2016 at 14:40
  • $\begingroup$ I'm also after an automatic solution; so far I've found the following tools which might be helpful along the way. Point Cloud Skinner: As of now I wasn't able to get BlenderSFM on Linux/Debian up running. It's sources some *.exe files, windows needed. BlenderSFM $\endgroup$
    – chris
    Commented Sep 20, 2016 at 20:47
0
$\begingroup$

You can give your images sequential file names and load them as a two frame image sequence. Treat it just like match moving a video. No need for any python - just have motion tracking auto detect points (or manually choose them) and track forward.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the suggestion. I didn't make it clear in the original question (I'll edit it) but I'm looking to potentially sample many hundreds of points (perhaps thousands) and use them to create a mesh that I could then project the images onto to create a 3d representation of the scene - so an automated solution would be necessary. $\endgroup$
    – Rich Sedman
    Commented Sep 15, 2016 at 11:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .