3
$\begingroup$

Maybe that's not the best title for this question, but I find it hard to describe what I'm trying to do in a generic way, so I'll just describe the situation. My friend and I are designing a human character based on reference photographs, and we got most of the way through modelling the face before realising that we had been modelling in orthographic mode, while the reference pictures were, of course, in perspective. Now, when we look at the model in perspective mode (as it is intended to look in an eventual render or export), it looks totally wrong, as you would expect.

The model looks correct in orthographic mode from both the side and the front. I know roughly how far the real life camera was from the subject, and rough relative locations. As far as I can tell, this should be enough information to figure out where the vertices would need to truly be located in order for what I currently see in ortho to be what I view from the relative camera positions in perspective. My question is similar to this one, and in the best answer there, the author mentions the reason it is impossible in that user's case is the loss of depth. Well, I have the two perspectives they say are necessary, but I can't quite make out how to properly use them.

This is the Python script I have so far that is based on my best guess of how the math should go:

import bmesh
import bpy
from mathutils import Vector
from mathutils.geometry import intersect_line_line as poi

#Location of the camera looking at the front view.
cf = Vector((0, -12, -0.7))
#Location of the camera looking at the side view.
cs = Vector((11, -1.8, -0.7))

#Get bmesh
data = bpy.context.object.data
bm = bmesh.new()
bm.from_mesh(data)

for v in bm.verts :
    #I'll explain my logic for this section in more detail below.
    fy = 1 / ((1 / (0.029 * 8)) - (1 / (-12 + v.co.y)))
    fx = v.co.x * fy / (12 + v.co.y)
    fz = v.co.z * fy / (12 + v.co.y)
    fp = Vector((fx, fy, fz)) + cf
    sx = -1 / ((1 / (0.029 * 8)) - (1 / (11 - v.co.x)))
    sy = v.co.y * -sx / (11 - v.co.x)
    sz = v.co.z * -sx / (11 - v.co.x)
    sp = Vector((sx, sy, sz)) + cs
    p1, p2 = poi(fp, cf, sp, cs)
    point = (p1 + p2) / 2
    v.co = point

bm.to_mesh(data)
bm.free()

So here's how I arrived at that. First, I look at the front view. For each vertex, I calculate the image distance from the camera, using the formula I found here. The 0.029 is for the 29mm focal length of the real-life camera we used. This becomes the depth value for the front analysis point. I treat the current location of the pixels as though they are the result of a projection, so the x and z of this point of analysis are based directly off of the x and z of the vertex, just scaled down (I figure if the analysis point is 99% of the way from the camera to the vertex, it should be 99% colser to the X and Z axes). I do the same for the side view, then look for the point of intersection (i.e. the point both of these analysis points could have been projected from), and make that the new location of the vertex.

Right now, it doesn't work very well. With cameras placed at the indicated locations, the face ends up looking stretched on the Z axis and/or not wide enough on the X. It does look a little more accurate, I think, but nowhere near what I was hoping. Does anyone know where I went wrong? Suggesting a modification to my method is valid, but rest assured, throwing my method away and showing me a top-down better way of doing it is also very welcome.

$\endgroup$
  • 1
    $\begingroup$ This reminds me of vimeo.com/221178360 . Why not just solve the main problem? If the model does not look right, chances are one should concentrate on modelling. This is an interesting Python and math exercise, I can imagine how it is fun and useful to do, but I doubt it will make the model look great in the end. You could look into Blender's camera tracking functionality to reconstruct locations, orientations and perspectives of cameras for the reference photos if you have a few of them and wish to view the model in correct perspective while modelling with the reference photos. $\endgroup$ – Martynas Žiemys Mar 12 '19 at 6:06
  • $\begingroup$ It might be possible to use a Lattice to deform your mesh - by eye rather than mathematically. You'd need to orient your 2x2x2 lattice around your mesh and line it up with your existing perspective as set your mesh to deform to that lattice. You can then adjust the lattice to manipulate the back face and the same transformation will be applied to your mesh. It would be helpful if you could upload an example Blend file and perhaps some screenshots to help people visualise your issue. $\endgroup$ – Rich Sedman Mar 12 '19 at 10:30
  • $\begingroup$ The reason I want to do this mathematically rather than by hand is that neither my friend nor I are experts at this, and it took us a very long time even to get as much as we've done so far. The amount of work it would take to go from what we have to what we need feels barely better than starting over. We've already sunk a ton of hours into just getting where we are. It seems like the math to do this should exist, and that would save us redoing all the work we've done. $\endgroup$ – Tim M. Mar 12 '19 at 13:30
2
$\begingroup$

Blender has build in camera tracker. Load the images as a video sequence and mark trackers in each by hand (spots and freckles work well). Then solve the images to get your trackers perfectly positioned in 3d space. Now you know exactly where things should be in 3d space.

You can deform your current geometry to conform to the trackers with big sculpt brushes.

It's hard to say if there is mathematical solution, because it is unclear how you modeled the mesh. You had multiple reference sources and you introduced error from each view, each error in different direction. Manual correction is the fastest way for now and using actual photogrammetry is the way for future, you don't need to waste time with this script (unless you want to develop your own photogrammetry tool:).

$\endgroup$
  • 1
    $\begingroup$ That's an interesting idea, but not really an answer to my question. I think I made it pretty clear how I modeled my mesh, but to be more specific, I had front and side reference photos and just traced them with a mesh. It now looks accurate in ortho view, but that means that the absolute positions of my vertices represent the result of a perspective transformation. Seeing as getting to that point is a mathematical process, I don't see why it shouldn't be mathematically possible to go backwards and find the particular mesh that would result in a given perspective transformation. $\endgroup$ – Tim M. Mar 12 '19 at 13:26
  • 1
    $\begingroup$ @TimM. Because to do the transformation you need to know the depth of the surface point. You can do that only from another reference image. If you have a single point in both reference images, you can do the tracker solve. If you don't have enough reference images to have each surface point in multiple, then there is no solution. The task proposed to do the math equals to writing your own camera solver and believe me that's a lot more work than adjusting the mesh you have. You should show what you have so we can know if it's salvageable.. $\endgroup$ – Jaroslav Jerryno Novotny Mar 12 '19 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.