In Blender I can render an animation by a camera with specific camera intrinsic settings:
Say I generate images of size 640x480, then the intrinsic matrix K becomes
and similar with beta for sensor height and image height. uc,vc are center points of the image.
Now, from blender I can get the camera Rotation-Translation matrix RT which basically describes the cameras trajectory and rotation within the synthetic world.
Now, I can map from image coordinates to world coordinates back and forth:
The issue here is, that I get an projection error that is quite large. That means, if I project the left-upper corner u = 0, v = 0 to world coordinates, then x_world is close to the true destination but not quite. If I manipulate the focal length in K then I can adjust the projection a little bit in a positive way but this seems hacky.
Has anyone ideas or thoughts on this? Thanks