Convert mouse movement into rotation value (not in bge)

Question

Trying to produce similar behavior to the standard Blender rotate operator, for a modal operator I'm building. This stackoverflow post seems to be on the right track but I'm confused as to how I might implement the math. Also, limiting the rotation to a single axis at a time (as the locked axis image below displays) would be fine. Basically want to emulate what Blender's rotate operator does once you've locked an axis (doesn't seem like it should be to hard).

I've got everything else worked out, I just need a bit of code that takes, mouse_x, mouse_y, object_x and object_y and spits out a corresponding angle for say Y rotation of the object.

EDIT: Though the title reads (not in bge) I'm not at all opposed to importing something from the bge module to use it if it can solve this problem (just meant that this is for use directly in Blender, not for the game engine).

Below is what I've got so far. Thought it might help if I traded out the direct object 3D coordinates with those from the screen (only found region coordinates via bpy_extras.view3d_utils though). Currently, the mouse rotation does not correctly equate to the y axis rotation. Any ideas? Another approach perhaps?

import bpy, math
from bpy_extras import view3d_utils


#Call from Space menu in 3D View. For this example
#you would want to be looking in front perspective.

class custom_rotate(bpy.types.Operator):
    bl_idname = "transform.custom_rotate"
    bl_label = "Custom Rotate"

    def modal(self, context, event):

        #Grab active
        ob = bpy.context.object

        #Region context for view3d_utils
        region = context.region
        rv3d = context.region_data
        coord = ob.location.x, ob.location.y, ob.location.z

        #Convert ob loc to region loc (maybe would work with screen loc 
        #if I could get it some how?)
        ob_region = view3d_utils.location_3d_to_region_2d(region, 
        rv3d, coord)

        val1_x = ob_region.x
        val1_y = ob_region.y

        val2_x = event.mouse_x
        val2_y = event.mouse_y

        #Get rotation angle and update object rotation
        alpha_angle = math.atan2(val2_y-val1_y, val2_x-val1_x)
        ob.rotation_euler.y = alpha_angle

        #Print angle in degrees to help with testing...
        print('alpha angle:',math.degrees(alpha_angle))



        #FINISH
        if event.type in ['LEFTMOUSE','ENTER']:
            print('finished')
            return{'FINISHED'}

        if event.type in ['ESC','RIGHTMOUSE']: 
            return {'CANCELLED'}

        return {'RUNNING_MODAL'}


    def invoke(self, context, event):
        context.window_manager.modal_handler_add(self)

        return {'RUNNING_MODAL'}



def register():
    bpy.utils.register_module(__name__)             
def unregister():
    bpy.utils.unregister_module(__name__) 

if __name__ == "__main__":
    register()

Answer 1

Emulating Rotation

Below is a sample operator to emulate the rotation operator. By default it rotates about the view vector self.axis = rv3d.view_matrix.transposed().to_3x3() * z_axis This axis of rotation is used to create a rotation matrix Matrix.Rotation(angle, size, axis).

Pressing XY or Z while the operator is running will make object only rotate about the corresponding global axis.

Pressing the same in XY or Z again will rotate its object about its corresponding local axis..

The angle of rotation is calculated by getting the signed angular difference between the two 2d vectors from pivot point (in this case object location) to the current and previous mouse region coords.

import bpy, math
from bpy_extras.view3d_utils import (
        location_3d_to_region_2d,
        region_2d_to_vector_3d)

from mathutils import Matrix, Vector

class CustomRotate(bpy.types.Operator):
    bl_idname = "transform.custom_rotate"
    bl_label = "Custom Rotate"
    pivot_point = Vector((0, 0))
    v2d = Vector((0, 0))
    axis = Vector((1, 0, 0))
    mw = Matrix()

    def modal(self, context, event):
        ob = context.object
        if event.type in ['LEFTMOUSE','ENTER']:
            print('finished')
            return{'FINISHED'}

        if event.type in ['ESC','RIGHTMOUSE']: 
            ob.matrix_world = self.mw
            return {'CANCELLED'}

        if event.value == 'PRESS' and event.type == self.axis:
            # two times make it local
            axis = Vector([int(c == event.type) for c in 'XYZ'])
            self.axis = ob.matrix_world * axis  - ob.matrix_world * Vector()
            return {'RUNNING_MODAL'}                
        elif event.value == 'PRESS' and event.type in "XYZ":
            # to rotate about global axes
            self.axis = event.type
            # local
            return {'RUNNING_MODAL'}

        v2d = Vector((event.mouse_region_x, event.mouse_region_y)) - self.pivot_point
        if not v2d.length:
            return {'PASS_THROUGH'}
        #Get rotation angle and update object rotation
        a = self.v2d.angle_signed(v2d)
        #Print angle in degrees to help with testing...
        #print('alpha angle:',math.degrees(a))
        t = ob.matrix_world.translation.copy()
        mw = ob.matrix_world
        mw.translation = (0, 0, 0)
        rot_mat = Matrix.Rotation(a, 4, self.axis)
        mw = rot_mat * mw
        mw.translation = t
        ob.matrix_world = mw
        self.v2d = v2d        

        return {'RUNNING_MODAL'}

    def invoke(self, context, event):        
        region = context.region
        rv3d = context.region_data

        ob = context.object
        self.pivot_point = location_3d_to_region_2d(region, rv3d, ob.matrix_world.translation)
        self.v2d = Vector((event.mouse_region_x, event.mouse_region_y)) - self.pivot_point
        z_axis = Vector((0, 0, -1))

        self.axis = rv3d.view_matrix.transposed().to_3x3() * z_axis
        self.mw = ob.matrix_world.copy()
        context.window_manager.modal_handler_add(self)

        return {'RUNNING_MODAL'}

def register():
    bpy.utils.register_class(CustomRotate)             
def unregister():
    bpy.utils.unregister_class(CustomRotate) 

if __name__ == "__main__":
    register()

Pays to get used to using matrices to rotate, to transform the world matrix, rather than adjusting values of a particular rotation type like Euler.

Answer 2

Woo hoo! Figured it out. I was almost there. The problem was that I was simply using event.mouse_x and event.mouse_y (which returned x and y values for the entire screen, not just whatever 3D View I happened to be in). Change this to event.mouse_region_x and event.mouse_region_y and the angle data will be generated based on the size of the 3D View window you happen to be in and rotation responds perfectly! There's a bunch of other little calculations required to make it prettier (object angle offset, inverting values etc...), but those are pretty straight forward. The hardest stuff is done.