I figured it out, my dimension/scalescale in the Z direction was set to 0. While I was only working in the x/y dimensions, having 0 scale broke the mouse from working, even when the camera was perfectly set and orthogonal.
To fix: Go to object mode and click the object, press 'n' to open the Transform options on the right-hand, make sure Scale/Dimension aren't isn't set to 0, if. If you set eitherdimensions to zero0 it willcan set the otherscale to 0 as well.