When I open your file, I see an object that has rotation that is only significant in its Y axis. But when I enable the display of object axes, I see this:
That Y axis is not in line with the axis that we want to rotate, so we have wobble.
Really, here, we don't want to rotate about a cursor (rotation about the cursor isn't the same as rotation about an object origin.) And we don't want to rotate in some special view axis either (because we're using Euler angles, we want our rotation to be aligned with our axes.) So we need to fix the axes of this object.
Enter edit mode. Change to a "normal" orientation (instead of "cursor" on top of your 3D view.) Select the center vertex of your knob, snap cursor to selection (shift s for me), and use "create orientation" operation (ctrl alt space for me). Your orientation will change from "normal" to "vertex"-- it now represents the normals of that vertex.
Now leave edit mode. On sidebar/tool/affect only, enable "origins". On the object menu, use object->transform->align to transform orientation. Then use object->set origin->origin to 3D cursor. You now see that your origin lies at the vertex we selected (which we selected because it ought to be the middle of the knob), and that our axes have changed. They're aligned, but they're aligned with Z. Because we already are affecting only origins, we can rotate in the X axis by 90 degrees to get the Y axis pointing out, like you wanted:
Take a look at your animation. The wobble is gone. Don't forget to turn off affect only origins for future work, or it might be very confusing until you remember that....