Using the same snapping setting as in your question, Blender will snap applying the smallest rotation possible.
So you can do that in two steps:
- Place it in a first side
- The grab it to another side
Similar to this question.
If you want the handle to be in another direction, rotate it first (for instance 90 degrees around Z) and snap again.