when you toggle "orbit around selection" the fact is that the viewport orbits the origin point of your selection. and that origin point is not always at the selection's geometry center, sometimes even at the world origin, which makes the viewport navigation so frustrating. Is there any way to force the viewport around the selection's geometry center no matter where its origin point is.
My frustrations with viewport navigation have ended once I started using Lock To 3D Cursor. I just simply Shift+RMB to set the 3D Cursor to my point of interest and Shift+MMB to pan drag the view until the 3D Cursor is in the center of my viewport and my viewport navigation works perfectly. Or in your case set it manually to the geometry center. This also has always solved my zooming far in problems. I'm just confused why so many people don't use this feature. I have never found any reason to disable this so it's permanently enabled for me.