You are not doing anything wrong, it seems to be a bug / limitation of Blender selection.
As described in the question, a workaround is to select multiple times from different views. Pan, rotate, zoom etc. to select all you need.
With a dense mesh I did not get anything selected when viewing from far away. Zooming closer, I was able to select, so have been panning around the model selecting everything in partial segments.
I suspect it has to do with how the selection works from the viewport and somehow the accuracy of finding vertices based on the selection. Or maybe the amount of vertices: in my case, I have a huge 17M verts object and it selects nothing if I try to box select all visible. But when I go closer and select only some area, it starts working, and it's possible to select what you need, jumping around with the view.
My guess is that it has to do with using OpenGL for the selection queries and precision of how the vertices are in the graphics system, maybe it fails to find all when they are very close to each other in view. But succeeds when you zoom closer.
Would be nice if that could be fixed somehow, maybe the selection code could be improved somehow, I don't know it close enough to know whether it'd be possible.