No, you're not doing anything wrong.
The Z-Pass is one of just a few, when not even the only pass that doesn't have a range from 0 to 1. It goes from almost Zero to infinity. Every pixels gets assigned a value that shows how far away it is compared to the camera.
If a pixel is 75 Blender Units away it gets assigned the value of 75.
If you output the Z-Pass through the viewer it will only show values between 0 and 1. Everything above 1, which is probably everything in your scene, will be shown as white.
If you are using nodes like Z-Combine it will work perfectly fine.
For other compositing purposes you'll want to use the Map-Range node:
Input the Z-Pass into the "Value". Than change the "From max" value to the object furthest away from your camera. You'll see the effect by cranking up the value.
For visualisation purposes only I recommend the Normalize-Node:
This node will take all values and shrink them down to a range between 0 and 1.
For Still images this probably is the easiest way, for animations use the method shown above.