cegaton's answer nicely covers the practical impacts of the size setting (i.e. the shadows), but what you are actually setting is the apparent size of the sun lamp.
In other words, the size controls radius of the sun as seen by the camera (or any other light ray), regardless of distance.
To see this in action, create a perfectly reflective plane and point a sun lamp straight at it:
Enable Multiple Importance Sampling on the sun lamp to allow caustics, then look straight down at the plane in rendered view (so that the sun lamp should be reflected back towards you)
If you are not in ortho perspective (in which case the sun lamp will appear infinitely large), you should see a circle which varies in size depending on the sun lamp's size setting.
This circle — a reflection of the sun lamp — shows that the lamp always appears the same size regardless of the distance to the camera, unlike the reflection of a sphere positioned a finite distance away:
In this gif the sphere is invisible to camera rays, so in rendered view only its reflection is visible
So you could say the sun lamp size is defined in camera space, rather than world space.