I constantly stay up to date with each new release of Blender, but this means my preferences are wiped clean each time. How do I export/import these preferences to avoid loosing my carefully crafted set up each time?
Depending which OS you're on, navigate to:
Once there you should see folder(s) corresponding to the version(s) you currently have.
In my case here I have 2.66 and 2.67, just open the previous folder (2.66) since I had saved my settings there earlier and copy the config folder to the new one (2.67). What you really want however is the startup.blend and userpref.blend inside.
If you have tried out a zip version of Blender and then tried to install a newer version of Blender you won't get the 'Copy Previous Settings' option noted in gandalf3's answer. This is because when you use a zip version, even if only to try it out, it will create its own user preference folder. When you then install a newer version of Blender, it will see the user preferences folder exists and so not offer to 'Copy Previous Settings'.
To fix this you can simply navigate to where the preferences are stored (as shown in iKlsR's answer) and delete the preference folder for the version you are trying to install. For example, if I was installing Blender 2.76 and found the 'Copy Previous Settings' option didn't appear, I would delete the '2.76' folder the zipped version of Blender had previously created. When you next open Blender, it will detect there are no preferences for 2.76 and should again offer to 'Copy Previous Settings'.