If you want to use CUDA for rendering with a downloaded Blender release, you do not need to install the CUDA toolkit. What you do need is (proprietary) NVidia drivers for your graphics card, any reasonably recent version will do.
Ubuntu and other linux distributions provide a way to do that, and you may already have them installed. If you don't you should for best Blender performance even when you don't use Cycles GPU rendering. If CUDA does not show up in User Preferences > System > Compute Device, you may need to install a newer NVidia graphics driver.
If you want to make your own Blender builds with CUDA support, you need to install the CUDA toolkit. To do that, download the CUDA toolkit 5.0 from:
And install with a command like this:
sudo sh cuda_5.0.35_linux_64_ubuntu11.10-1.run
It may ask you about installing drivers, but it's usually best to skip that part and only install the toolkit in the default location. The drivers will interfere with drivers that you installed through the package manager. You do not need to set any environment variables or make links like that tutorial, it's really this one command.