I am trying to select the cube that in the middle of the scene. How can I select it? All I see is a crosshair appearing everywhere I click. Is this a bug?
That circle/crosshair is the 3D cursor.
To select you need to Right click instead of Left click.
You can change this behavior in File > User Preferences (CtrlAltU) > Input:
Open the User Preferences:
Go to the Input tab. The select left/right options are under Mouse:
If you want to save these settings as the default for the next time you open blender, click Save User Settings in the Header (at the bottom of the window):
This is one of the most controversial parts of blenders UI.
For some further information you might want to check out:
This video by Sebastian König explaining some advantages to using RMB to select
You can change this in the User Preferences pane found at the top left of the main window in the File menu.
Next click the Input tab, then locate Select With, you will see two options which are Left and Right.
You may also want to look a little further down for Invert Zoom Direction if the current mouse wheel zoom seems backwards to you.
At the very bottom, you should see a button that says Save User Settings, click this to save the change. Just be sure to remember this when you are following any tutorials or you will end up really confused. It's really not a big deal once you know what's going on here.