Whenever I add an Object or import an SVG object, I am able to extrude it either by E or by the extrude tool in object data, but I can't do that for a plane ( and just saw for a cube as well ). I get no options for it in the Object Data section nor does Pressing E Extrude it. What have I done wrong or not installed?
In order to edit an object, you must be in Edit mode.
You can set the mode in 3D view > Header, or by pressing ↹ Tab:
When you import a SVG file it will be imported as a curve.
For Curve Objects extrusion is set in the Geometry Modification Extrude section. For this operation it doesn't matter if you are in Edit or Object Mode (Tab).