I have imported what appears to be a clean SVG into Blender (works fine for 2D) but when I try to extrude it, I am seeing spikes appear. Can anyone tell me what is the issue and how I can fix it?
Hard to tell for sure without post a blend file we can inspect. Most likely overlapping vertex or segments in your curve. Try checking your curve for duplicate geometry or overlapping segments as described in the link posted by Georges.
Otherwise try entering edit mode in your curve, and check if the radius of all your vertex are set to 1. Go into edito mode, select all vertex and head to the Tool Shelf > Tools > Curve Tools > Set Curve Radius Button and set all to 1 or any other value has long as they are all the same.