You are confusing extrusion e and grabbing g. While extrusion generates new geometry, grabbing and rotation just displaces or rotates existing geometry.

The portion of the video you mention uses grabbing g and rotating r, thus creating no new geometry.

To delete vertices you don't want, you can just press x or undo. Please keep in mind, that there is an object mode undo and an edit mode undo.

You are confusing extrusion e and grabbing g. While extrusion generates new geometry, grabbing and rotation just displaces or rotates existing geometry.

The portion of the video you mention uses grabbing g and rotating r, thus creating no new geometry.

To delete vertices you don't want, you can just press x or undo. Please keep in mind, that there is an object mode undo and an edit mode undo.

Remove doubles only works when vertices are in the same place (share x,y,z coordinates) otherwise you can only use x.

PS:

You probably never need to start from scratch, your mesh can be cleaned up quite easily.

Modelling a human head can be very challenging, I would suggest you put this on hold and do another tutorial on modeling in the meantime. I can recommend Andrew prices for example.