Vertex normals are determined by the interpolated face normals of the connected faces. When you have the transform orientation set to Normal the Z axis is aligned with the vertex normal.
Trouble is that there is no way for blender to know how to align the manipulator on the corners. It still has the Z aligned correctly, but that is not helping you here.
See in this image I have a mesh similar to yours, selected at different times in two different spots. Note that even though the manipulator is pointed in the "wrong" direction in the bottom corner, it is still correctly aligned with the vertex normal (blue line on each vertex).
As a work around you could align the view to one of the faces, with ShiftNumPad 7. Then change the Transform Orientation (in the 3D view header) to "View".