When I hit 'X' and select 'vertices', a gaping hole is created with adjoining faces/edges deleted.
In Edge Select mode (keyboard shortcut 2), select the edge you want to get rid of, press the space bar and type dissolve... then choose Dissolve Edges.
To remove an edge while keeping surrounding geometry, in Blender terminology you Dissolve it.
This can be done 2 ways:
- Delete then Dissolve Edges or...
(when you're in edge mode - since this is context-sensitive dissolve - which dissolves verts/edges/faces based on the current mode).