It is completely normal that the subdivided surface crosses the original mesh's edges when they make a concave geometry, and it's not an issue with the mesh. This becomes evident in Edit Mode because, together with your subdivided mesh, you also see the un-subdivided wireframe:

This effect doesn't appear in Object Mode since, in that mode, you don't see the underlying un-subdivided geometry.

However, you have the option to see the effect of Subdivision Surface also on the Edit Mode's wireframe. There is a button () in the modifier panel for this:

It is completely normal that the subdivided surface crosses the original mesh's edges when they make a concave geometry, and this shouldn't be considered an issue with the mesh. This becomes evident in Edit Mode because, together with your subdivided mesh, you also see the un-subdivided wireframe:

That "artifact" is just the subdivided mesh "coming out" from the cage of the un-subdivided mesh. Nothing to worry about. This effect doesn't appear in Object Mode since, in that mode, you don't see the underlying un-subdivided geometry.

However, you have the option to see the effect of Subdivision Surface also on the Edit Mode's wireframe. There is a button () in the modifier panel for this: