What should I do? Should I even connect the faces at all?
I think you shouldn't have connected them in edit mode. First off, select any one of the plane in object mode. Then add the boolean modifier to it. Select the second plane as boolean object and set the type to union.
Alternatively, you can try sculpting your mesh with smooth brush.