This is a Q&A so I answer it myself. I searched for an answer to this question for ages and didn't find anything so I'll add this here if anybody else struggles with the same issue.
So, how can I merge two edge loops to one? I do not want to connect them (I have seen many answers that suggest to just bridge the edge loops), nor do I want to merge each vertex by hand. I just want each vertex to be merged with the closest opposing vertex.
Also, I find it highly unpractical and in most situations useless to remove doubles after moving the vertices to overlap each other by scaling and moving the edge loops. This is another common suggested answer.
Ctrl+E
> Bridge Edge Loops $\endgroup$