My high poly and low poly models are almost identical. The only difference is that the high poly model has a few extra loop cuts and extrusions. Here is the bake. I have circled the elements that coincide with the actual differences between meshes in yellow. To be clear, those are the parts I want to keep. I would like to get rid of the rest.
I find the presence of the remaining elements quite puzzling, as they represent parts of the mesh that are identical between the high and low poly versions. Here is a look at the model and its UV's
As you can see the additional artifacts appear to line up with the UV islands of the low poly mesh, and more specifically appear where different parts of the geometry intersect.
I have tried applying all transforms to both meshes. This did seem to help the problem a little but did not solve it. I have also messed around with all kinds of ray distance and extrusion values but to no avail. I tried deleting all parts of the high poly mesh that were identical to the low poly one, but this didn't seem to do anything either. Lastly I tried using a cage, but that generated an empty map regardless of ray distance setting.
Obviously I could just manually edit out the parts I don't want in photoshop or something, but from a workflow perspective that seems pretty rubbish.
At this point I am at a loss.