As you can see below, I am getting some hard edges along my UV seams when baking from high-poly to low-poly. I've tried baking in Substance Painter as well as in Blender with many different settings, and I get the exact same issue. If I bake to a 4k map the seams are less noticeable, but still definitely there. I just can't seem to find a rock-solid solution to fixing this issue. Now i'm just wondering if this is just something that we have to deal with while baking.
Details: Low poly mesh has no hard edges and is smooth shaded with no autosmooth. I've looked it over multiple times and can confirm that it is the ideal mesh for baking. It and the high poly mesh match up very well and I have to do very little finagling with ray distance.
What am I overlooking?
Edit: I've uploaded a .blend file with the high and low poly rocks here: https://ufile.io/gqn8bhbo
The problem gets much worse with lower res textures.