I baked Position pass, that works without issues for Displacement modifier.
When I use this texture in Geometry Nodes for Set Position node the result is OK, only vertices marked as Seams are positioned wrong. Vertical loop is mirrored and rotated, horizontal inner loop follows outer loop (= positions of middle UV loop).
I tried to scale down UV a bit (0.99) ... to test if Margin make some difference, but it didn't.
GN node-tree issue ...? Thanks :)
File was big with texture ... so first press Bake for selected Torus object, than make visible "02..." object to see texture effect in geometry node displacement ...