Is there a performance (or any other) difference between these methods?
If you end up with the same mesh in the end, no. Displacement (for Cycles anyway) is tessellated to a regular mesh at render startup, so the resulting object loaded into the renderer is the same as if you exported that geometry directly.
But! The benefit of displacement is that you have a number of ways to use a different, more efficient mesh. You could, for example, use view dependent adaptive subdivision to only have the full detail mesh near the camera. You can also use only a few subdivision levels in the viewport to use a lighter weight mesh while placing objects on the terrain.
You can also make use of the "displacement and bump" setting for Cycles displacement:
This will give the normals of a theoretical infinite resolution mesh (within the resolution limits of the displacement map anyway) even though the actual displaced mesh is of limited resolution. For example, let's say your original terrain used 4 subdivisions for some very fine detail that doesn't really affect the silhouette of the terrain. If you only use 3 subdivisions for the displaced mesh, it would lack that detail extra detail. However, by using the "displacement and bump" option, the normals will be those of an infinite resolution mesh. Thus, the shading will match the original 4 subdivision mesh, even though your actual mesh is only 3 subdivisions and thus 1/4th of the memory usage.
If you're exporting to another software, you can export your un-subdivided base mesh and the displacement map, which tends to be a much lighter weight file than the full resolution mesh.
Now, let's say the terrain has a lot of big flat patches, which I can decimate to be one flat n-gon each. Now when baking displacement and applying it outside of blender, somewhere where I cannot control mesh topology and subdivision process, it may create more polygons on that flat areas.
Does that mean that when exporting a model to another software, a detailed geometry is preferred over displacement?
This is very situational. In some cases, yes, it may be better to export the full resolution rather than using displacement. However, this assumes you cannot control the resulting mesh in the other software as precisely as you could in Blender, and that those additional polygons will make much of a performance difference anyway. A lot of times, one or both of those things won't be the case, and you're better off with the lighter weight files of base mesh + displacement map. For example, you may be importing to Unreal, where you're much better off using your displacement map with a Nanite landscape actor than trying to manage landscape meshes yourself.
