I was able to replicate your issue:
The problem comes from the subdivision surface modifier and the option "adaptive subdivison"
There are several ways to fix this, depending on how you want your end result to look like.
Adaptive Subdivision doesn't like triangles very well (from my experience, could be wrong tough), so one step would be to try and reduce those as much as possible.
It also yields better results if edges are parallel to each other. So, essentially just a subdivided plane.
You can also turn off the adaptive subdivision, depending on what you want to do with your scene.
My question to you is, what do you want to do with the adaptive subdivison, do you want to make the stones "stick-out" as a real mesh? If not you can use a normal map.