It looks like you're doing something like this to generate the terrain.
Regardless, with whatever method you've used it should be possible to identify the handle that affects the coastline.
So in my case I could duplicate this Color Ramp and use the current position of that handle to mark the coast, and then create a new mask on the other side of it for a foam shader mix factor. This does have the unfortunate side-effect of catching a bunch of the terrain. I don't know if this happens with every method, but regardless we'll fix that next.
You've said that the water is at Z = 0, and I've made sure mine is the same. To isolate only the coast, we can take Object Coordinates and separate out the Z axis. Run this through an inverse operation and Clamp the result.
This has given us a mask for the islands. The jaggies are down to the level of subdivision. I was using Adaptive so I set my Dicing Scale really low to make them go away for this screenshot.
We can subtract this mask from our Color Ramp output, also Clamping, and get the coast, at that point we can go back to the Color Ramp and adjust the threshold to taste.