Sample the nearest surface's normal, scale it depending on the distance to that surface:
For a better effect consider detecting if a point started inside the collider. If so, first move it to the nearest surface, and then move it away as if it was at the surface from the beginning. Which simply means that you need to use the standard technique of raycasting down to detect if you're inside, and in such case simply displace by the maximum displacement + found distance to nearest face.
Below I'm doing something slightly more complicated: use the normal of the nearest face, 'move' A LOT into that direction, and from there raycast in reverse direction to find the outermost face of some complex mesh that could have many cavities and holes. Then move to that outer surface point, and displace in that surface's normal (which could be quite different than the original face's normal).
- The points suddenly switch the direction of displacement, because the found faces and so their normals suddenly switch. I don't know of any existing feature that would take an interpolated normal used in smooth shading, but such a normal can be calculated. It's complicated, however… First, you need to find the nearest face and edge, and check if the found points on those are at the same position (because the nearest face could be a wrong face as explained in the already linked post). If so, simply interpolate the normal of the two vertices of this edge. If not, get all vertex normals of the face, multiply each by the distance to the "nearest" position, add these all up and normalize.
- If your mesh has thickness, it could be a point is in empty space inside it, but it still wants to be be put on the outer shell before being displaced. In such case the raycast still probably can be used, because the raycast down will hit, but the dot product with normal comparison will fail (negative dot product), so in such case either you're inside an object with thickness, or simply outside (above) it, and more raycasts would need to be done to ensure either or another, see Hulifier's answer.