This can happen when you have a Non-manifold mesh or intersecting geometry (see this post).
Non-manifold basically means that it cannot exist in real world 3D space. E.g. A 2D plane with no thickness is a non-manifold mesh. See What is non-manifold geometry? for more detail.
Select your mesh in edit mode and press CtrlAltShiftM to select non-manifold geometry. Without looking at your .blend or at least some screenshots of your model, it's hard to say exactly how to fix the problem.
Automatically fixing a non-manifold mesh:
The Remesh modifier always results in a manifold mesh, however it works by creating an entirely new mesh, so all topology, vertex groups, UV coordinates, etc. will be lost. It also often requires a dense mesh in order to match the shape of the input model accurately, so suffice it to say that it is better to fix your mesh manually when possible.
Using boolean modifiers to fix a non-manifold mesh:
If you don't mind messing up your topology a bit (which is probably the case, giving that it's going to be fractured anyway), using boolean modifiers to connect tricky parts of your mesh can save some time.
Here's an example workflow:
Find parts of your mesh which are disconnected/non-manifold with CtrlAltShiftM:

If there are no minor easily fixable issues (I don't see any in the above example), select everything and press P> Loose parts to separate all the disconnected parts of the mesh into separate objects. If there are some quick fix issues, fix them first and then separate.
Go over each object and fill holes/fix other issues identified with CtrlShiftAltM. Using Numpad / to go into local view can help de-clutter the view and focus on one object.

Once all your individual objects are fixed and have issues such as intersecting geometry taken care of, you can use boolean modifiers to join the separate objects. You can press E while hovering over the object selection field to select an object by clicking on it in the 3D view.

The workflow which worked best for me is:
First boolean join two objects and apply the modifier.
Delete the extra leftover object (the one which was specified in the modifier).
Repeat until all the objects are joined.
This useful addon might help too.