I don't know how to solve it though. Is there a way to just 'fill' the jaw bone model and make it completely solid? Thanks for any advice.
Booleans ussually fails on non-manifold mesh.
To see these problematic parts - from within Edit Mode, using Vertex selection, with no geometry selected, Search for "manifold" (Select non-manifold operation appear). For more info see Blender Manual
Q: Is there a way to 'fill' model and make it completely solid?
A: In a meaning - closed mesh without holes - Yes
Search for Fill operator and increase number of vertices in tool properties until the holes are filled. Like here it was value 14.
But N-gons are not the best for boolean operations so probably better to try ...
Like here. Important for this function is to have a loop of even number of vertices. If you don't have, you can select one edge and subdivide it. (White vertex in the screen is one I added.)
Or even better to Triangulate whole mesh Ctrl+T
Or you can try Remesh modifier > Sharp if there is too much work, but even on hi Octree Depth in wouldn't be precise.
But that's normal, right? The boolean asks you which mesh you want to substract by which other mesh. So it's notmal that the lower part of your blue mesh is substracted from, if it touches the jaw bone. If your goal is to keep that from happening, so that only the teeth make an imprint, I suggest you select your teeth, even if roughly, and press P, for seperate, and "selection", so the teeth you selected are now their own mesh, you can rename that teeth. Now this time using the boolean modifier, you select the teeth mesh and not the jaw mesh, and only the teeth will make a mark. You can now in object mode select teeth and jawbone and press ctrl+J to join the meshes together.
That may not be the best way to do it, but that's what I'd do. Though I hope I didn't misunderstand your question.