I made the objects by creating a cube in Object mode, going into Edit mode, deleting the vertices, clicking shift+LMB new vertices, making a face and extruding.
On the light colored objects, I can add a Boolean modifier and set it to Difference, and it would make a hole in the base object the shape of the new object. However, inspite of these dark objects were made the same way as the lighter ones, when I try to apply a Boolean modifier, it deletes the base object.