But If I try to delete it, I siply delete this face (I can't select it anymore after deleteting), but I do not see through hole, so the basic cube face is still here. So how to remove the hole bottom?
Edit: Actually what I said was not completely accurate, so:
Other than that you can indeed use the faces of a mesh to dig a hole in the same mesh, using the ctrlF > Intersect (Boolean) tool. Just push the face a bit further down.
Or use a second mesh: