This feature is available from version 2.90 (https://builder.blender.org/download/)).
It's called Extrude Manifold and is located in the toolbox T, under the "Extrude" dropdown, or by pressing Alt+E.
↑ Here I'm using Face select mode (3) but it works in vertex select too
Blender Pre 2.90
You are expecting Extrude to work in the "Destructive" or "Push" mode,1 i.e to be able to remove faces (and portions of them) as well as creating them. Unfortunately, Blender's Extrude doesn't currently offer that mode of operation.
For older versions of Blender, namely 2.79, there is a free user-contributed add-on that does that: Destructive Extrude for Blender, but it hasn't been updated for Blender 2.8x.
To model a mesh like the one you're after, you luckily have a very easy alternative: instead of extruding inwards, extrude outwards!
1. Using terminology from the Blender addon and from SketchUp respectively.