Ok, I found the solution. Turns out you can hit Apply on the Boolean modifier to actually apply the effect. Then the modifier is gone and the vertices are created in the cube object. You can do that by hitting this button in the modifier tab:
After exporting the STL looks like expected, great
