It looks like there are some places where the distortion is focused. There's someplace where Blender just can't give us a flat map, and it's distributing the distortion across the map a bit to make it so that no one place is awful. If we enable sync-- those doubled diagonal arrows in the upper left corner on my UV editor below-- then select the distortion plus a bit more, and hide everything else, we can get a closer look at the problem area:
So when we look more closely, we can see that there are some places where we have what amounts to an unseamed box, at this particular doorway. Let's seam appropriate edges so that we can unwrap this without distortion:
The selected areas are places that need seams. One way to think about seams is where we'd need cuts if we were going to wrap this object up for Christmas. For things that are roughly box-shaped, sometimes I think about making fold-up dice as a craft project as a child.
Now that we've created a few more seams, we can unwrap and see how we did. Don't forget to unhide everything if you don't want to create more islands than you need.
Much better! There are still a few issues which you didn't ask about (those lines stretching across the UV map). You should take a look at those as well, using UV sync to find them and then hiding stuff so you can see what's happening. (Hint: they're due to non-manifold geometry. You can find non-manifold geometry by using the "select non manifold" operation and learning what its options do.)