I've been working through the new abandoned house tutorial from Blender Guru and I'm having issues with the siding. When I add the Boolean the siding does all kinds of weird things. When I scale along the normals it creates gaps in the siding as shown in the first picture. Without scaling the siding has gaps as shown in the second photo. I've also tried reordering the modifiers and it creates the issue in the third photo. I don't know what else to try. I've included a copy of the blend file. Any help would be greatly appreciated.
.blend file is here - Blend File