I've been running into a problem where whenever I increase the bump the shiny specular goes into a halo shape, here's a gif to show it in action
At default settings, this:
is 1 meter. To illustrate, here are a couple of cubes, one with Bump applied, and one with some Subdivision and Displacement.
In essence, Bump is perturbing the normal data to give the left cube the appearance of the right cube without adding any geometry. That's probably not even anywhere close to the appearance you had in your head for this wall/window, and so you see something you don't like and can't quite explain in the viewport.
By lowering the Strength you are mitigating the issue, but not solving the root cause. Instead, the Distance for this (imo) should be somewhere around .001 to .01, and that may be enough to solve it.
For reference, .0254 (2.54cm) is an inch, so when you're doing stuff like wood grain and stucco you're going to have really small values. The image above got just a little destroyed by compression, but you can see that with the values scaled down the cubes retain similar appearances.