If your glass is always just planar and you don't need to use refraction for artistic reasons (all the light that enters your glass just goes to the other side in a parallel fashion, or gets reflected), then you can replace the Glass material with a material that is mostly transparent, but reflects at extreme angles:
"flat" glass material
when you use this material, your window needs to be just a plane, not a solid.
These are the two options compared: a) glass shader on a solid window, b) "flat" glass material (see above) on a flat (2D) window.
As you can see, reducing the unnecessary light path calculations "through the glass" makes the render less noisy.