The answer for the question is quite simple - Diffuse setting, but the main problem might be the scene light setting, as part Why do this behavior happen? explained.
You can change the diffuse setting in
Properties Panel > Object > Cycles Settings >> Ray visibility : Diffuse
The color you see is mainly from the diffuse ray in that orange color, by default it light up nearby objects like it is an emitting material. It is not a mistake but a natural behavior in real world. The most easily way to FIX this problem is to turn the diffuse behavior down. Making the material won't bounces light in the diffuse light path.
Turn off the diffuse ray will cause the rendered scene not accurate, making the render image seems to be wrong in some certain way.
Material Node Setup
You can also change that behavior in material node editor as follow:
With the Input > Light Path node, using diffuse value, you can change that material by using a mix shader with is diffuse factor. If you replace it by a white transparent shader, it will work as turn off the diffuse setting I mentioned above.
Why do this behavior happen?
And another issue is that in this scene, you have your light and environment poor set up. Making it too simple from some certain light path, in this case, the orange diffuse bounces.
The main reason of that is maybe you are using point/sun light, and toward a specified angle, which cause the white wall capture a lot of bounces from orange roof top.
You can change the angle of it, or you might also want to add more light around your scene, and object around it, making the light path more balance. Changing the radius of the point light may help as well.