Rather than connecting the brick texture to the colour input of your shader, you can use it to define the mix factor of different colours. With this approach you can use a colour ramp node to adjust the shading of the brick texture so you get less variation in the colour.
By connecting the fac of the brick node into a colour ramp you easily get a choice between brick and mortar (assuming you have a mortar size)
Then connect the colour output to a colour ramp that is set to constant interpolation and adjust the positions until you get a variation you are happy with.
Connecting the black and white colour from the colour ramps into the fac of the mix nodes lets you choose which colour is used.
Which leaves you with two brick colours and a mortar colour.