5
$\begingroup$

Why is this?

For example, if I type in 0.65 to one of the RGB channels, after pressing Enter it changes to 0.651:

enter image description here enter image description here

This sort of thing seems to happen for all different color settings in Themes, even RGBA ones:

enter image description here enter image description here

What is the technical reason for this?

$\endgroup$

1 Answer 1

6
$\begingroup$

internally the value is stored as an 8-bit value (char C type, range 0..255). This means to get from the 0..1 range to the actual stored value, multiply by 255:

0.65 * 255 = 165.75 ~ 166 (rounded to integer)
0.1  * 255 = 25.5   ~ 26  (rounded to integer)

When 166 is in turn displayed in 0..1 range again, it gets divided by 255:

166 / 255 = 0.650980392... ~ 0.651 (rounded to 3 digits)
26  / 255 = 0.101960784... ~ 0.102 (rounded to 3 digits)

So the double rounding when entering and then displaying the value causes this slight error. If you read the same value and set it back again these errors will mount up, but otherwise it's not a big problem.

In addition to cases like this where a color value is actually an 32-bit integer (8 per component) internally, there is also the usual issue of floating point representable numbers. But in this case the integer rounding overshadows it.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .