2
$\begingroup$

I have a question about how colors are stored in computer graphics. I had watched this video previously which had explained about why colors are stored and displayed differently than one would assume in computer graphics : https://www.youtube.com/watch?v=LKnqECcg6Gw

They explain that, as a consequence of camera capturing color values, but storing the color values as roots instead, 1) more space is saved 2) more black colors are crammed in with 8 bits than white colors, and I heard this topic was important for computer graphics as I heard quite a few apps dont take the square rooting into consideration when performing blurring operations

Here is where I am extremely confused. I have little intuition for why it works this way. Just because we store square roots, why is more space saved?

And mainly, how do we store more blacks just because we store square roots? I decided to calculate the root of values from 0 to 255 in Python and I found out the the difference between the root of x and the root of x+1 becomes smaller and smaller the larger x is

Should this not logically mean that since we try to be more precise storing the higher or whiter values since the difference between square roots of white values is smaller and smaller, thus we would have MORE white values than more black values?

$\endgroup$
1
  • $\begingroup$ You can actually instruct photoshop to do it with linear color assumption. (But you may want to use 16-bits per channel color if you do) $\endgroup$
    – joojaa
    Jul 18 at 18:19
4
$\begingroup$

In Computer Graphics, we most often represent colours using a red, green, and blue value, also known as RGB. And, in Computer Graphics, we generally represent those red, green, and blue values with numbers ranging from 0 to 1. A value of 0.0 means 0% brightness or pure black. A value of 1.0 means 100% brightness or pure white.

Now, you might think that middle gray, what we see as precisely in the middle between black and white, would be a value of 0.5 or 50% brightness. However, this is actually not how our eyes perceive the world. Our eyes perceive the world in a non-linear fashion. So, middle gray is actually 18% brightness or a value of 0.18 (and some might even say it's 12% brightness). An object that is middle gray, reflects 18% of the incoming light.

This isn't problematic, but storing decimal numbers ranging from 0 to 1 does take up quite a bit of space. So, we can store it as a whole number using a few bits. Using 8 bit numbers ranging from 0 to 255 is very populair, mainly because it works nice with computers and in the vast majority of the cases we cannot see that the image only had 256 possible values for red, green, and blue. And that's where we get into trouble.

Our eyes roughly perceive the same amount of tonalities between pure black and middle gray and between middle gray and pure white. However, 18% middle gray converted to an 8 bit number is 46. Which means that for all values between pure black and middle gray, we have 46 possible values and from middle gray to pure white we have 210 possible values. That is a huge difference. What's going to happen is that all values above middle gray are going to look perfectly fine, but values below middle gray are going to get banding artifacts. Here's a foto from pexels by Frans van Heerden. On the left is what it should look like and on the right is what it might look like if we just convert it to an 8 bit number directly. Image showing quantization artifacts You can see that the brighter parts of the image stay about the same, however the darker parts of the image are heavily distorted and you can see that there are only a few possible values. Clearly, this isn't what we want.

So, how do we fix this? Well, the answer is to have values between pure black and middle gray to have about the same amount of possible values as values between middle gray and pure white. How do we do that? We use a transfer function (also called OETF or inverse-EOTF). A transfer function simply remaps the input values to output values. In our case, the transfer function will make sure that values above and below middle gray will get the same amount of possible values. Two flavours of transfer functions are very popular, gamma (like sRGB and Rec.709) and log (mostly from cinema cameras) based functions. The square root that you saw in the video is essentially a gamma transfer function. Here are a few graphs showing linear, gamma and log transfer functions.Various transfer functions You can see that gamma and log will more evenly distribute the 256 possible values that we have between pure black and middle gray, and middle gray and pure white. One thing you might notice is that transfer functions actually operate on the 0 to 1 numbers, and that is true. You first apply the transfer function and then you convert them from 0 to 1, to the 8 bit number ranging from 0 to 255.

In a mathematically correct program, we want to use numbers ranging from 0 to 1 whenever we are doing things with the RGB values (like changing exposure, white balance, curves, saturation, and so on), since the formulas that we use assume that. Whenever we want to store the image to a file (like a PNG or JPEG file) or when we want to send it to your monitor, we will apply the transfer function and then convert it to an 8 bit number ranging from 0 to 255, in order to save space.

When it comes to which transfer function to use, we look at colour spaces. Colour spaces define how to interpret the RGB values. An RGB value of 255 red, 0 green, and 0 blue is obviously red, but what kind of red? There are a lot of reds that all look red, but they are different. The same is for green, blue, and even white. Additionally, is a value of 128 for red, green, and blue middle gray or maybe some other brightness? Colour spaces define all of this. They say what kind of red red is, what kind of green green is, what kind of blue blue is, what kind of white white is and what the transfer function used is. So, all we need to make sure is that we use the transfer function from the same colour space as our monitor or image file and the resulting image will look correct.

So, why does Photoshop give the wrong results according to that video and why does it say that computer colour is broken? Well, computer colour isn't broken. However, all of the formulas that we (and Photoshop) use, all want linear values between 0 and 1. But, Photoshop doesn't do that. It uses the 8 bit numbers directly, but those have that transfer function applied. This means that the resulting values aren't mathematically correct.

But, why does Photoshop do that? Well, most likely, it's because Photoshop was created a long time ago, when computers were slow. So, to take the 8 bit numbers, convert to numbers between 0 and 1, then apply the inverse of the transfer function, then whatever colour operation (like blurring), then apply the transfer function, and lastly convert it back to 8 bit numbers, well that is going to make it unbearably slow. So, they simply didn't do that and performed the colour operations on the 8 bit numbers directly. The results, while incorrect, looked good enough, so they never bothered to fix it. Nowadays, Photoshop is such a large application that to try and fix this now, would be out of the question. Computer colours aren't broken, Photoshop just does it the wrong way.

To conclude, when actually working with the RGB values themselves, we use values between 0 and 1. However, when storing them to a file or sending them to our displays, we use 8 bit (or sometimes 10 bit) numbers in order to save space. To make sure that the images still look good, we have to make use of transfer functions. Some applications decide to do things the mathematically wrong way, just because it is faster and they think the results are good enough (but not as good as doing it the right way).

$\endgroup$
10
  • $\begingroup$ Im confused. You had said we use these transfer functions to evenly space out all the blacks and grays and whites from 0 to 255 right? But in the video they say more colors are dedicated specifically for black because of the transfer function. Arent both cases then different? $\endgroup$
    – Hash
    Jul 18 at 10:33
  • $\begingroup$ @Hash bram0101 was being inaccurate. The spacing is not uniform, it matches the curve. For optimality wrt human perception the curve ought to match the sensitivity of the human visual system (it's not linear, see Weber's law). So the simplest answer is that you want to store data in less space in a manner that best exploits peculiarities of the human visual system. You can see the transfer function + quantisation step as non-uniform quantisation, i.e. errors are not all equal in different intervals, and instead one tries to weight them according to human perception. $\endgroup$
    – lightxbulb
    Jul 18 at 11:22
  • $\begingroup$ @lightxbulb Sorry, that is the exact thing I dont understand. I know our eyes are more sensitive to differences in black colors as opposed to brighter colors. But how do these transform equations happen to store more colors or dedicate more of the bits for darker colors? Intuitively, how does the function logically dedicate more of the 255 bits to darker colors? $\endgroup$
    – Hash
    Jul 18 at 11:42
  • $\begingroup$ @Hash Let $v$ be in $[0,1]$. You want to quantize it, e.g. to assign to it a corresponding number in $0-255$. So you can imagine you split $[0,1]$ in $256$ equal intervals/bins: $[0,\frac{1}{256}), \ldots, [\frac{255}{256}, 1]$, and the quantization index is the number of the bin $v$ falls in. This is known as uniform quantization, non-equal-sized bins result in non-uniform quantization. Applying a map (transfer curve) $f$ to your data and quantizing is equivalent to applying $f^{-1}$ to the bins, creating non-uniform ones. Many small bins in an interval correspond to more colors there. $\endgroup$
    – lightxbulb
    Jul 18 at 14:03
  • 1
    $\begingroup$ @bram0101 if you want to teach OP then you need to say how to read the graph. But anyway quite a lot of things in graphics is implemented wrong not just photoshop. We still teach people things like the Bresenham line drawing that primes people to think wrong. As a result quite a lot of 2D applications do AA wrong. But the thing is as long as you think of a application as drawing then it does not matter, theres a human in the loop to drive and act as a adjuster. Its only if you want to move beyond this and do some scientific, physics or replicate real processes when this starts to matter a gre $\endgroup$
    – joojaa
    Jul 18 at 17:35
1
$\begingroup$

The video is over simplifying a problem that has a long history and is far more complex then just "how a camera captures images". Especially today when we have cameras that can capture flat out amazing amounts of data.

If someone shows you a color wheel, that never changes, then starts changing the brightness of the light illuminating that color wheel, and you press a button every time you see a difference on the wheel, you will press the button more often when the lights are dim, then when they are bright. It's simple physiology, so doesn't it seems reasonable to capture, then subsequently reproduce more dark colors then bright colors with our cameras and monitors?

The bias is intentional, it is built into the system on purpose to make up for the short comings of technology, and take advantage of properties of our eyes. Yeah, modern equipment has over come many of they limitations but the vast majority of installed monitors in the world still have 8 bits of color built into them.

And that is the reason for the "square roots", its actually not square roots but lets not go there.

Here is a fairly detailed explanation of how the "compression" works:

To understand the compression, first lets normalize the range of values to be between 0 and 1. And knowing that the normalized values between 0.0 and 0.25 will have more visual impact then values between 0.75 and 1.0. Then what math function will "stretch" out the space for lower values, and "compress" larger values? Exponents do this quite neatly. For example $0.1^2=0.01$ while $0.2^2 = 0.04$ for a difference of $0.03$ so the values only moved a little bit along the number line. But closer to $1$ at say $0.8^2=0.64$ and then $0.9^2=0.81$ for a difference of $0.17$ so values close to 1 move a lot more then values close to zero. But the input values had the exact same difference of $0.1$. This is the opposite of what we want, so instead use the sqrt for our "compression" and then square the compressed values for "decompression". Here is an example:

Divide the range between 0 and 1 into 10 equally sized sections, call them buckets, then each bucket will be 0.1 in size. (computers use 256 buckets for each color) And we can compute a bucket. $sqrt(.1) = 0.316$ and $sqrt(.2)= 0.447$ so the nearest buckets are 3 and 4 respectively. While for $sqrt(.8)=0.894$ and $sqrt(.9)=0.948$ they both have the nearest bucket of 9! And that's it, the dark values are spread out into more buckets then the bright values.

It really is that simple, the computers aren't actually doing any compression, they are just doing some math on each value. .

$\endgroup$
4
  • $\begingroup$ 'you will press the button more often when the lights are dim, then when they are bright. It's simple physiology, so doesn't it seems reasonable to capture, then subsequently reproduce more dark colors then bright colors with our cameras and monitors?' Yes I do understand that , but how does us using square roots or something similar exactly help us capture darker colors more? $\endgroup$
    – Hash
    Jul 18 at 10:08
  • $\begingroup$ I added an explanation of the compression. $\endgroup$
    – pmw1234
    Jul 18 at 12:02
  • $\begingroup$ So in the first case, 0.1^2 = 0.01, 0.1 is the actual color captured by the cam and 0.01 is how this hypothetical PC that uses such model, stores the color, square rooting it back to display the original color? $\endgroup$
    – Hash
    Jul 19 at 14:09
  • 1
    $\begingroup$ yes. (there are other steps involved in the actual process, like adjusting the camera signal to match a standard) $\endgroup$
    – pmw1234
    Jul 20 at 12:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.