Color Spaces

by Rishidev Chaudhuri

I have a child's affection for color, for broad swathes of bright saturated colors, for unapologetic reds and greens and blues and yellows. And yet this strong visceral and emotional reaction often feels immature and undifferentiated. I never really learned the names of the colors properly. This is partly from an artist mother, who would always give me very specific answers when I asked for the name of a color (burnt sienna, cerulean), so that i never quite figured out the broad categories correctly, didn't really remember the specific terms, and came to experience the names of the colors as magical incantations that descended upon sensory impressions according to uncertain principles. And this is partly the result of decreased red-green sensitivity, so that while I can tell pure reds and greens apart easily, and can distinguish expanses of color, I start to stumble at blue with a little bit of red or green added, or at intermediate points along a red-green mixture, or at thin lines of color¹. Recently, I've been making graphs that require a large number of data traces on the same figure, and I need each trace to be a sufficiently different color that I can easily tell them apart. And so I've found myself paying more attention to the way colors are described and how to get them on a computer.

It has been known for some time that colors can be described by three numbers. If I show you light of a certain color and ask you to match it by combining lights of three other colors and varying their intensities, you'll typically be able to find a combination that looks indistinguishable. But the wavelengths you combine might be very different from the wavelengths I showed you. Light of the wavelength corresponding to yellow and light of the right combination of red and green wavelengths will look the same, even though they are physically quite different. This structure is reflected in the retina. For the most part, we have three types of color-sensitive cells (cones) and so make three measurements of any color we see, corresponding to light centered around three different wavelengths. Informally, these are said to be peaked around blue, green and red, though the peaks don't quite line up at these colors. Any information that isn't captured in these three numbers is literally invisible. Dogs and cats (and most mammals) measure only two numbers to make a color, rather than three, and seem to see like red-green colorblind people. There is some speculation that a subset of women have cones that make measurements at four frequencies and so differentiate colors that look identical to most people.

There are all sorts of complexities and caveats of course.

Color input is processed at every stage of the nervous system, starting from the retina, and the experience of color is affected by numerous contextual and historical factors. Nevertheless, in broad strokes we can describe a color by three numbers. A common trope in science and math is that if we can describe an object by a set of numbers, we can think of these numbers as its coordinates in an abstract space, with each point in the space corresponding to a different object, and we can study the geometry of the space². The number of numbers needed is the dimension of the space. We can describe a given mix of wavelengths of light by specifying one number for the intensity of each wavelength; this space has many dimensions (possibly infinitely many). Color, on the other hand, lives in a three-dimensional space.

So how should we choose these numbers? One possibility is that we measure how much a particular color activates each of the three types of cone cells, and use this as our set of numbers. But this isn't the only way. For example, we could take the three numbers corresponding to the activations of the three types of cells and represent a color by the sum of the first two numbers, the difference of the second and the third and the sum of the third and the first. Again, this is a common mathematical trope: given a space we can describe points in it by various systems of coordinates, and different systems offer different insight. A system that describes what the retina does, for example, might not be best for describing how higher parts of the brain process color.

Monitors typically mimic a desired color by combining varying amounts of red, green and blue light, and so a natural system to describe a color emitted by a monitor is the triplet of numbers indexing the intensities of red, green and blue. Such a system is called RGB. While this organization is reminiscent of the three types of cells in the retina, it is not the same. Most significantly, this system is usually specified in terms of the monitor, not the eye, and different devices might use different red, green and blue lights. The three numbers describing the amounts of red, green and blue are usually chosen between 0 and 1, with 0 corresponding to the absence of the color and 1 corresponding to its maximum intensity. So each number runs between 0 and 1 and the color space defines a cube, with each point in the cube being a different color. See the picture of the cube above on the right (taken from Wikipedia).

Another common trope in thinking about coordinates and spaces emerges from the relatively mundane observation that it's easier to imagine things in fewer dimensions. In particular, if you have a three-dimensional space it's often most easily understood by reducing it to a collection of two-dimensional spaces³. For our color cube, this corresponds to taking a series of slices through the cube. Each of the slices I've shown on the right comes from holding one of the coordinates constant and plotting the two-dimensional space that results. So the slice in the top left corner has no red in it and all possible combinations of blue and green. The slice in the top right corner has the maximum amount of red in it and all combinations of blue and green, and so on. We can even look at the one-dimensional spaces that go into a slice. For the slice in the top left corner, the colors along the bottom edge of the slice have no blue in them and are just greens; the colors along the left edge have no green in them and are just blue.

RGB corresponds quite naturally to what a monitor does and seems broadly similar to what the cone cells in our retina do. But it seems unnatural to look at a color and think of it in terms of combinations of red, green and blue (you can see this in the slices). Another coordinate system, the hue, saturation and value color system (HSV) tries to separate out what we would name a color (the hue, such as “orange” or “green”) from how bright or dark it is (the value) and how much of the color there is (the saturation). Value is typically measured from 0 (black) to 1 (white, but also bright colors). Saturation also goes from 0 (various sorts of grays, including black and white) to 1 (pure colors). Hue is described by a color wheel, and goes around in a circle, with red at 0 degrees, yellow at 60, green at 120, cyan at 180, blue at 240, magenta at 270 and red again at 360 degrees. While RGB traces out a cube, these coordinates draw out a cylinder, with value usually chosen to go from bottom to top, saturation chosen to go from inside to outside and hue going around the cylinder. The picture below is from Wikipedia:

And here are four slices, taken from different levels of the cylinder, with each corresponding to a different value:

For the most part, these ways of describing color space can describe all the colors we see. But there are number of problems with them. First, they correspond to what a monitor does, not what a human sees. The same combination of numbers might mean different things on different monitors. But this can be accounted for by setting an appropriate standard. More interestingly, while the axes of the coordinate system roughly correspond to things we experience (at least in HSV), there are many differences. For example, value can reflect perceived lightness poorly, and changing hue but not value can still seem to change how light or dark colors in an image look. A related problem is that these spaces often don't encode perceptual distance well. Given a pair of colors, a good set of coordinates would encode perceptual distance, so that the perceived distance between the colors would be reflected by the distance in coordinate space (calculated by drawing a line between the points corresponding to the two colors)⁴. If this were true, small changes in color would correspond to small changes in the numbers used to describe the color and large changes in color would correspond to large changes in the coordinates.

These shortcomings suggest explicitly seeking a color space based on perceptually relevant dimensions. This is harder, but more interesting. As I mentioned, to be able to describe a point in color space we just need three axes. Almost any choice of axes will give a usable coordinate system: all we need are three “directions” in color space, and we could get these in a number of different ways. But for the directions to encode something of particular perceptual relevance, we need to know more about human color vision beyond the fact that it is three-dimensional.

The most common such system is CIELAB, which is based on the theory that the brain processes input to the three cones not by simply tabulating the amount of light incident at the three different wavelengths, but instead by computing the differences between these responses. So rather than computing the amounts of red, green and blue, we locate colors on a red-green axis, a yellow-blue axis and a light-dark axis⁵. The three coordinates (L*, a* and b*) in this system correspond to values along these axes. L* is near 0 for very dark colors and near 100 for white; a* is negative for green colors and positive for red colors; and b* is negative for blue colors and positive for yellow ones. Moreover, in this coordinate system the distance along a straight line between coordinate points is meant to correspond to perceived distance between the colors they represent. Rendering this space is more complicated, and the slices take different shapes depending on where in the space you are, so I'll stop here.

A lot of this discussion has been simplified, intentionally and perhaps unintentionally (I'm far from an expert). But computers make color very easy to play with. One way is to open an image editing program and mess around with the sliders in the color generation feature. Or, if you program, most high-level languages allow you to make plots and easily select colors by giving their coordinates in one of several different systems, which is how I generated the slices shown above. There are also a number of color meter programs (one of which comes preinstalled on Macs) that let you hover your cursor over a spot on the screen and see the color underneath described in various coordinate systems.

Finally, I should reemphasize that our perception is a complex blend of context, history and expectation. We fill in colors that are not present, adapt to ones that are and highlight certain distinctions while ignoring others that we are capable of. And none of this touches upon how social and linguistic factors shape our experience of color, or how colors get assigned to names and categories. Coordinate systems don't really help me decide whether something should be described as reddish blue, blueish red or purple. We could divide a reasonable color space into equally spaced segments, of course, and give those the names of colors, and this would tell us something about how we actually decide to name colors, but at the cost of circularity (since our idea of a “reasonable” color space already relies on how we break up color space in language). And so we're left with the names of the colors on the one hand (tangerine and burnt orange, fuchsia and periwinkle) and a set of mathematical and biological coordinates on the other, and our perception somewhere in between and beyond, always on the verge of yielding to analysis and always slipping away.