Chapter 5: Color Perception: How We See Color

Welcome to Last Minute Lecture.

This free chapter overview is designed to help students review and understand key concepts.

These summaries supplement not replaced the original textbook and may not be redistributed or resold.

For complete coverage, always consult the official text.

Welcome back to the Deep Dive.

Today we are tackling something that is literally right in front of your eyes.

In fact, unless you're listening to this with your eyes closed, it is dominating your perception of reality right now.

Absolutely is.

But here is the kicker.

It might not actually exist in the way you think it does.

It's one of those topics that feels incredibly simple, intuitive, even until you scratch the surface.

Right.

Then suddenly you're deep in philosophy, physics and neuroscience all at once.

Exactly.

We are talking about color and to get us started, I want to throw a paradox at you.

The source material for this deep dive calls this the orange paradox.

A great place to start.

I want everyone listening to just visualize their breakfast table.

Picture an orange sitting there, bright, textured,

undeniably orange.

You pick it up, you peel it.

It looks orange.

Simple enough.

It's a fruit.

It's a color.

Nothing complicated there.

Or so you think.

Now imagine you take your smartphone,

you snap a photo of that orange and you text it to a friend.

You look at the photo on your screen.

And it also looks orange.

It also looks orange.

And common sense tells us, well, of course, the orange is orange.

So the photo is orange.

It's just a picture of the thing.

It should look like the thing.

But here is the aha moment that our source material leads with.

And it's a bit of a brain breaker.

The physical basis of those two experiences,

the real orange and the photo of the orange is completely different.

Completely.

The light bouncing off the real fruit is a messy, complex soup of different wavelengths.

All sorts of light waves are hitting it from the sun or your kitchen light.

And a specific jumble of them are bouncing off that peel into your eye.

A continuous spectrum of light, yeah.

But the light coming out of your phone screen.

That's totally different.

It's just a tiny, rigid grid of red, green and blue pixels.

Little microscopic light bulbs, essentially.

Physically they're nothing alike.

One is nature's messy fingerprint and the other is a highly structured technological trick.

Exactly.

One is a continuous spectrum.

The other is a digital illusion built from just three colors.

But to you, to your conscious mind.

They're both just orange.

They're both orange.

And that leads us to the fundamental truth of this entire deep dive.

If you, listening, take away only one thing, it should be this.

This is the big one.

Color is not a physical property of things in the world.

That is such a hard pill to swallow.

I look at a fire truck and I think,

that truck is red.

You're telling me the red isn't on the truck.

Not in the way you think it is.

The truck's paint has physical properties for sure.

It reflects certain wavelengths of light.

Physics gives us wavelength.

Right, nanometers.

But red.

Red is a mental event.

It's a computation your brain performs.

Okay, that's wild.

There is a fantastic quote in the text from a researcher named Stephen Chevelle that sums this up perfectly.

Let's hear it.

He says, there is no red in a 700 nanometer light, just as there is no pain in the hooves of a kicking horse.

Oh, I love that image.

Isn't it great?

The pain isn't in the horse.

The pain is in you when you get kicked.

Precisely.

So the red isn't in the light.

The red is in you when the light hits your eye.

Color is the result of a nervous system interacting with a stimulus.

It's a translation.

And today, we are going to walk through exactly how that translation happens.

We have a mission.

We do.

We are going to dive deep into chapter five of Sensation of Perception, the sixth edition.

We're going to follow the path exactly as it's laid out.

We'll trace the signal.

We're going to start with the physics of light, then move to the biology of the eye, then dive into the messy wiring of the brain.

And then finally, back out to the real world of berries, dresses, and peacocks.

So buckle up.

We are going from nanometers to neuroscience.

Let's start with the basics.

Section one, the physics.

We say we see light, but what are we actually seeing?

Well, we're seeing a very, very narrow slice of the electromagnetic spectrum.

The universe is absolutely humming with energy waves.

You've got gamma rays, x -rays, microwaves, radio waves.

The whole gamut.

The whole thing.

But humans, our eyes are only tuned to perceive wavelengths between about 400 and 700 nanometers.

Which is tiny.

If the electromagnetic spectrum were a road trip from New York to Los Angeles,

the part we can see would be, what, a couple of inches wide.

Something like that.

It's a very specific biological window we've evolved to look through.

And crucially, as you said before, most of the light we see is reflected light.

Right.

Unless you're looking directly at the sun, which please don't, or a light bulb, you are seeing light that has bounced off something else to get to you.

So let's go back to that sire truck.

Sunlight or white light hits the truck.

That light contains all the colors, right?

The whole rainbow is packed into it.

Roughly, yes.

White light contains that full visible spectrum.

When it hits the truck, the red paint acts like a kind of filter.

A filter.

How so?

It absorbs, or you could say it soaks up most of the short and medium wavelengths.

The blues, the greens, the violets, they all get trapped in the pigment of the paint.

So they get absorbed.

They don't bounce back.

Exactly.

They're subtracted from the light.

The only thing that really escapes, the only thing that bounces back efficiently to your eye are the long wavelengths, around 600 to 700 nanometers.

And our brain translates that specific wavelength into the experience of red.

That's the translation.

So a dark object like a black t -shirt is just a light sponge.

It's absorbing almost everything.

Essentially, yes.

The more light a surface absorbs across the whole spectrum, the darker it appears.

The color we see is just the leftover mix of wavelengths that survive the impact and make it to our eyes.

Okay, so the light bounces off the truck and it hits my eye.

What happens next?

The source material breaks this down into three distinct steps.

And I think this framework is really helpful for keeping us organized.

It is.

It's a great roadmap for the chapter.

The three steps are detection, discrimination, and appearance.

Detection.

That's step one.

That's just catching the photons, right?

Correct.

Your eye needs a sensor to even know the light is there in the first place.

That's detection.

A photon has to be captured.

But just detecting isn't enough.

That leads to step two, which is discrimination.

Right.

You need to be able to tell the difference between one wavelength and another.

You have to distinguish the long wavelength from the short one to see red instead of blue.

And finally,

step three.

Step three is appearance.

And this is a really complex one.

That's about assigning a stable, perceived color to a surface and making sure that fire truck still looks red, whether it's under the bright noon sun or a yellowish street lamp at dusk.

Okay.

That's a great roadmap.

Let's zoom in on step one, detection.

This is where we get into the hardware of the eye, the photoreceptors.

We've talked about rods and cones before, but for color, the cones are the absolute stars of the show.

Right.

Rods are for night vision.

Cones are for daylight and color.

And in most humans, we have three different types of cone photoreceptors in the retina.

Three types.

Now, this is where we have to be careful with our language because we usually call them the blue, green, and red cones.

We do.

And it's a useful shorthand.

But the text is very specific that this is a bit misleading.

It is because it implies they only see that one color, which isn't true at all.

It's much better to call them S cones, M cones, and L cones.

S, M, L standing for short, medium, and long wavelengths.

Exactly.

It's a more accurate description.

It refers to the physical property of the light they're most sensitive to, not the color experience they create.

Okay.

So let's break them down.

S cones.

S cones have their peak sensitivity at about 420 nanometers.

That's the short blue -violet end of the spectrum.

They're also the rarest of the three, by the way.

And encodes.

M cones peak around 535 nanometers, right in the green region of the spectrum.

And finally, the L cones.

L cones peak at about 565 nanometers, which is in the yellowish -red region.

Now I'm looking at figure 5 .1 in the text, which shows the sensitivity curves of these three cones.

Basically, it's a graph showing how strongly each cone fires when it gets hit by different wavelengths of light.

And something jumps out at you immediately.

These are not neat little separate buckets for each color.

Not at all.

They're messy.

There is massive overlap between them.

The S cone is kind of off on its own little island at the short end of the spectrum.

It doesn't overlap a ton with the others.

Right.

It's a bit of an outlier.

But the M and L cones, they're practically sitting on top of each other on this graph.

They are incredibly close.

I mean, think about it.

The peak for the M cone is 535 nanometers.

The peak for the L cone is 565 nanometers.

That is a tiny, tiny difference in the grand scheme of physics.

So if a light wave comes in at, say, 550 nanometers right between their peaks, what happens?

They both go crazy.

Both the M cones and the L cones will fire very strongly.

It's not one or the other.

It's both.

Just at slightly different rates.

Okay.

But why would nature design it that way?

It seems really inefficient to have two different sensors doing almost the exact same thing.

It seems that way, but that's where evolution comes in.

It's a fascinating story.

It turns out the green M cone is actually a genetic duplication of the red L cone.

A copy.

A copy that happened relatively recently in our primate evolutionary history, maybe about 30 to 40 million years ago.

Wait, so they're basically genetic siblings?

They are.

They come from the same original gene.

Before that split, our distant ancestors were what we call dichromats.

They were essentially red -green colorblind.

So they only had two cone types.

That's right.

They had the S cone and a single long wavelength cone that covered the whole red -green part of the spectrum.

So the world was basically just blues, yellows, and grays to them.

Pretty much.

Then a mutation on the X chromosome copied that long wavelength cone gene.

A few more mutations shifted its sensitivity just a tiny bit, and suddenly, boom, we had a new channel.

A third cone.

A third cone.

And suddenly, a whole new world of color opened up.

We could now distinguish a red berry from a green leaf, which is a pretty big deal if you're a primate looking for lunch.

So that's why they overlap so much, they're just slightly different versions of the same original sensor.

Exactly.

The overlap is an artifact of their shared history.

Okay, so we've got these three buckets, S, M, and L catching light.

That seems like enough to see color, doesn't it?

Yeah.

If the L cone fires, I see red.

If the S cone fires, I see blue.

Why do we need step two?

Discrimination?

This seems solved.

Ah, but it's not.

Because of a very sneaky, very problematic issue in vision science called the principle of univariance.

The villain of our story.

It really is the villain.

Right.

I mean, if you don't understand the problem of univariance, you can't understand why we need three cones at all.

You can't understand why color vision works the way it does.

Let's break this down, because the tech spends a lot of time on it using figures 5 .2 and 5 .3.

It asks us to imagine we only had one type of photoreceptor.

Let's say just the M cone.

Right.

Let's run that thought experiment.

Yeah.

You are a single M cone photoreceptor.

Your entire job is to tell the brain what's happening out in the world.

But you have one fundamental limitation.

And what's that?

You're essentially a bucket catching rain.

Okay, I'm a bucket.

I can do that.

You can tell the brain how much water is in the bucket at any given moment.

You can signal, hey, brain, I'm half full or I'm totally full.

But you cannot tell the brain how the water got there.

I don't know if it was a light drizzle that lasted for an hour or a massive fire hose that blasted me for two seconds.

The end result is the same.

I'm half full.

That is the perfect analogy.

Envision the drizzle might be a color you aren't very sensitive to, like a reddish light, but there's a lot of it.

It's very bright.

The fire hose might be a color you love, your peak sensitivity in the green range, but it's very dim.

So let's put some numbers on it to make it concrete.

Let's say I'm that M cone.

I love 535 nanometer light.

That's my absolute favorite.

Okay.

If a dim 535 nanometer light hits you, you fire, let's say, 20 times a second.

That's your signal to the brain.

20 clicks.

Got it.

Now, let's take a light you don't like as much, maybe 600 nanometers over in the orange part of the spectrum.

You aren't very sensitive to it.

Normally, a dim orange light would barely make you fire at all.

Right.

But if I crank up the intensity of that orange light, if I make it blindingly bright, I can force you to fire at the exact same rate, 20 times a second.

Oh, I see the problem.

The brain is sitting there listening to me.

I'm shouting, 20 clicks, 20 clicks.

The brain asks, hey, is that a dim green light out there or is it a really bright orange light?

Then what do you say?

I basically have to shrug and say, I don't know, I'm just a bucket.

All I know is how full I am.

That is the principle of univariance.

A single photoreceptor's response is, by itself, ambiguous.

It can only tell you how much light it absorbed, not what wavelength that light was.

It hopelessly confuses intensity with wavelength.

And the text gives a perfect real -world example of this that we have all experienced,

night vision.

Yes.

At night, in what we call scotopic conditions, dim moonlight starlight, the cones don't have enough light to work, so we switch over to using our rods.

And crucially, we only have one type of rod.

We only have one type of rod.

So at night, our visual system is like that single M cone in our thought experiment.

We are trapped by the principle of univariance.

Which is why, under moonlight, we can't see color.

Exactly.

It's not that the world has been drained of color.

The red rose outside is still reflecting long wavelengths.

Your blue car is still reflecting short wavelengths.

But your rods can't distinguish them.

A bright blue flower and a dim red flower might produce the exact same firing rate in the rods.

They could, yes.

We are effectively colorblind at night.

We see shades of gray because our visual system can only register how much light, not what kind of light.

OK, that makes so much sense.

So how does having three cones solve this devastating problem?

This is the trichromacy solution.

It solves it by comparison.

This is the core of the young Helmholtz trichromatic theory.

Imagine that same scenario again.

A light comes in.

But now, instead of one bucket, the brain has three different buckets to check S, M, and L.

OK, so a single light hits all three cones at once.

Yes.

And each one responds differently based on its sensitivity curve.

So if I see a blue light, say 450 nanometers.

The S cone bucket is overflowing.

It's shouting.

The M and L buckets, which aren't very sensitive to blue light, are nearly empty.

They're just whispering.

So the brain gets three signals.

Yeah.

A loud one from S and two quiet ones from M and L.

Right.

The brain looks at that ratio.

It says, OK, lots of S, not much M, and even less L.

That specific pattern, that triplet of values means blue.

And if I turn the dimmer switch up and make that blue light brighter.

All three cones will shout louder.

The volume of the signals goes up across the board.

The S cone shouts even louder.

And the M and L cones go from a whisper to a normal speaking voice.

But the relationship between them stays the same.

The ratio stays the same.

The S cone is still firing way more than the others.

The melody of the signal remains the same, even if the whole orchestra is playing louder.

So the color isn't defined by the output of one cone, but by the unique relationship between the outputs of all three.

Exactly.

Color is a code.

It's a triplet of values, a ratio,

S, M, L.

And with three variables, you can untangle the wavelength from the intensity.

This is a great system.

But it also leads to another mind -bending concept from the chapter, metamers.

Metamers.

This is where we really start to see that our eyes are not telling us the literal physics of the world.

They are just giving us a summary.

The text uses figure 5 .7 to explain this in its great illustration.

It compares a pure yellow light to a mixture of red and green light.

Right.

Let's walk through it.

Imagine on the left side of a screen, I project a spotlight of pure yellow light, a single wavelength of, say, 580 nanometers.

Okay, pure physical yellow.

That 580 nanometer light hits your retina.

It stimulates your L cones pretty strongly and your M tones also pretty strongly.

Let's just make up some numbers.

Let's say they both fire at 50 units of activity.

S cones aren't interested, so they're at zero.

Okay.

L is 50, M is 50, S is zero.

And your brain sees that code and says yellow.

Yellow.

Now, on the right side of the screen, imagine you don't have a yellow light at all.

Instead, you have two separate spotlights.

One pure red, maybe 650 nanometers, and one pure green, maybe 530 nanometers.

You shine both of them on the same spot.

So you're mixing lights.

You're mixing lights.

The red light primarily hits the L cones.

The green light primarily hits the M cones.

If you carefully balance the intensity of those two spotlights, you can get the L cones to fire at exactly 50 units and the M cones to fire at exactly 50 units.

So to the brain,

the incoming signal is identical.

L forward 50, M point 50, S by 2.

Identical.

The brain receives the exact same three numbers from that patch of the retina.

So you perceive the exact same color.

You see yellow on the right side, too.

Even though there is no yellow light there at all.

Not a single photon of yellow light.

Those two different physical stimuli, the pure yellow light and the red -green mixture, are called metamers.

They are physically different, but perceptually identical.

This is wild because it proves that mixing light is a mental event, not a physical one.

When I mix red and green light on a wall,

the photons don't physically smash together and become yellow photons.

No, absolutely not.

The physics doesn't change one bit.

A 530 nanometer photon plus a 650 nanometer photon does not magically become a 580 nanometer photon.

They stay what they are.

But the cone output is the same as if it were a 580 nanometer light.

And that's all the brain cares about.

That is all the brain knows.

It doesn't have a spectrometer.

It just has three cones.

Speaking of mixing, we need to make a very important distinction that the text highlights.

The difference between mixing lights and mixing paints.

Because every kindergartner knows that when you mix red and green paint, you get...

Brown sludge.

Yes, the brown sludge effect.

This is the crucial difference between additive and subtractive color mixing.

Lights are additive.

Right.

When you mix lights, you are adding wavelengths to the eye.

You start with black, which is the absence of light.

You add red light.

You add green light.

And the result is yellow light, which is brighter.

Add blue light to that and you get white.

You're adding energy together.

This is how your computer screen or your TV works.

The RGB system.

But paints are subtractive.

Completely the opposite.

Paints, inks, pigments, they all work by absorbing light.

They subtract wavelengths from the white light that hits them.

So blue paint looks blue because it absorbs everything else.

It absorbs the long and medium wavelengths, the reds and greens, and reflects the short blue wavelengths.

Yellow paint absorbs the short blue wavelengths and reflects the medium and long ones, which we see as yellow.

So when I physically mix blue and yellow paint on a palette...

Now you have a mixture of two different kinds of filters.

The blue pigment is trying to kill all the red and yellow light.

The yellow pigment is trying to kill all the blue and violet light.

So they're fighting each other.

They are.

And the only thing that's left, the only wavelength that neither of them is very good at absorbing, is the medium wavelength.

Green.

That's the only survivor.

So with paint, you're taking light away until you're left with the muddy remainder.

Exactly.

And the text makes this wonderful connection to our history, specifically to pointillism and artists like Georges Seurat or Paul Signac.

Oh, the paintings made of little dots.

The very same.

They were amateur vision scientists.

They realized they could hack the visual system.

Instead of mixing blue and yellow paint on the palette to make a relatively dull, subtractive green...

They would put a tiny dot of pure blue paint right next to a tiny dot of pure yellow paint.

Exactly.

And if you stand back far enough, your eyes can't resolve the individual dots.

So the light from both dots enters your eye at the same time and lands on the same small patch of your retina.

And that is additive mixing.

Your eye is combining the light reflected from the dots, not the pigments themselves, and the result is a vibrant luminous green that feels much brighter and more alive than the mixed paint ever could.

That is so cool.

They were doing vision science with a paintbrush.

They really, really were.

OK, so we've covered a lot.

We have caught the light with our cones detection.

And we've used the ratio of the three cone types to distinguish wavelengths discrimination.

Right.

But the signal doesn't just go straight from the retina to the brain as a raw SML code.

The text talks about a crucial intermediate step.

It calls it repackaging.

Yes.

This is about efficiency.

The brain isn't just interested in the raw number.

It's especially interested in differences.

It's a contrast detector.

So before the signal even leaves the eye in the retinal ganglion cells, and then later in a brain structure called the lateral geniculate nucleus, or LGN.

Some math happens.

Some math happens.

The system computes new signals based on the cone outputs.

It creates what are called cone opponent cells.

OK, what does that mean?

It means instead of just sending an L signal and an M signal up to the brain, the system computes the difference between them, L minus M.

Why would it do that?

Why not just send both?

Because, as we saw in that graph, the L and M cone signals overlap a huge amount.

Their responses are highly correlated.

If you just sent both raw signals, you'd be sending a lot of redundant information, which is metabolically expensive.

So by sending the difference, you're just sending the part that's actually new and useful.

Precisely.

You highlight the useful contrast.

And the text mentions that this L minus M channel is basically our red versus green channel.

Roughly, yes.

It's most sensitive to the differences between reddish and greenish colors.

And there are other channels too, right?

Yes.

There's another color channel that compares the S cone signal to the sum of the L and M cones.

So L plus M minus S.

And that gives us a yellow versus blue signal, because L plus M together signals yellow, as we saw with metamoles.

Exactly.

And then finally, you have a channel that just adds L and M together.

L plus M.

That one doesn't care about color, it just tells us about the overall brightness or what we call luminance.

So the brain is taking the raw RGB -like signal from the cones and converting it into a new format.

Luminance.

Red -green.

And blue -yellow.

That's the repackaging.

It's a more efficient and useful way to encode the information.

Before we move to the next section, I have to ask about this fascinating sidebar in the chapter.

The white cone mystery.

The text shows figure 5 .12, where researchers used adaptive optics to shoot a tiny laser into a single cone in a person's eye.

This is just mind -blowing work by researchers like Rorda and Williams.

Using a system of lasers and mirrors to correct for the natural imperfections and blur of the eye's lens, they could stimulate a single cone photoreceptor in a living, breathing human eye.

That's incredible precision.

It is.

And you'd expect that if they hit an L -cone, the person would say, I see red.

If they hit an M -cone, I see green.

But that's not what always happened.

It happened a lot of the time.

But a surprising number of times, especially for the L and M -cones, when they stimulated a single cone, the person reported seeing white.

White from a single cone?

That seems to violate everything we just said about ratios and comparisons.

How can one cone signal white?

Well, it suggests that the wiring isn't as clean as our diagrams might imply.

It's possible that same single cone is feeding its signal into multiple downstream circuits.

It might be feeding into a color opponent circuit and into that L plus M brightness circuit.

So if the brightness signal is stronger or gets processed first, the brain just says light rather than red.

That's one hypothesis.

It shows we still have a lot to learn about the microcircuitry.

The brain is a messy place.

OK, that feels like a good transition.

Let's move to section 3.

Color appearance.

We have these three numbers.

We have this opponent repackaging.

What is the final result?

How big is our perceptual world of color?

How many colors can we actually see?

It's a tricky question to answer precisely.

But if we're talking about the ability to just distinguish one colored surface from another under ideal conditions, the estimates are around two million distinct colors.

Two million?

That's a staggering number.

It is.

If we ignore the dimension of lightness, so we're just looking at the hue and its richness, it's probably closer to about 26 ,000.

Still a lot more than you get in the big box of crayons.

So to try and organize all these perceptions, we use conceptual maps called color spaces.

The text mentions two main ones, RGB, which it describes as a cube.

Right.

You can imagine a cube where one axis is how much red you have, another is green, and the third is blue.

Any color you can make on your computer is a point inside that cube.

And the other is HSB hue, saturation, brightness, which is more like a cylinder or a cone.

And HSB is often more intuitive for how we actually think about color.

Hue is the color name itself, red, green, blue, as you go around the circle.

Saturation is how pure or rich the color is, from a dull gray in the center to a vibrant color at the edge.

And brightness is the third axis, going from black at the bottom to white at the top.

Speaking of that color circle,

this has always bothered me.

The spectrum of light is a line, right?

It goes from 400 nanometers on one end to 700 on the other.

It's linear.

You're right.

But the color wheel is, well, a wheel.

It connects back on itself.

How do we get from a line to a circle?

That is the mystery of the color purple.

Purple doesn't exist on the spectrum.

Not as a single wavelength of light, no.

Think about a rainbow, which is just the spectrum of sunlight spread out.

You see violet at one end, the shortest visible wavelength.

You see red at the other end, the longest, but you will never see a band of purple or magenta in a rainbow.

I guess I haven't.

I never thought about that.

Purple is what we call a non -spectral hue.

It doesn't correspond to any single wavelength.

It is a bridge that the brain builds when both the S -cones, which love short wavelength blue light, and the L -cones, which love long wavelength red light, are stimulated at the same time, but the M -cones in the middle are not.

So purple is literally a perceptual shortcut, a figment of our imagination that connects the two ends of the physical spectrum.

In a sense, yes, it's our brain's way of representing that unique combination of very short and very long stimulation.

It wraps the line of physics into the circle of perception.

That's incredible.

And that feeling of colors having relationships, like red and blue making purple, brings us to the next big idea,

opponent color theory.

Right.

So far we've been talking about trichromacy, the three cone types, but that's a theory about the retina.

It doesn't really describe our psychological experience of color.

The text mentions that this idea came from Ewald Herring in the 19th century.

He was looking at color not as a physicist, but as a psychologist.

Exactly.

And he noticed something really weird about the structure of our color experience.

He talked about what we now call illegal colors.

Illegal colors.

Yes.

Think about it.

You can easily imagine a reddish yellow.

Sure, that's orange.

You can imagine a bluish green.

Turquoise or cyan.

Yep.

Now try to imagine a reddish green.

A reddish green, like at the same time, in the same spot.

Not mixed like paint to make brown, but a color that is simultaneously red and green in the same way that orange is simultaneously red and yellow.

I can't.

My brain won't do it.

It's like trying to imagine a square circle.

It's impossible.

Or try to imagine a bluish yellow.

You can't.

Herring realized that our color experiences seem to be arranged in opponent pairs.

Red fights green.

Blue fights yellow.

You can't have both at the same time in the same place.

So for a long time, this was seen as a rival theory to trichromacy.

For decades, yes.

It was a huge debate in vision science.

You had the young Helmholtz camp saying, it's all about three cones.

And you had the Herring camp saying, no, it's all about these four unique hues in opponent pairs.

How did they finally prove that Herring was onto something real in the nervous system?

They used a very clever psychophysical method called hue cancellation, which was famously done by Hurwitz and Jameson in the mid -20th century.

How does that work?

It's a neat experiment.

Imagine I show you a test light that looks slightly bluish green.

I ask you, how much blue is in that light?

How would I even answer that?

A little bit.

Seven units of blue.

Exactly.

It's hard to quantify.

So instead, I give you a dial.

The dial controls a pure yellow spotlight that we will add to the bluish green light.

And I tell you, your job is to add just enough yellow light to cancel out the blue.

Why hello?

Because if Herring is right, and yellow and blue are opponents, then yellow should act like a perceptual antidote to blue.

It should kill the blueness.

And does it?

It works perfectly.

You turn the dial, adding yellow, and you can find a point where the blue sensation completely disappears, leaving you with a pure neutral green.

The amount of yellow you needed to add is a direct measure of how much blue was in the original light.

So they mapped out these pairs, blue versus yellow, red versus green, by seeing how much of one color it took to cancel out its opponent.

Precisely.

And in doing so, they could identify what they call the unique hues.

Unique green, for example, is the specific wavelength of green that contains no blue and no yellow.

It's the point where the blue -yellow opponent channel is perfectly balanced at zero.

So who won the big debate?

Was it trichromacy, or was it opponents -y?

In the end, they both won.

That's the beautiful synthesis that modern vision science has arrived at.

It's a two -stage process.

Okay, lay it out for us.

Stage one happens at the level of the retina.

The three cone types detect the light, just like Young and Helmholtz said.

That's trichromacy.

It's the input stage.

And stage two.

Stage two happens in the ganglion cells and the LGN.

Those cells perform the math we talked about.

They subtract the cone signals to create the opponent channels, just like Herring proposed.

L minus M and L plus M minus S.

That's opponents -y.

So the brain takes the raw data from the cones and immediately reformats it into these more intuitive opponent channels for us to actually experience.

Right.

And as we go even deeper into the brain, specifically into the visual cortex, we find specialized areas that are obsessed with processing this color information.

The text mentions blobs.

Yes.

Which is a terrible name, but it's what they're called.

In the primary visual cortex, or V1, there are these little clusters of neurons that are rich in an enzyme called cytochrome oxidase.

They look like little polka dots or blobs.

And they process color.

They seem to be color -processing specialists.

And later in the visual pathway and other areas, you find even larger clusters, sometimes called globs.

Scientists really need to hire a marketing team.

Blobs and globs.

Yeah, it's very descriptive.

But the point is, there's dedicated neural real estate for color.

And we know this is crucial, because if you have brain damage to these areas, specifically an area called V4, you can get a condition called cortical achromatopsia.

That's when you lose your color vision, but not because of a problem with your eyes.

Exactly.

Your cones are working perfectly.

They are sending the S, M, and L signals.

But the part of your brain responsible for interpreting that data and constructing the experience of color is gone.

You see the world in shades of gray.

It's another piece of evidence that color really is a construction of the cortex.

It's not out there in the world.

It's a created experience.

Okay.

Let's shift gears to section four, which is about individual differences.

We've established how the machinery generally works.

But does it work the same for everyone?

This is the age -old question, isn't it?

Is my experience of red the same as your experience of red?

The text approaches this first through the lens of language.

There was a very famous study by two researchers, Berlin and Kay, about basic color terms.

Yes.

They looked at languages from all across the world.

Some languages, like English, have many words for different colors.

Some have very few.

But they found a surprisingly universal hierarchy in how languages add color terms.

So if a language only has two color words, what are they?

There are always, without fail, words for light and dark, or white and black.

Always the first distinction.

And if a language has three color terms...

The third one is always red.

If it has four.

It's usually yellow or green, then comes blue.

English has what we call 11 basic color terms.

So the big question is, if you grow up in a culture that only has two color words, do you actually see fewer colors?

Does language shape perception?

This is the classic debate of cultural relativism.

The text cites a landmark study with the Doni tribe in New Guinea.

They only have two basic color terms, mola, which means light or warm colors, and mili, for dark or cool colors.

So they don't have a specific word for blue or green.

No.

But when researchers tested their perception, they found that the Doni could still distinguish between blue and green ships just as well as English speakers.

They could see the categorical boundaries even if they didn't have words for them.

So that's a point for the universalist view.

Perception is the same for everyone, regardless of language.

Well, it's not that simple.

A later study with another tribe, the Burinmo, complicated things.

The Burinmo have a color boundary that doesn't exist in English.

They have a term null and another war.

And the dividing line between them falls right in the middle of what we call green.

So half of our greens are null to them, and the other half are war.

Exactly.

And when researchers tested them on memory tasks, like showing them a color chip, taking it away, and then asking them to pick it out from a pair of similar chips, the Burinmo were better at remembering which chip they saw if the two choices crossed their null war boundary.

But they were worse than English speakers if the choices crossed our blue -green boundary.

So language might not change the raw data coming in from your eyes, but it helps you categorize and remember colors more efficiently.

That seems to be the consensus.

Language gives us mental buckets to store the information in.

If you have a word for it, it's easier to label and recall later.

Now what about when the biological hardware itself is different?

Let's talk about color blindness.

The text points out we should really call it color anomalous vision.

That is the more accurate term, yes.

It's very rarely true blindness to all color.

It's a genetic condition, and in most cases, it's sex -linked because the genes for the M and L cone pigments are on the X chromosome.

Which is why about 8 % of males have some form of it, but only about 0 .5 % of females.

Men only have one X chromosome, so if that one has a faulty gene, they're stuck with it.

Women have two, so a normal gene on one can compensate for a faulty one on the other.

Okay, let's run through the common types.

What happens if you are born without any M cones?

You're a deuteranope.

No medium wavelength or green sensor.

And if you are missing the L cones?

You're a protenope.

No long wavelength or red sensor.

And in both of those cases, what happens to your perception?

The world flattens from three dimensions of color down to two.

Exactly.

They have trouble distinguishing reds from greens because they don't have that crucial L -M comparison mechanism.

Those colors fall along a single line of confusion for them.

Then there are tritenopes people missing the S cones, which the text says is very rare and not sex -linked.

And finally, the true monochromats.

Right.

A cone monochromat is someone who has only one type of cone.

It could be S, M, or L, but there's only one.

So they are truly colorblind.

They see the world entirely in shades of gray.

Yes.

Because they're stuck back in that world of the principle of univariance.

With only one cone type, they can't tell wavelength apart from intensity.

A bright red ball and a dim green ball might look identical to them.

But on the complete flip side of deficiency, the chapter mentions something extraordinary.

The possibility of tetrachromacy.

The possibility that some people, specifically some women, might be super -seers.

How would that work?

Well, because women have two X chromosomes, it's possible for them to inherit two slightly different versions of the L cone or M cone gene.

One might be the normal version, and the other might be a variant that's sensitive to a slightly different peak wavelength.

So they would have four types of cones instead of three.

S, M, L1, and L2.

Theoretically, yes.

And if their brain can learn to use that fourth channel of information, they might be able to perceive millions of colors that are completely invisible to the rest of us trichromats.

They might look at a beige wall and see a subtle kaleidoscope of different hues.

It's a tantalizing possibility, though it's been very difficult to prove definitively.

That is wild.

And speaking of wild perceptual experiences,

synesthesia.

Ah yes, the story of Van Gogh being kicked out of his piano lesson.

Right.

He was taking piano lessons, and he kept telling his teacher things like, this note is this particular shade of blue.

The teacher thought he was insane.

But he was probably a synesthete.

Synesthesia is when a stimulus in one's sensory modality automatically and involuntarily evokes an experience in another.

The most common type is grapheme color synesthesia, right, where letters or numbers have inherent colors.

Yes.

For a synesthete, the letter A isn't just associated with red.

It is red.

The experience is automatic and perceptual.

And it's not just them making it up.

It's incredibly consistent.

That's how they test for it.

If you test these people years apart, they are nearly 100 % consistent.

You ask them what color is the number seven, they say indigo.

Ten years later, you ask them again, indigo.

Seems to be a fascinating case of cross -wiring between different sensory areas of the brain.

Okay, section five.

So far, we've mostly been talking about colored lights in a dark, controlled lab setting.

But we don't live in a lab.

We live in the real world, and the real world is messy and complicated.

It is.

The text calls this the problem of the real world.

And the first issue it raises is that context changes everything.

Take the color brown.

Brown is a weird color.

There's no brown wavelength of light, is there?

Correct.

You cannot go to a physics lab and ask for a beam of brown light.

Brown is basically a dark orange or a dark yellow.

But it only looks brown when there is something brighter next to it for comparison.

It is what's called a related color.

So if I took a brown patch from a piece of cardboard, cut it out, and shown a spotlight on it in an otherwise pitch black room.

It would look orange, or maybe yellow.

It absolutely needs a brighter context to be perceived as brown.

The same is true for colors like olive green.

Context also creates these powerful effects of contrast and assimilation.

Yes.

Contrast is when nearby colors push each other apart perceptually.

A gray square will look slightly bluish when it's on a bright yellow background.

The yellow background induces the perception of its opponent color, blue, in the gray area.

So a green patch looks even greener on a red background.

Exactly.

That's opponent induction.

And assimilation is the opposite.

Assimilation is when colors bleed into each other.

If you have very fine red and yellow stripes on a shirt, from a distance, they won't contrast, they'll assimilate, and you'll just see an orange shirt.

And then there's the effect of adaptation, which leads to afterimages.

The book has that classic red dot experiment.

This is a great one that listeners can try right now.

If you have an image with a bright red dot, and you stare at it without moving your eyes for about 30 seconds.

What's happening in your eye?

You are selectively fatiguing your photoreceptors.

Your L -cones are being bombarded with long wavelength light, so they start to adapt.

Their response gets weaker.

You are tiring out the red part of your red -green opponent channel.

Okay, so I've stared for 30 seconds.

Then, if I look away at a blank white wall.

The white wall reflects all wavelengths, equally red, green, and blue.

But your L -cones are tired.

They don't fire as strongly as they normally would.

Your M -cones, the green ones, are fresh and ready to go.

So the signal is unbalanced.

It's totally unbalanced.

The red -green channel tips way over to the green side.

And you see a ghostly floating green dot that's a negative after image.

It's more proof of that opponent process.

You fatigue one side, and the system rebounds to the other.

It's the visual system recalibrating itself.

But the text says the biggest, most important problem the brain has to solve in the real world is color constancy.

This is maybe the most magical and computationally difficult thing the visual system does.

And the book uses the zebra in the library example.

Right.

So you look at a zebra.

You take it into a library that's lit by yellowish tungsten light bulbs.

Its stripes look black and white.

Okay.

Then you take the same zebra outside into the bright sunlight, which is much bluer.

The stripes still look black and white.

Of course they do.

A zebra is black and white.

But physically, the light that is entering your eye from its white stripes is completely different in the two settings.

In the library, the white stripes are reflecting yellow light into your eye.

Outside, they're reflecting blue light into your eye.

Yet, we don't perceive the zebra as changing color from yellowish to bluish.

It stays white.

How on earth does the brain do that?

It feels like magic.

It feels like magic, but the text explains it's actually a very complex calculation.

The brain seems to understand the physics, at least implicitly.

It knows that the light hitting the eye, L, is a product of the illuminant eye, the light source times the surface reflectance, S.

L equals I times S.

The brain gets L.

That's the data from the retina.

But it doesn't care about L.

It wants to know S.

It wants to know the true, inherent color of the surface.

So to solve for S, it has to figure out what I is and then divide it out of the equation.

Exactly.

It has to discount the illuminant.

It has to figure out the color of the lighting in the room and then subtract that color cast from everything it sees.

So my brain is constantly, unconsciously asking, what's the lighting in here?

And then color correcting the whole world for me.

That's what it's doing.

It uses all sorts of cues.

It might assume that the brightest thing in the scene is white.

It might take the average color of the whole scene and assume that's gray.

It looks for specular highlights, the little shiny reflections that show the pure color of the light source.

It makes a sophisticated best guess.

But sometimes that guess goes spectacularly wrong.

And that brings us to the dress.

Oh, the dress, the meme that broke the Internet in 2015.

The Great Civil War.

Was it blue and black or was it white and gold?

I love this example because it's not just a funny meme.

It's a perfect, real -world demonstration of color constancy failing because the brain's assumptions are violated.

Explain it.

Why did people see it so differently?

There was no middle ground.

You were on one team or the other.

The problem was that the photo itself was incredibly ambiguous.

It was overexposed.

The lighting was terrible.

And there were no good cues to tell your brain what the illuminant was.

So your brain had to make a gamble.

It had to make an assumption.

It had to guess the I in the L -I -X -S equation.

Exactly.

So let's take group A, the white and gold people.

Their brains made an unconscious assumption.

They assumed this dress is being viewed in a shadow.

And shadows on a sunny day are usually lit by the blue sky, so they're bluish.

Right.

So their brain says, OK, the illuminant here has a strong blue cast.

To see the true color of the dress, I need to subtract that blue out.

And what happens when you computationally subtract blue from the pixels in that image?

You are left with white and gold.

Mind blowing.

OK, what about group B?

Group B, the blue and black people, which were the correct colors, by the way.

Their brains made a different assumption.

They assumed this dress is under a bright, warm, artificial light, maybe in a department store.

So they assumed a yellowish illuminant.

And if your brain assumes the light is yellow, it subtracts yellow from the image to find the true color.

And when you do that, you are left with blue and black.

It's terrifying because neither side chose to see it that way.

Their brain performed the calculation before the image even reached their conscious awareness.

It's a profound revelation.

We don't see reality.

We see our brain's best hypothesis about reality based on its built -in assumptions about how lighting works in the world.

Finally, that brings us to section six.

Why do we even have this complex system?

What is color vision good for?

As we said, we could survive without it.

We could.

Plenty of mammals do.

Most dogs, for example, are dichromats.

But for primates like us, having trichromatic color vision provides some distinct evolutionary advantages.

The text highlights two really big ones.

Eating and sex.

The two pillars of survival and reproduction.

Pretty much.

For eating, it's huge.

Primates are largely frugivores.

We eat fruit.

Spotting a ripe red berry against a backdrop of green jungle foliage is nearly impossible for a monochromat or a red -green colorblind dichromat.

It's perfect camouflage.

But for a trichromat with that L -M channel, the red berry just pops out.

It pops.

And that channel is also perfectly tuned to judge ripeness.

The shift from an unripe green fruit to a ripe red or yellow one is a massive signal on that specific color axis.

It also seems to affect our perception of flavor.

The text mentions a wild white wine experiment.

This is hilarious and a little disturbing.

Researchers took a perfectly good white wine and dyed it pink with an odorless, tasteless food coloring.

Then they gave it to a panel of wine experts to describe.

These are people who do this for a living.

They have trained palettes.

The best of the best.

And what happened?

The experts started using flavor descriptors typically associated with red wines.

Words like strawberry, raspberry, cherry.

No way.

Yes.

Their visual input completely overrode their taste and smell input.

They tasted what they saw.

That is a scary testament to how much vision dominates our other senses.

Okay, so eating is a big one.

What about sex?

Well, look at the animal kingdom.

Peacocks, mandrills, tropical fish.

Nature is bursting with color used for sexual signaling.

A bright peacock tail is what we call an honest signal of genetic health.

It says, I am so healthy and have such good genes that I can afford to waste all this energy growing this giant conspicuous colorful target for predators.

And for primates and humans specifically.

The book suggests it goes back to our skin.

We are relatively bare skinned primates.

That L -M channel is exquisitely sensitive to changes in blood oxygenation levels in the skin.

You mean like blushing?

Blushing, the flush of arousal, the paleness of fear, the glow of health.

These are all crucial social and sexual signals that our color vision is perfectly tuned to detect.

We have to wrap up, but we cannot leave without mentioning the super seers of the animal kingdom that the chapter ends with.

Oh, we have to talk about the mantis shrimp.

The champion of color vision.

They have 12 types of photoreceptors.

We have three.

They have 12.

12 distinct channels for color?

You would think they must see a psychedelic explosion of color that we can't even begin to comprehend.

A whole new rainbow.

But this is a great twist.

The text actually throws a bit of a wet blanket on that idea.

It turns out when you test them in the lab, despite having 12 cones, they aren't actually very good at telling similar colors apart.

Much worse than us.

Wait, really?

Then why on earth would you have 12 cones?

Researchers think they might use a completely different strategy for processing color.

Instead of the complex compare and contrast math our brain does to get a precise reading, they might use their cones more like a barcode scanner.

Cone 1 fired.

That's UV.

Cone 12 fired.

That's deep red.

It's very fast, but it's not very nuanced.

So they chose speed and simplicity over perceptual resolution.

That seems to be the idea.

But then, to blow that out of the water, there is the silver spiny fin.

The deep sea fish.

Right.

It lives in the deep dark ocean where almost no sunlight penetrates.

You'd think it would be colorblind.

But it has 38 different types of rod photopigments.

38.

We have one type of rod.

It seems they have evolved this incredible array to detect the very specific, faint wavelengths of bioluminescence.

The light given off by other glowing creatures in the pitch black.

They're seeing a complex light show in the abyss that we can't even begin to imagine.

38 channels for seeing in the dark.

That is poetic.

It really is.

So we've gone on quite a journey.

From the physics of a single wavelength of light, to the biology of the cones, the mathematical problem of univariance, the psychological experience of oponency, and the philosophical puzzle of the dress.

It's a long and winding road from photon to perception.

I want to leave our listeners with the final philosophical provocation that the text raises.

The thought experiment known as inverted coelia.

This is the one that keeps philosophy students awake at night.

Right.

Since we know now that color is a mental construct created by the specific wiring of your specific We can never truly prove that my experience of red looked anything like your experience of red.

We can both agree to call the fire truck red.

We can both pass a color vision test.

Our behavior is identical, but inside my head when I look at the truck, I might be having the exact same internal experience that you have when you look at a patch of grass.

My red could be your green, and we would have no way of ever knowing it.

We can measure all the machinery, the wavelengths, the cone responses, the neurons firing, but we can never truly share the subjective experience itself.

On that slightly existential note, thanks for listening to this deep dive into sensation and perception.

This has been the Last Minute Lecture Team.

See you next time.