Chapter 2: Perception: How We Experience the World

Welcome to Last Minute Lecture.

This free chapter overview is designed to help students review and understand key concepts.

These summaries supplement not replaced the original textbook and may not be redistributed or resold.

For complete coverage, always consult the official text.

Welcome back to the Deep Dive.

Today, we are taking on what might be the most fundamental process of the human mind.

Absolutely.

It's the foundation for everything else.

It's perception.

I mean, it's how our brains even begin to handle the, you know, the constant flood of raw data coming from the outside world.

And this is such a critical topic because if you don't get how perception works, nothing else.

Memory, judgment, even social interaction really make sense.

Our source for this chapter two on perception from a textbook of human psychology, treats it not as a passive thing, but as an aggressive act of engineering.

Right.

It's not just about opening your eyes and letting reality pour in.

The goal, as the book puts it, is to transform the physical world into something useful.

These mental images.

And here's the first big counterintuitive idea.

For that transformation to be useful, the mental image must be different from the physical world it's representing.

Which sounds completely backwards.

You'd think the goal is a perfect copy, right?

Like a high definition photograph in your head.

Yeah, but a perfect photograph would be totally worthless.

Because if you had a perfect picture of the world inside your brain, you'd still have the massive problem of how to interpret that picture.

You'd need a little man in your head to look at it and a little man in his head.

Right, right.

It's an infinite loop.

So the better analogy is a map.

Exactly.

A map is useful because it's imperfect.

It throws away most of the information.

It keeps only a few key things and then it adds stuff that isn't even there.

Like gridlines or conventional signs.

It simplifies.

It organizes.

That's what makes it a tool.

To really drive that home, the chapter kicks off with this wonderful little story from Borsch's.

Oh, it's a fantastic anecdote.

It's about this ancient empire where the art of cartography became so advanced, so perfect, that the cartographers eventually created a map that was the exact same size as the empire itself.

A one -to -one scale.

A perfect representation.

And what happened to it?

Later generations realized it was, well, useless.

It didn't simplify a thing.

It just duplicated the territory.

So they abandoned it to the elements.

A perfect metaphor.

So if the perfect map is useless, then the number one job of our nervous system is information reduction.

That's the core mission.

Our brains are built to be economical.

We have to ignore as much of the environment as we possibly can.

Why?

To save effort?

To save effort and crucially to avoid overloading our really limited memory capacity.

We just can't store everything.

So how does this reduction work?

Well, two main ways.

The first is obvious.

Just discarding information that's irrelevant for survival.

We can't see ultraviolet light.

We can't hear the frequencies bats use.

It's filtered out from the start.

Okay.

That makes sense.

What's the second way?

This is the more interesting one.

Exploiting redundancy.

Redundancy.

Yeah.

It just means that one aspect of the environment gives you the same information as another, so you can safely ignore one of them.

Like the beard example in the text.

Exactly.

If, in a certain environment, a beard is a highly reliable indicator of male,

the brain can just use that one simple cue and not bother processing all the other more subtle sexual characteristics.

It's a shortcut.

And when those shortcuts break down, like when cultural styles change and redundancy is lost, that's when we get confused.

Precisely.

And this is where perception links directly to higher order thinking.

By finding these relationships, these redundancies, and imposing organization, perception is doing the exact same thing as abstract thought or language or concept formation.

It's like what a scientist does with a huge spreadsheet of data.

Exactly.

You don't look at every single data point.

You reduce it.

You find the mean, the standard deviation.

You find the most economical description.

And that connection between having limited memory and needing to think abstractly is profound.

It is.

The chapter mentions these incredible cases, some studied by Loria, of people with almost perfect photographic memories.

You'd think that would be a superpower.

You would.

But the consequence was that their capacity for abstract thought was severely impaired.

Why?

Well, think about it.

If you can remember every single bird you have ever seen, every leaf on every tree,

you never have the need to form the concept of bird or tree.

Wow.

You don't need the rule if you can remember all the members.

Exactly.

The ability to forget, to ignore, to reduce.

That's what makes us intelligent.

Okay.

Let's dive into the mechanics, starting at the very bottom, the raw senses.

Right.

The interface.

Where physical energy light vibration pressure hits our receptors.

And even here, at the very first step, it's intensely selective.

Massively selective.

We're blind and deaf to most of what's going on around us.

That information is just deemed irrelevant and discarded before it even gets in the door.

And we can actually see a map of this selectivity in the brain itself, in the somatosensory cortex.

Yeah, the part that processes touch.

If you look at how much brain tissue is dedicated to different parts of the body, it's wildly distorted.

This is the sensory homunculus, right?

The little man.

That's the one.

In us humans, the face, the lips, the tongue, and especially the hands, the thumb and index finger, are just enormous.

They take up huge areas of the cortex.

But the rest of the body, like your back or your legs?

Barely represented at all.

Your lips get more neural real estate than your entire torso.

Because that's where the most critical information for survival comes from.

For eating, for manipulating things, for speaking.

And that map changes completely depending on the animal.

It's all about what's most important for that creature's lifestyle.

So this leads us into the field of psychophysics.

Right, which is the formal study of the relationship between a physical stimulus and the mental sensation it produces.

And the big question they ask is, how much do you have to increase a stimulus, say the brightness of a light,

to produce a noticeable change in sensation?

And the answer, almost universally, is what's known as the Weber -Fechner law.

Which says the relationship isn't linear, it's logarithmic.

Exactly.

Let's break that down.

If you're in a dark room with one candle and you light a second one, the change is huge.

It feels twice as bright.

But if you're on a brightly lit stage with a hundred spotlights and you add one more,

you won't even notice.

So the change you perceive depends on the starting point.

It's proportional to the ratio of the change.

And the nervous system does this for a very smart economic reason.

It concentrates its sensitivity where information is scarcest at the low energy end of the scale.

It's a trade -off.

It sacrifices the ability to tell the difference between a hundred and one lights to be exquisitely sensitive to the difference between one and two.

The text uses that great analogy of wages and salaries.

It's a perfect fit.

If you earn 20 pounds a week, a two pound raise is a big deal.

You notice it.

But if you're a company director earning thousands a week, two pounds is, it's nothing.

It's below the threshold of notice.

Right.

And that's why raises are done by percentages, which is just another way of using a logarithmic scale.

Your nervous system is basically a financial analyst.

Okay.

So to measure these things, psychologists use thresholds.

Yeah, because people are frankly terrible at describing sensations.

You can't ask someone, how bright is that light on a scale of one to ten?

But you can ask them, did it just get brighter?

Exactly.

We measure the points where they can detect a change.

That's the difference threshold.

And the point where they can detect a stimulus at all is the absolute threshold.

Like the faintest sound you can hear or the lightest touch you can feel.

Right.

And we know these thresholds are very sensitive to things like drugs or fatigue or brain damage.

But here's where it gets really complicated.

The methodological problem of response bias.

This is huge because when a stimulus is right at the threshold, the person is genuinely uncertain.

Did I see it?

Did I not?

And what they say they saw depends on their psychology, their willingness to say yes.

The book gives that great example of the radar operators.

The Cold War operator versus the airport bird detector.

The Cold War operator can't afford a false alarm.

That could start a war.

So their bias is incredibly conservative.

They'll only say yes if they are absolutely certain.

Whereas the airport operator is worried about birds hitting a plane.

A false alarm is no big deal, but missing a flock is a catastrophe.

So their bias is liberal.

They'll report anything that even looks suspicious.

And the critical point is they'll have different apparent thresholds.

Not because their eyes are different, but because their attitudes are.

Their willingness to report is different.

So psychologists needed a way to separate genuine sensitivity from this reporting bias.

And the classic study on this was about pain.

Clark's study in 1969.

They gave people a placebo, right?

A sugar pill they were told was a painkiller.

Yes.

And on the surface, it worked.

People reported being able to withstand more pain.

Their pain threshold went up.

What?

But when they did a more detailed analysis, they found that the subject's actual physical sensitivity to the painful stimulus hadn't changed at all.

So what changed?

Only their willingness to report pain.

They thought they were supposed to feel less pain, so they said they did.

Their bias shifted.

A real analgesic.

A real analgesic changed their underlying sensitivity.

A beautiful piece of research that shows how perception is always a mix of raw sensation and psychological decision.

Okay.

So let's move up a level.

Beyond just registering energy to the first real act of transformation, contour extraction.

Right.

It's still automatic, still happening peripherally.

But this is where the brain starts building its economic description of the world.

And it does this by taking advantage of a key statistical property of the environment.

Which is that it's full of local uniformities.

A wall is a big block of the same color.

A clear sky is a big block of blue.

The world is mostly uniform patches.

It's redundant.

Hugely redundant.

And the genius of the system is that it ignores the uniformity and pays attention only to where the uniformity breaks.

The edges.

The contours.

So a black square on a white background isn't processed as a bunch of black next to a bunch of white.

No.

It's instantly transformed into just the outline of the square.

All the information inside the uniform areas is discarded.

Economical.

And the mechanism for this is called lateral inhibition.

It's so simple and elegant.

Imagine all your visual receptors are in a grid.

When one receptor is stimulated by light, it sends out a signal that actively inhibits its neighbors.

So if the whole area is uniformly bright.

Then all the receptors are inhibiting each other equally.

They all cancel out and the signal is minimal.

The brain effectively hears silence.

But at the edge, at the contour.

At the contour, the receptors on the bright side are stimulated, but their neighbors on the dark side are not.

So they aren't being fully inhibited.

And that imbalance creates a huge spike in the signal right at the edge.

So the system is literally wired to scream when there's change and be quiet when things are constant.

That's the fundamental principle.

And it applies to everything in perception.

Which raises the question, what happens when the world violates those assumptions?

Then the system breaks down or it behaves weirdly.

That's what optical illusions are.

They're just situations that exploit the brains, built in economic shortcuts and make them fail.

We can even quantify this expectation for uniformity.

The text uses those letter sequences.

Right.

Sequence one is excusu.

Sequence two is sock sock sock sock socks.

Both have the same number of Xs and Os.

But the first one is made of big uniform blocks, which is one change in the middle.

The second is all change.

And our system is built to expect the first type.

We find it easier to remember, easier to find a rule for.

We're tuned to look for clusters.

It's a fundamental bias.

And because contour is so basic, it almost becomes a physical property of stimulus.

Like it's brightness.

Which is why some patterns can seem dazzling.

Exactly.

A radial starburst pattern, for example, has an insane density of contours.

It just overloads the lateral inhibition system.

And you get this sensation of intense visual energy.

And camouflage is the opposite.

It works by breaking up an object's natural contour lines so the system can't group it into a single figure.

Perfect.

OK, so contour extraction gives us an outline.

But that's still a lot of data.

The next step toward a more economic description is feature extraction.

Right.

We go from a line drawing to an abstract summary.

A square isn't just four lines.

It's four straight lines, four equal lines, four right angles.

These are abstract features.

And the neurophysiology here is incredible.

Hubel's work on cats and monkeys.

Groundbreaking stuff.

He found single cells in the visual cortex that would fire only for a line of a specific orientation.

Say 45 degrees.

And it didn't matter where in the visual field that line was?

Nope.

The cell was detecting the abstract property of 45 degree -ness.

The idea is you have a whole hierarchy of these detectors.

Some for lines, some for corners, maybe even some for whole letters or faces.

But the brain would only wire itself up like that for things that are really important and really common.

Exactly.

And the environment dictates which features get chosen.

This happens through evolution, but also in early infancy.

The kitten experiments are just mind -blowing.

They are.

Blake Moore and Cooper raised kittens in an environment with only vertical stripes.

And the result?

Their brains never developed the cells to detect horizontal ones.

They were functionally blind to horizontal lines.

And it has to be active experience, right?

That's the key finding from Hell's experiment.

The kitten that was just passively pulled along in a carriage, seeing the same things as the active kitten, it was visually inferior.

You have to interact with the world to learn its features.

And this isn't just for vision.

The source mentions a study on learning Morse code.

Yeah, Chepard's work.

By looking at what letters people confused, you could figure out what features they were paying attention to.

And it changed over time.

It did.

At first, they focused on simple things like the number of dots and dashes.

But with practice, they shifted to a more abstract feature, like the mix of dots and dashes.

It shows how we're always selecting a small, relevant set of features to do the job.

Okay, let's add a dimension.

How do we perceive depth?

Well, it's a mix of feature extraction and building a model based on assumptions about the world.

We have some direct mechanisms, like binocular disparity.

Right.

The small difference between what your left eye sees and your right eye sees.

The brain uses the size of that discrepancy to calculate how near an object is.

And this is a very low -level automatic process.

Very.

Jules showed you could perceive depth even in random dot patterns, where the only cue was this disparity.

But with one eye, or for things far away, we have to rely on monocular cues, which are basically learned expectations.

Exactly.

One of the big ones is texture gradient.

Okay, explain that.

Imagine you're looking at a gravel path stretching into the distance.

The texture of the gravel appears finer and more compressed the farther away it is.

Your brain reads that change in texture density as a direct cue for distance.

Like railway lines seeming to get closer together in the distance?

The perfect example.

And the visual system uses this automatically.

The source has that great illustration with the three posts.

Right, they are all the same physical size.

But because they're placed on a texture gradient that implies depth,

the one that's interpreted as being farthest away is perceived as being much larger.

That's size constancy compensation kicking in.

The brain says, I know that post is far away, so even though its image on my retina is small, it must be a big post in reality.

It's an automatic calculation.

And we use other cues too, like overlap near things, block far things, and movement parallax.

Where nearby objects seem to fly past faster than distant ones when you move your head.

And illusions happen when you violate these powerful expectations.

This all speaks to the brain's preference for the simplest interpretation, the wire cube.

A perfect example.

On the page, it's just a complex 2D pattern of lines.

But we almost never see it that way.

We see a 3D cube.

Because a simple regular 3D cube is a much more economic description of that pattern than a weird irregular 2D shape is.

And the fact that the cube image seems to flip back and forth.

That's the brain trying to decide between the two equally simple 3D interpretations.

It can't settle.

And you can break this effect.

If you draw the cube from an angle where the 2D pattern is also simple and regular, people are less likely to see it in 3D.

The brain just picks whichever description, 2D or 3D, is the most economical at that moment.

This drive for simplicity leads to the ultimate expectation illusion, the Aims Room.

Oh, the Aims Room is fantastic.

The room is physically a bizarre distorted trapezoid.

But our brains are so hardwired with the expectation that rooms are rectangular.

That it forces that interpretation onto the image.

It sees a normal rectangular room.

And the consequence of that is that people standing inside it look like they're impossible A giant in one corner, a dwarf in the other.

What's so fascinating is that the expectation of a rectangular room is so strong it overrides the expectation that people don't change thighs.

The brain is willing to accept a person magically shrinking to preserve its simple model of the room.

Which brings us to constancies.

The brain's automatic compensation for changes.

Right.

We just talked about size constancy.

We use depth cues to make an object's apparent size stay the same even as it moves away.

And we're so good at this, we're actually terrible at judging the real size of an image on a retina.

We see what we know, not what we see.

The Dürer illustration in the text shows this perfectly.

Artists struggled for centuries with perspective because our natural perception is not like the retinal image.

They had to invent mechanical aids or learn geometric rules to overcome their own brain's constancy mechanisms.

And there are other constancies too.

Color, shape.

All working on the same principle.

A piece of white paper looks white even in deep shadow.

And a piece of black paper looks black even in bright light, as long as we have context.

That's brightness constancy.

A round plate looks round even when you see it from the side and its retinal image is an ellipse.

That's shape constancy.

And these mechanisms are lightning fast, unconscious,

and according to some research may even be partially innate.

Bauer found evidence of size constancy in infants just over a month old.

So before we can apply these constancies, we first have to group the scene into objects.

Exactly.

How does the brain decide what belongs together?

This is where the Gestalt principles come in.

These are the rules of perceptual organization.

Rules like proximity, things that are close together get grouped together.

Closure, our tendency to complete incomplete figures.

And good continuation.

We see lines as following the smoothest path.

The illustrations in the book show this really well.

A few dots are just dots, but if you arrange them with proximity, they become a group.

Or if you have a circle with a chunk missing and a square with a chunk missing, and you place them so they overlap, you don't see two weird shapes.

You see a simple circle in front of a simple square.

The brain chooses the simplest, most complete interpretation.

It's all about finding the most economic description.

Always.

It all boils down to that.

The system is always trying to find the underlying invariant property by compensating for transformations.

It's the same model science uses.

Water is water, whether you see it as ice, liquid, or steam.

And when the system fails, say under the influence of drugs like LSD, the constancies can break down.

The subject becomes aware of the raw, unsimplified retinal image, and the world becomes chaotic.

It's a failure of the economic description process.

Okay, so we've built stable objects, but a complex scene still has too much information.

How do we handle that?

Now we move to higher order processes.

And the simplest way to cut down information is just don't look at it all.

This is the role of attention and eye movements.

Yarbus did this amazing research in the 60s tracking where people's eyes look.

And it's not random at all.

Far from it.

When people look at a painting, their eyes follow contours.

But more importantly, they fixate on details that are relevant to their current goal.

He showed this by giving people different questions about the same painting, The Unexpected Visitor.

Exactly.

If he asked them to guess the family's wealth, their eyes went to the furniture, the clothes.

If he asked them to remember the people's positions, their eyes moved in a totally different pattern.

Our eyes are actively searching for goal -relevant information.

It's a research project, not passive viewing.

And when this guiding process breaks down, you see the kind of random unstructured fixations found in some schizophrenic or brain -damaged patients.

It's also seen in developmental conditions.

The study on autistic children was striking.

Yeah.

They spent more time looking at the plain black background than at the interesting objects in the foreground.

It suggests a fundamental failure to attend to what's important.

Which might explain the avoidance of faces and eyes, which are the most information -rich parts of our social world.

A very strong possibility.

So beyond moving our eyes, we have internal selective attention, the cocktail party problem.

Right.

How do you follow one conversation in a noisy room?

It shows two things about attention.

We can direct it, and its capacity is limited.

And that limit is set by the information content.

Yes.

Mowbray's study showed people could handle two tasks at once if the information was easy.

But as soon as it got difficult, performance crashed.

There's a limited bandwidth.

The classic experiments for this used dichotic listening.

Putting different messages in each ear.

And the finding was that people could tell you almost nothing about the verbal content of the ear they weren't attending to.

But they could report physical characteristics.

Right.

Was it a man's voice or a woman's voice?

Was it speech or tone?

They could get that.

Which led to Broadbent's filter model.

The idea that we have an early filter that selects input based on simple, physical cues.

Which ear it's in, the pitch of the voice.

But crucially, that early filter can't select based on meaning.

But it's a leaky filter.

It is.

People usually hear their own name if it's spoken in the unattended ear.

Some high -priority signals get through.

Which suggests a later, second filter.

Yeah, a less efficient one that can select based on abstract features like word meaning.

That's how you can, for example, track all the color words you hear.

Even if they're mixed with numbers and switching between ears.

So the brain is always trying to find patterns to make things more economical.

But what if there are no patterns?

This is where we see two complementary processes.

Discovery of patterns and imposition of patterns.

Frith's study with the horse -spoon -horse sequences showed this.

Normal children discovered the simple repeating rule and remembered the sequence easily.

Their errors were rule -based.

But the autistic children?

They struggled to find the rule.

They remembered simple sequences no better than complex ones.

And their errors suggested they were imposing their own, idiosyncratic rules onto the material.

And we all do this imposition when things get random.

All the time.

When you're guessing coin flips after a long string of heads, you feel like tails is due.

You're imposing a pattern of alternation on a random process.

It seems like the issue for some clinical populations is that they find the world so random, they are constantly forced to impose these private idiosyncratic patterns.

Which makes it hard to operate in a shared reality.

And you see this in language, too.

Because language is full of rules and constraints.

Exactly.

Normal people use those rules to reconstruct meaning.

But the speech of some schizophrenic patients is much harder to reconstruct.

Suggesting they're using different, or no, constraints.

So it's not just a language problem.

No, it suggests a general difficulty in finding and using shared rules and patterns.

Which is the very foundation of economic perception.

So another economic strategy is to combine lots of simple features into one abstract concept.

Right.

The human face is a perfect example.

We don't catalog every feature.

We summarize it into concepts like attractiveness or age.

And we do that by learning which features correlate.

Gray hair goes with wrinkles, for example.

Age is just a highly economical label for a whole cluster of correlated features.

And studies using schematic faces showed that normal people are very consistent in how they combine features to make these judgments.

Like friendliness.

But yeah, and this process can break down.

Some schizophrenic patients were inconsistent in how they combined the features to form a judgment.

They couldn't create a stable economic concept.

Which supports this idea of schizophrenia as a kind of perceptual overload.

Exactly.

If your system for creating economic descriptions fails,

you're just bombarded with raw, meaningless data.

You lose your normal expectations.

It's interesting to think about the link to creativity.

It is.

Creativity is about seeing new patterns, new relationships that others miss because they're stuck in the standard economic description.

The difference between creativity and madness.

Might just be whether you can get enough other people to eventually see the pattern you see.

And what about hallucinations?

How do they fit this model?

The hypothesis is that they are the ultimate act of imposition.

It's the brain imposing patterns and meaning on random or minimal input.

So they happen in sensory deprivation.

Where there's no external input, so the brain creates its own.

Or under drugs that create novel, chaotic sensory input that the brain tries to force into a meaningful pattern.

Like imposing the meaning of voices onto random noise in the ear.

A perfect example of the brain's desperate need to find an economic description, even when there's nothing there.

Okay, let's bring this all together and apply it to the biggest perceptual challenge we face.

Other people.

Right.

If we apply this principle of economic simplification to people, we get prejudice.

Because treating every single person as a unique individual would be an impossible memory overload.

We have to use shortcuts.

We have to assume certain cues.

Race,

sex, class, age are important, and ignore the rest.

And for communication to work, we all have to share roughly the same set of expectations about which cues matter.

There was that incredible study by Alport and Kramer.

Astonishing.

They found that people with strong anti -Semitic prejudice were better at identifying Jewish faces in photographs than unprejudiced people were.

How is that possible?

It's not about eyesight.

It's about attention.

Because they consider that classification to be so vitally important, their perceptual system was trained to pay hyperattention to any cue that fit their model, and to ignore any information that contradicted it.

It's selective attention driven by attitude.

And to maintain these rigid economic categories, you have to deny the existence of anything on the boundaries.

Like the apartheid system needing to classify some groups arbitrarily as white or colored, to maintain its simple binary model.

Exactly.

But here's the kicker.

This denial of boundary objects isn't just a feature of prejudice.

It's a feature of normal perception.

Playing card experiment.

Bruner and Postman.

They showed people a black heart or a red spade.

The brain's economic model says hearts are red, spades are black.

And it will fight to defend that simple rule.

So most people just didn't see the anomaly.

They reported a normal card.

Their brain denied the evidence to preserve the simple economic category.

It's a fundamental process we all use all the time for cognitive convenience.

So as we wrap up this deep dive, the central theme is just, it's so clear.

It really is.

Perception is not a mirror.

It's a process of ruthless information reduction.

It's about building simple, efficient models of the world by discovering, or if it has to, imposing patterns.

Which leads to a pretty profound conclusion.

The question of whether the patterns we see really exist in the world, it might be meaningless.

What matters is that our model of the world, our patterns, our expectations,

is similar enough to the models of the people around us.

That's what allows for communication and social cohesion.

Our perception of reality is profoundly shaped by our shared culture.

Efficiency and stability are bought at the price of ignoring complexity.

That's the trade -off.

It's built right into our operating system.

So a final thought for you to take away from all this.

If you are biologically hardwired to seek the most economical, simple description of the world,

what everyday cultural assumption are you currently accepting?

Something that feels as obvious as a room being rectangular or a heart being red.

What necessary simplification are you relying on that might completely break down if you started paying close attention to the actual overwhelming complexity it's masking?

Great question to ponder.

Thank you for joining us for this deep dive into the fascinating, filtered world of perception.