Chapter 4: Object Perception & Recognition

Welcome to Last Minute Lecture.

This free chapter overview is designed to help students review and understand key concepts.

These summaries supplement not replaced the original textbook and may not be redistributed or resold.

For complete coverage, always consult the official text.

Welcome back to the Deep Dive.

It is wonderful to have you with us again.

Great to be here.

Today, we are going to dismantle something so fundamental to your daily existence that you probably haven't thought about it since.

Well, ever.

We are going to figure out how you see.

And not just the simple parts.

No, I don't mean the optics.

I don't mean how light hits your eyeball or how the lens focuses.

That is the easy part.

That is just the plumbing.

Exactly.

We are talking about the hard part, the computational miracle.

We are talking about how your brain takes a messy, chaotic soup of light and shadow and instantly, without you even trying, says, that's a cat.

Or that's a table.

Or that's a picture of Jennifer Aniston.

It's deceptive, isn't it?

Because it feels so effortless.

You open your eyes and the world is just there.

It feels like a passive act, like a camera just taking a picture.

But if you actually look at the machinery required to make that happen, to go from raw visual input to recognizing a specific object, it is arguably the most complex computational feat your brain performs.

It really is.

It engages more of your cortex than almost any other single function.

And what we are doing today is looking at that specific journey.

We are basing this deep dive entirely and strictly on chapter four of Sensation and Perception, sixth edition.

Our mission, really, is to decode what neuroscientists call the what pathway.

We are going to trace the signal from the moment it leaves the early visual system until it becomes a conscious thought, a label, an identity.

OK, so to set the stage, we have to recap where we left off in our previous deep dives.

We were in the primary visual cortex, or V1.

This is located at the very back of your head in the occipital lobe.

Right.

And you can think of V1 as, say, the mail room of the visual system.

It's incredibly busy.

It's processing millions of signals.

But the workers there have a very, very limited job description.

They're not exactly big picture thinkers.

Not at all.

They are more like security guards looking through tiny keyholes, right?

That is the perfect analogy.

We call those keyholes receptive fields.

A single cell in V1 can only see a tiny specific patch of your visual field.

Just a little window.

A tiny window.

And within that patch, it is only looking for one thing.

A line,

a vertical line, a horizontal bar, maybe an edge moving to the left.

But that is it.

So if I am sitting here looking at a house, which is a complex object with windows, a a roof, a chimney,

a cell in my V1 doesn't see house.

No, not even close.

It sees something like vertical line segment number 4000 to wall at a 90 degree angle.

And its neighbor sees.

Its neighbor sees horizontal line segment number 200.

It's a completely fragmented, atomized view of the world.

It's like looking at a giant mosaic through a straw.

You only see one tiny tile at a time.

Which leads us right to the central problem of this chapter.

How do we get from a bunch of disconnected edges to seeing a cat or a table or yes, Jennifer Aniston, how does the brain glue it all back together?

The text illustrates this with a classic parable, which I think is just the perfect analogy for what your neurons are doing.

It's the story of the blind men and the elephant.

I love this story.

So for those who haven't heard it, you have to imagine a group of blind men encountering an elephant for the very first time.

They have no concept of the whole animal.

None at all.

So one of them grabs the trunk and he feels the, you know, the scaly skill and the movement.

And he says, oh, surely this is a snake.

And another one of them grabs the leg.

It's thick and rough and sturdy.

He says, no, you're completely wrong.

This is obviously a tree.

And then a third one, he touches the broad side of the elephant and says, it's a wall.

And the fascinating thing is they are all right locally.

From their limited perspective, they're correct, but they are all wrong globally.

And in a very literal sense, our V1 cells are those blind men.

Exactly.

One cell sees a vertical edge and thinks tree.

Another sees a curve and thinks snake.

They are all seeing valid local features.

The massive challenge for the brain and the entire topic of today's discussion is how do you synthesize those thousands of local opinions into a single global consensus that says elephant.

And the text makes a really interesting point about novelty right at the start.

It's shows figure 4 .1.

It's a collection of elephants.

Yes.

And they're all different.

Right.

There's a photo of a real elephant.

There's a simple line drawing.

There's a cartoon.

And then there is a picture of an elephant made entirely out of Legos.

Which is something you've probably never seen before in your life.

Never.

And yet you recognize all of them as elephant instantly.

You don't even hesitate.

Which suggests we aren't just matching images to like a hard drive of photos we've seen before.

If recognition was just template matching, like holding a cut out over a picture,

the Lego elephant would fail completely.

It doesn't match the photo.

Not at all.

We understand the structure of objects.

We extract the elephantness, the essential form, from the visual data, regardless of the texture or the medium.

But to get to that level of abstraction, we have to leave the mail room.

We have to leave V1 and move up the ladder.

Okay.

So let's march up that visual pathway.

We are leaving the primary visual cortex and entering what's called the extratistriate cortex.

Which just means outside the striate cortex.

So this includes regions bordering V1, like V2, V3, V4, and so on.

And what's the biggest change as we cross that border from V1 to V2?

Well the first thing that changes is the scope.

Remember, in V1 the receptive field is tiny.

It's a keyhole.

Right.

As we move to V2 and beyond, those fields get bigger.

The cells start looking at larger chunks of the visual world.

And more importantly, they stop caring about just edges and start caring about relationships.

And specifically, they start caring about ownership.

Yes.

This is a concept that really, it just blew my mind when I read the section on border ownership.

It sounds like a legal dispute, not neuroscience.

It is a sophisticated leap in logic.

So on edge you have a black square sitting on gray background.

There's a sharp edge where the black meets the gray.

Okay.

Simple enough.

Now to a V1 cell, that's just a contrast line.

Dark on one side, light on the other.

It fires because there is a line.

It doesn't know anything else.

But V2 is smarter.

V2 is a lot smarter.

V2 asks a critical question.

Who owns this edge?

Does the edge belong to the square?

Or does it belong to the background?

Correct.

Because in our physical world, edges are almost always the boundaries of objects in the foreground.

The background just, it continues behind them.

And the text details a really elegant study on this by Zhu, Friedman, and Vonderheide.

It's in figure 4 .5 that really nailed this down.

This is in the scientists at work section.

Yes.

It's a fantastic piece of work.

They recorded from single neurons in the V2 area of monkeys.

So they showed the monkey an edge.

A simple vertical line, dark on the left, light on the right.

The V2 cell fired.

Standard stuff so far.

But then they flipped the context.

Exactly.

They kept the edge exactly the same.

Same contrast, same location on the retina, everything.

But in one case, that edge was the right side of a black square that was sitting to its left.

So the square owned the edge.

Yes.

In the other case, the edge was the left side of a black square sitting to its right.

So visually, the local edge is identical.

But the object it belongs to has moved.

And the cell knew the difference.

This is the amazing part.

They found neurons that would fire enthusiastically if the edge belonged to an object on their left, but would go silent or fire much, much less if the exact same visual edge belonged to an object on their right.

That is just wild.

The cell isn't just seeing the light.

It's seeing the meaning of the light.

Yes.

It knows this line is the boundary of the thing over there.

It's the beginning of object segregation.

It's the first step to pulling an object out from its background.

And this gets even more complex when you look at something like transparency.

Right.

There was another study by Kew and Vonderheide mentioned, figure 4 .6.

Yes.

They used these clever displays where it looked like a transparent strip, like a piece of see -through colored tape was laid over a background with a different color.

So you still see a line, a change in color, but your brain interprets it not as the edge of a solid object, but as the edge of something you can see through.

Exactly.

Now, strictly speaking, there is still an edge there.

There is a change in brightness and color.

But they found that these border ownership cells in V2 are suppressed.

They stop firing when the brain interprets that edge as part of a transparency.

It's like the cell says, oh, wait, that's not a real object boundary.

That's just a see -through overlay, false alarm.

I'm going to stand down.

Precisely.

It filters out the irrelevant information.

The text uses the penguin example from figure 4 .7 to show why this matters in the real world.

Imagine a group of penguins standing against white snow.

Black and white birds, white background.

So you've got high contrast everywhere.

Right.

There are black to white edges all over the place.

Some of those edges are the outline of the penguin against the snow.

But some black edges are just a patch of black feathers against white feathers on the same penguin's belly.

If you couldn't distinguish them, you'd just see a kaleidoscope of blobs of black and white patches.

You wouldn't see birds.

You'd see texture.

V2 is where the brain starts to say, this edge here is the outline of the bird.

That's important.

That's a figure.

And this edge over here is just a pattern on the surface.

Ignore it for now.

That is border ownership in action.

OK.

So we've established that as we move up the chain, the brain gets a lot smarter about context.

But now the road forks.

The text describes two massive superhighways of information leaving the back of the brain.

The what and the where.

Yes.

The dorsal and ventral streams.

This is one of the most fundamental divisions in all of neuroscience.

So if you imagine the visual cortex, the back of your head, one path goes up kind of over the top of the brain.

That's towards the parietal lobe.

That's the dorsal pathway.

And we call that the where pathway.

Right.

Where, meaning location, yes, but also action.

Where is the baseball coming on my face and how do I position my hand to catch it?

So it's about interacting with the world.

Very pragmatic.

It's about navigating space and guiding your actions.

OK.

And the other path?

The other path goes down towards the bottom of the brain into the temporal lobe.

That's the ventral pathway.

And that is the what pathway.

And that's our focus for today.

Recognition.

Correct.

The ventral stream is the librarian.

It wants to know the identity of the thing.

Is that a baseball?

Is it a rock?

Is it an apple?

It's all about identification.

The evidence for this split is really dramatic.

The text discusses lesion studies, cases where parts of the brain are damaged.

I was reading about Kluver -Bessie syndrome.

This sounded truly terrifying.

It is bizarre.

Heinrich Kluver and Paul Bessie were researchers back in the 1930s.

They did these experiments where they removed the temporal lobes, the what pathway, from monkeys.

But they surgically removed the part for object recognition.

Right.

And when the monkeys woke up, they could see it perfectly well in a navigational sense.

They could walk around the lab.

They wouldn't bump into walls or fall off tables.

So the where pathway, the dorsal stream was working perfectly.

They could navigate space just fine.

Exactly.

But they had what was called psychic blindness.

They would walk up to a snake,

something monkeys are instinctively terrified of, and they'd just pick it up.

No fear at all.

None.

They'd pick up a rock and try to eat it.

They'd pick up a piece of food and try to put it in their ear.

They'd lost all connection between the visual form and its meaning or its use.

They could see the object, but they had absolutely no idea what it was.

The link between visual input and meaning had been completely severed.

This is essentially visual agnosia, the failure to recognize objects despite being able to see them.

There's that famous case study of the patient D .F.

that's mentioned a lot in the literature.

Yes.

She had damage to her pathway from carbon monoxide poisoning.

It's a classic case.

If you held up a card with a slot cut in it and asked her to orient her hand to match the angle of the slot, she couldn't do it.

She couldn't describe the orientation at all.

She couldn't perceive it.

No.

But if you then said, okay, now post the card through the slot like mailing a letter, she could do it perfectly.

She could act on it.

Her motor system guided by her intact dorsal stream could see the angle and guide her hand.

But her conscious perception, her what system was blind to it.

She could act on it, but she couldn't perceive it.

That separation between action and perception is fascinating.

It really implies we have these two different visual brains working in parallel.

We do.

But let's stay on the what pathway.

We are moving deeper into the temporal lobe, into a region called the inferotemporal cortex or IT cortex.

The text says the cells here get even weirder.

They do.

In V1, we had cells that liked lines.

In V2, we had cells that cared about borders.

By the time you get to the IT cortex, the receptive fields are huge.

Some of them cover half your entire field of view.

And they don't care about simple lines anymore.

Not at all.

They care about things, very specific things.

This is where we found the toilet brush cell.

Yes.

It sounds like a joke, but it was a real finding from a real study.

Researchers were showing monkeys all sorts of different objects to see what would make a specific IT cell fire.

It wouldn't fire for a face.

It wouldn't fire for a hand.

It wouldn't fire for a banana.

And then they just happened to show it a toilet brush.

And the cell went crazy, fired off the charts.

But why?

Why would evolution give a monkey a toilet brush cell?

It doesn't make any sense.

Well, it probably wasn't evolved for toilet brushes.

It was likely a cell tuned to a very specific combination of shapes and textures, maybe a long cylinder with a particular kind of spiky texture on top, that the toilet brush just happened to satisfy perfectly.

So it's about the complexity.

It is.

The point is the leap in complexity.

We have gone from I see a vertical line in V1 to I see a complex 3D shape with a specific texture in IT.

But there is a huge gap in the middle, isn't there?

We can't just jump from a line to a brush.

There has to be some kind of middle management, a layer that organizes the lines into shapes before we can identify them.

And that is what we call mid -level vision.

The text calls this the logic of perception.

I really like that phrasing.

It implies that seeing isn't just a passive recording.

It's a logical process.

It's an argument.

It is a problem -solving process.

Mid -level vision's job is to take the chaotic output from V1 and organize it into something coherent.

And one of the first problems it has to solve is simply,

where are the edges, really?

Which sounds easy.

We just talked about edge detectors.

But in the real world, edges are messy.

The text gives a great example in figure 4 .11, comparing human vision to computers.

If you tell a primitive computer program to find the edges in a photo, it just looks for sharp differences in brightness, luminance, steps.

But what happens if a dark rock is sitting on dark sand?

The edge disappears.

There's no contrast for the computer to find.

Right.

Or what if a shadow falls across a white table?

The computer sees a huge black line and thinks edge, object, boundary.

But humans don't make that mistake.

We ignore the shadow.

And we fill in the gap where the rock matches the sand.

We literally hallucinate lines that aren't physically there to make the world make sense.

This brings us to illusory contours.

The text shows the canissa arrow as an example.

A classic demonstration from figure 4 .1.

So imagine an arrow shape that is white, sitting on top of a black background.

Easy to see.

But now,

remove the black outline of the arrow.

So you just have a black background with a specific chunk missing that looks like an arrow shape.

Like a stencil cutout.

Right.

Even though there is no line drawn there, it's just white paper meeting white paper.

You see a sharp, crisp, white edge defining that arrow.

It even looks a little brighter than the rest of the page.

It's the Pac -Man illusion too, isn't it?

If you arrange three Pac -Men with their mouths facing each other.

In a perfect triangle formation.

You see a bright white triangle in the middle that isn't actually drawn.

It looks like a white triangle is sitting on top of three black circles.

And the question is, why do we see it?

Why does the brain invent a line where there is none?

Because mid -level vision is using logic.

It says, look, the only reason these three black circles would have these precise, perfectly aligned wedges cut out of them is if a white triangle was sitting on top of them.

It infers it assumes something is blocking the view.

It honors the laws of physics.

In the real world, things block other things.

The brain decides that three Pac -Men arranged in a perfect circle is a highly suspicious coincidence.

But a triangle on top of three circles is a common physical probability.

So it draws the triangle for you.

Honoring physics.

That seems to be a recurring theme here.

The brain has this built -in assumption that the world follows physical rules.

It has to.

Otherwise, the visual data is just far too ambiguous to interpret.

Yeah.

And this connects directly to the history of psychology.

This phenomenon, seeing a triangle that isn't there, was basically the nail in the coffin for an old school of thought called structuralism.

Structuralism.

Let's define that.

That was the old school, right?

Vrun and Titchener in Germany.

Late 19th, early 20th century.

Structuralism was the idea that perception is just the sum of its atoms, like pixels on a screen.

They believed that if you added up all the little red sensations and blue sensations, you got the final picture.

Red plus red plus blue.

Perception.

But the Kinesa Triangle completely breaks that math.

It shatters it.

Because in the gap where you see the edge of the triangle, there are no pixels defining it.

The sensory input is zero, yet the perception is edge.

So the whole is demonstrably different from the sum of the parts.

And that's where the Gestalt psychologists came in.

The Gestalt Revolution.

This was happening in Berlin in the 1920s.

Thinkers like Max Wertheimer, Wolfgang Kohler, Kurt Kafka.

Their central argument was that the brain isn't passive.

It actively uses a set of grouping rules to organize and make sense of the world.

Mid -level vision is basically a committee in your head using these rules to sort the mail.

Let's run through these rules because they are constantly, unconsciously working in our The text calls them the rules of evidence.

The first one is good continuation.

This is the rule that says we tend to group elements that lie on a smooth contour.

The text shows a picture of two lines crossing each other, forming an X.

Now theoretically, that image could be two V -shapes kissing at the point.

Right, like two little boomerangs touching noses.

Oh, exactly.

Visually, that is a perfectly valid interpretation of the lines on the page.

But you never see it that way.

Not once.

You always see two long straight lines crossing over each other.

Why?

Because lines in nature tend to be smooth.

They don't make abrupt sharp 90 -degree turns for no reason.

So the brain groups the segments into the smoothest possible lines.

It makes a bet on continuity.

And the text mentions a study by Geisler and Perry that actually backed this up with hard data from nature photography.

Yes, they did this fascinating analysis of natural images.

Photos of forests, rivers, landscapes.

They measured the statistics of edges.

And they found that if you have an edge at one point, the edge right next to it is highly, highly likely to be aligned with it.

Nature doesn't do jagged randomness as often as it does flow.

So our brain has evolved to bet on that probability.

Okay, so that's good continuation.

Then you have the rule of similarity.

Similar things group together.

This one is really intuitive.

If I show you a grid of shapes that's half circles and half squares, your brain automatically groups the circles as one unit and the squares as another.

This is the basis of texture segmentation.

How we separate different surfaces.

Exactly.

It helps us separate a leopard from the tall grass.

The spots on the leopard have a similar texture, so they group together.

The blades of grass have a similar texture, so they group together and you can tell them apart.

And of course, proximity.

Things that are close together form a group.

Yes.

If you see two people standing very close to each other at a party and everyone else is far away, you automatically assume those two are a group.

They're interacting.

Visual objects work the exact same way.

The text also brings up parallelism and symmetry.

These are powerful grouping cues.

If you see two wavy lines that are parallel to each other, you are very likely to perceive them as belonging to the same object, like the two banks of a river or the sides of a snake.

Because it would be a huge coincidence otherwise.

A massive coincidence.

The brain hates coincidences.

It assumes there's a common cause.

Symmetry is similar.

Symmetrical shapes are almost always single objects.

And then there's a cool illustration in the book, Figure 4 .22, that shows how these rules can compete.

It has dots arranged and you can see how proximity is overridden by a common region or connectedness.

Right.

So you have a grid of dots.

The ones that are close together group by proximity.

But if you draw a circle around two dots that are far apart, suddenly they become a group

overriding proximity.

That's common region.

And if you draw a line connecting them.

Connectedness wins.

It's the strongest grouping cue of all.

A line physically connecting two things almost forces you to see them as a single unit.

Okay.

So these rules help us organize the world, but the text brings up a really cool counter example.

Camouflage.

Camouflage is basically weaponized gestalt psychology.

It is the art of using these grouping rules against the viewer.

You want to break your own grouping to prevent the viewer from separating you from the background.

Exactly.

If you are a moth on tree bark, you want your texture to be as similar as possible to the bark.

So the predator's brain groups you with the tree.

But the text mentions a different kind.

Dazzle camouflage from World War I, which I find fascinating.

The ships, right?

With the crazy patterns.

Yes.

They painted warships with these wild, high contrast,

zigzag geometric patterns.

Black and white stripes, triangles, things that made no sense.

And it didn't make the ship invisible.

You could clearly see there was a ship there.

No, it wasn't about invisibility.

It was about confusion.

The patterns were designed to break up the gestalt rules of good continuation and symmetry.

The bold stripes of the paint made it almost impossible for the enemy submarine captains looking through their periscopes to tell which way the ship was pointing or how fast it was moving.

Because the visual edges of the paint were stronger and more salient than the actual visual edges of the ship's hull.

Precisely.

It jammed the mid -level vision of the enemy.

They couldn't form a coherent whole object quickly enough to calculate a trajectory and fire a torpedo accurately.

It's brilliant.

It's visual jamming.

So we have all these rules.

Good continuation, occlusion, similarity, proximity.

The text uses a metaphor that I really like.

The perceptual committee.

It's the perfect way to think about it.

You don't have a single dictator in your visual cortex making decisions.

You have committee.

And they are all yelling their opinions based on these rules.

So the similarity specialist is saying, group these red dots together.

And the proximity expert is yelling, no, no, group these two dots because they're close to each other.

And usually they agree.

The red dots are also the ones that are close together.

Right.

Usually the world is redundant.

The cues reinforce each other and the committee reaches a quick unanimous decision.

But sometimes they get deadlocked.

And that's when we get ambiguous figures.

Like the Necker cube.

Figure 4 .24.

We've all seen it.

The simple wire frame drawing of a box.

You look at it and the front faces down and to the left.

Then you blink and suddenly the front faces up and to the right.

It just flips back and forth.

Why does it flip?

What's happening in the committee room?

The committee is hung up.

Both interpretations, front face down and front face up, are perfectly valid 3D shapes.

Neither one violates the laws of physics or the Gestalt rules.

There is no depth cue, no shadow, no occlusion to tell you which surface is in front.

So the brain oscillates.

It tries one hypothesis, then gets bored or fatigued, and switches to the other.

It's the same thing with the duck -rabbit illusion, isn't it?

Is it a duck looking left?

Or is it a rabbit looking right?

Exactly.

The sensory data, the actual drawing on the page, doesn't change by a single pixel.

But your perception changes dramatically.

That is the ultimate proof that perception is an active decision made by the committee, not just a passive recording of the input.

But the text makes a crucial point here.

Ambiguity is actually very rare in real life.

Usually the committee solves the problem instantly, and they do it by following a very strict overarching rule.

Avoid accidents.

This is the accidental viewpoint principle.

It's one of the smartest, most fundamental things your brain does to make sense of the world.

Okay, break it down for us.

Imagine looking at a photo where a tourist is standing in the foreground and the leaning tower of Pisa is way in the background.

We've all seen those forced perspective photos.

Right, where they line it up perfectly so it looks like the person is holding up the tower.

Their tiny hand seems to be touching the side of the massive building.

Forced perspective, exactly.

Yeah, your brain can see that illusion in a 2D photo.

But if you were actually standing there in the 3D world, your brain would reject it immediately.

Why?

Because for that person's hand to line up perfectly with the edge of a building that is a mile away, you have to be standing in one precise exact millimeter of space.

It's a one in a billion coincidence of alignment.

If you moved your head one inch to the left, the whole illusion would break.

And the Perceptual Committee absolutely hates relying on one in a billion coincidences.

It operates on the assumption that we are in a generic viewpoint, a standard random position.

So if two shapes line up perfectly, the brain assumes they're actually touching or related, unless there is very strong evidence otherwise.

It rejects the accidental interpretation.

So the committee says, as in figure 4 .26, it is highly unlikely that those four random L shapes just happen to align to form a perfect square from this one angle, so they must actually be a square viewed from an angle.

Exactly.

It's probabilistic reasoning.

It's always asking, what is the most likely state of the world that would produce this image on my retina?

And a weird alignment is almost never the most likely answer.

Let's move to another big job for this committee, figure and ground.

This seems so basic, knowing what is the object and what is the background.

But the Rubin Vase illusion shows us it's not simple at all.

The Rubin Vase, figure 4 .29, is that classic image that can look like a white vase, or two black faces in profile looking at each other.

It's bastible, just like the Necker cube.

But the key thing to notice is the edge, the contour between the black and white.

The text points this out.

The edge always belongs to the figure.

When you see the vase, the contour is the curved side of the vase.

The black area is just a formless void behind it.

But when your perception flips to the faces, that exact same contour becomes the profile of a nose and a chin.

The white area becomes the formless void.

So ownership of the border flips.

How does the committee decide in the real world?

We don't usually see vases turning into faces.

There are rules for figure -ground assignment, too, just like for grouping.

One is surroundedness.

If one region is completely surrounded by another, the surrounded thing is almost always the figure.

Like a cow in a field.

The cow is surrounded by grass, so the cow is the object, the figure.

Right.

Another rule is size.

Smaller things are usually figures.

The cow is smaller than the meadow, therefore the meadow is ground.

Symmetry is another big one.

Symmetrical shapes are much more likely to be objects than asymmetrical background gaps.

The text also mentions extremal edges, which has to do with shading and how it tells us what's in front.

But I want to talk about the relatability concept, because this goes back to occlusion.

The text has this great diagram, figure 4 .33, of a square covering a line, and we have to decide if the line continues behind the square.

This is from Kelman and Shipley's work.

They called it the relatability heuristic.

Basically, the brain acts like a connect -the -docs puzzle solver, but with a rule.

Imagine a wire going behind a box.

You see the wire enter on the left side and exit on the right side.

Are they parts of the same wire?

Well, it depends on how they line up.

Exactly.

If the line segment entering and the line segment exiting can be connected with a smooth, simple curve, what they call relatable, the brain says, yes, that is one continuous wire running behind the box.

But if connecting them would require a crazy jagged S curve with sharp turns.

The brain says, no way.

They conclude they are two different wires that just happen to end at the edge of the box.

It's basically Occam's razor for vision.

The simplest explanation, the smoothest curve, is the one the brain assumes is right.

The text also mentions a set of cues called non -accidental features, like T junctions and Y junctions.

This sounded technical at first, but the visual is actually very clear.

It's surprisingly simple geometry that your brain calculates instantly to understand 3D structure.

Look at a corner of a cardboard box.

Where the three sides meet, the lines form a Y shape.

That Y junction is a robust, non -accidental signal to the brain that says, this is a convex corner of a solid object.

OK, but now look at where a table leg passes behind a chair seat.

The edge of the table leg hits the edge of the chair seat and stops.

It forms a T shape.

The top of the T is the continuous edge of the chair, the front object.

The stem of the T is the edge of the table leg, the back object.

So T junctions are a dead giveaway for occlusion.

Y junctions signal corners.

And your brain spots these junctions instantly all over the visual scene to map out the 3D space.

If you erased all the T junctions in a complex drawing, the sense of depth would collapse.

You wouldn't know what was in front of what.

OK, we're moving steadily through the committee's rulebook.

Now we get to the section on parts and holes and the global superiority effect.

This is basically the force before the trees concept and vision.

David Navon did this famous study in the 70s using what are now called Navon letters.

So imagine a giant letter H.

But if you look closely, the lines of the H are made of lots of tiny letter S's.

OK, I'm visualizing it.

A big H made of small S's.

Now, if I flash that on the screen and ask you to identify the big letter, you do it instantly.

It's an H.

But if I ask you to identify the small letters, what are the little letters it's made of, it takes you slightly longer to say S.

So we see the whole the global shape of the H before we process the parts, the S's.

Exactly.

We process the global shape first.

This really challenges the simple bottom up idea that we always build objects from scratch pixel to line, line to shape, shape to object.

Sometimes it seems we grasp the whole object structure first and then we fill in the details.

And explaining all of this, the probability, the rules, the guessing, the favoring of one interpretation over another brings us to the Reverend Thomas Bayes, the Bayesian approach.

We can't do a deep dive on modern perception without talking about Bayes.

It's the mathematical framework that formalizes everything we've been discussing.

At its core, Bayes' theorem is about calculating the probability of a hypothesis being true, given the evidence you've observed.

Okay.

Can you put that in plain English for us?

How does my brain use this?

Okay.

The formula, in essence, involves two key things, the prior probability and the likelihood.

The prior is your pre -existing belief.

How likely is this thing to exist in the world in general, the likelihood is?

How well does the sensory data I'm getting right fit that thing?

The text uses the green square example in figure 4 .37 to explain this.

That's a great example.

Imagine you look at a table and you see a green square, but it has a perfect circular chunk missing from one corner.

And right next to it, you see a green circle.

Hypothesis A is that there is one green square occluding or blocking another green square.

Hypothesis C in the book is that it's one square occluding a circle.

Let's use that one.

So the sensory data could be explained by Hypothesis C.

A whole square is sitting in front of a whole circle.

But it could also be explained by another hypothesis, say Hypothesis B, which is that you're looking at two weirdly shaped objects, a square with a bite out of it and a separate circle that just happened to be lined up perfectly.

Right.

Sensory -wise, on your retina, both hypotheses match the data perfectly.

But the Bayesian brain then asks about the priors.

The prior probability of a simple square is very high.

Squares are common shapes.

The prior probability of a circle is also high.

But the prior probability of that specific weird Pac -Man shit is incredibly low.

And the probability of it lining up perfectly with a circle by accident is even lower.

Exactly.

So the brain multiplies the prior high by the likelihood high for the occlusion hypothesis and gets a high probability.

It does the same for the accidental alignment hypothesis and gets a very low probability.

It concludes it's a square in front of a circle.

We are constantly making these bets.

We are betting machines.

We are betting machines.

Our brains are constantly betting on the most likely reality.

And 99 .9 % of the time we win the bet.

That's why we survive.

Speaking of reality, we don't just see shapes.

We see stuff.

The section on material perception really resonated with me.

I look at a surface and I just know if it's slippery or fuzzy or cold or wet.

This is a For decades, vision science focused almost entirely on geometry.

Is it a cube or a sphere?

But knowing what something is made of is absolutely crucial for survival.

Is that rock stable or is it a sponge?

Is that food fresh or is it rotten?

The text mentions a study by Sherian and colleagues showing we can identify materials.

Metal, plastic, wood, stone in as little as 40 milliseconds.

40 milliseconds.

That's effectively instantaneous.

It's faster than you can even identify the object itself sometimes.

But how?

We aren't touching it.

How do we know from light alone?

We use a whole host of optical cues.

Specifically, we look at how light interacts with the surface.

Think about a chrome bumper versus a rubber tire.

They might both be the same curved shape, but the chrome has sharp, crisp highlights, what we call specular reflections.

The rubber has a soft, diffused glow.

So the brain analyzes the sharpness of the shine to tell metal from rubber.

It analyzes the specularity, the texture, the color distribution, the transparency, and it does it so fast that by the time you realize you are looking at a cup, you already know it's a plastic cup.

The material perception happens in parallel with the shape perception.

So we have edges, we have mid -level groups, we have materials.

Now we finally arrive at the destination.

High -level object recognition.

We are deep in the temporal lobe now, the IT cortex.

This is the end of the line for the wet pathway.

And this is where things get really, specific.

We talked about the toilet brush cell, but we need to talk about the concept of the grandmother cell.

The famous grandmother cell.

The theoretical idea that there is one single neuron in my brain that fires if and only if I see my grandma.

It was a very controversial idea for a long time.

It implies what's called specificity coding.

One cell, one object.

For decades, people thought it was too simple, too brittle.

They favored the idea that recognition was distributed, a broad pattern of activity across millions of cells.

But then came the Jennifer Aniston study.

This is one of the most famous studies in modern neuroscience.

Kuroga and his colleagues in 2005.

It's just wild.

It is.

They were working with epilepsy patients who had electrodes implanted deep in their temporal lobes for medical reasons.

This gave them a rare opportunity to record from single human neurons while the patient was awake and looking at pictures.

And they found it.

They found a neuron in one patient that fired vigorously, but only when shown a picture of the actress Jennifer Aniston.

Not just photos of her face, right?

That's the amazing part.

Photos of her, drawings of her, even her name written down on a card.

But if they showed that same neuron,

a picture of Julia Roberts.

Silence.

A picture of a famous building.

Silence.

It was, for all intents and purposes, the Jennifer Aniston cell.

They found another one for the Sydney Opera House and I think another for Bill Clinton.

They did.

So it proves that the brain does create these highly specific abstract representations for familiar concepts.

It suggests that your temporal lobe is a kind of library of all the important people, places, and things in your life.

But, and this is a big but, we shouldn't take the grandmother cell idea too literally, should we?

No, probably not.

If that one single cell died, you wouldn't suddenly forget who Jennifer Aniston is.

It's more likely a sparse network of cells, maybe a few thousand cells that are dedicated to representing her.

But compared to the billions of neurons in the brain, that is still incredibly specific.

The text also lists these specific modules in the brain that fMRI studies have found.

Regions that just seem to do one thing.

Yes.

Functional imaging has shown us we have dedicated hardware for important categories.

We have the FFA,

the fusiform face area.

It lights up for faces and basically nothing else.

We have the PPA, the parahippocampal place area.

It lights up for scenes and places like houses and landscapes.

And the EBA?

The extra storage area, body area.

It lights up for body parts, arms, legs, torsos, but not faces.

And the VWFA, the visual word form area.

This one is fascinating because it proves the brain is plastic.

We didn't evolve to read.

Writing is far too new.

Exactly.

5 ,000 years is a blink of an eye in evolution.

Yet when we learn to read, a specific part of the visual cortex, usually on the left side, repurposing some real estate that might have been used for faces,

dedicates itself to recognizing letter strings.

That is so cool.

It's like our brain downloads a software patch for reading and installs it on the existing hardware.

That's a great way to put it.

So we know where recognition happens, but how does it actually work?

This leads us to the models of recognition.

How do you actually build a system to recognize the letter A?

The oldest and simplest idea was templates.

The brain has a little stencil of an A.

It tries to match the visual input to the stencil, a lock and key model.

But that fails immediately because an A can be big, small, italic, handwritten, a different font.

You'd need an infinite number of templates.

It's just not practical.

So in the 1950s, a guy named Oliver Selfridge came up with the pandemonium model.

Pandemonium.

I love the name.

The idea is to imagine a tower of demons all shouting.

Demons in the brain.

It's a metaphor.

At the bottom layer, you have feature demons.

They just look for simple features.

One demon screams if it sees a vertical line.

Another screams for a horizontal line.

Another for a curve.

Okay, so they're like the V1 cells.

Exactly.

Above them are cognitive demons who are looking for letters.

The H demon, for example, listens to the feature demons below.

If he hears the vertical line demon screaming and the other vertical line demon screaming and the horizontal line demon screaming, the H demon starts screaming.

And at the top of the tower.

At the very top, a decision demon listens to all the shouting cognitive demons and simply picks whoever's screaming the loudest.

And that's the letter you perceive.

I love the imagery.

It's recognition by democratic mob.

It's a classic feature -based model.

It works better than templates because an H is always made of the same features, no matter the size or font.

But the big modern breakthrough, which the text highlights,

is deep neural networks or DNNs.

This is the AI stuff we hear about constantly today.

Self -driving cars, image recognition.

Yes, but the architecture is directly inspired by the biology we just discussed.

A DNN has multiple layers.

The bottom layer acts like V1.

It finds simple edges and lines.

The next layer pools those features into more complex shapes like V2 and V4.

And the top layers combine those shapes to recognize whole objects like the IT cortex.

And how do they learn?

We train them.

We show them millions of pictures of cows and for each one we say, This is a cow.

If the network guesses dog, we send an error signal back down to the network and tweak all the connections slightly, a process called backpropagation until it gets it right.

And they are getting scarily good.

The text mentions deep dream and these decoding studies where they can actually stand a person's brain while they look at a picture, feed that pattern of brain activity into a computer, and the computer can make a pretty good guess as to what the person is looking at.

It's essentially mind reading via pattern recognition.

It's incredible.

But the text adds a very important cautionary note.

DNNs are not exactly like human brains.

They can be fooled in ways that humans never would be.

They're susceptible to adversarial attacks.

This is where you can add a tiny bit of static noise to a picture of a panda.

Right.

A layer of noise that is completely invisible to a human.

To you, it still looks exactly like a panda.

But the computer sees this specific engineered noise and its recognition process breaks completely.

It might suddenly say given with 99 % confidence.

That's really scary.

It is.

It shows that while these AI systems mimic our brain's hierarchical structure, they still lack that robust mid -level vision common sense.

The understanding of physics, the context, the avoiding accidents principle that we talked about earlier, they're brittle.

Okay.

We have one last stop on this tour of the what pathway.

And it's maybe the most important object in our social world.

Faces.

Faces are special.

The brain absolutely treats them differently than it treats chairs or cars or books.

We process faces holistically.

Meaning as a whole, not as a collection of parts.

Correct.

When you look at a friend, you don't see eye plus eye plus nose plus mouth, and then add them up to equal friend.

You see the entire face configuration all at once.

The evidence for this is a powerful illusion called the face inversion effect.

The Thatcher illusion.

This freaks me out every single time I see it in a textbook.

It is deeply disturbing.

Can you describe it for the listener?

Figure 4 .49 in the book.

Okay.

So you take a picture of a face.

Margaret Thatcher's was the original.

You leave the face right side up, but you digitally cut out the eyes and the mouth, and you flip them upside down within the face.

So the face is upright, but the eyes and mouth are inverted.

And it looks.

It looks monstrous, genuinely grotesque, like a demon.

But, and this is the critical part of the illusion.

If you take that monstrous photo and turn the whole thing upside down, It looks fine.

Almost normal.

You can barely notice that anything is wrong with it.

That's because when a face is upside down, our dedicated holistic face processor in the brain shuts off.

It doesn't work on inverting faces.

So we switch to recognizing it by its parts.

And the parts, the eyes and the mouth, are technically right side up relative to the page.

So they look normal to our part based object recognizer.

But as soon as you flip the photo right side up again, The holistic face processor kicks back in,

sees that the configuration is all wrong.

The eyes are wrong relative to the nose.

The mouth is wrong relative to the eyes.

And it screams monster at you.

That is such a concrete and powerful proof that faces are a unique processing category in the brain.

It is.

It shows that object recognition isn't a single monolithic system.

It's a collection of specialized committees, specialized brain areas, and specialized rules.

All working together in this incredible cascade to create the seamless visual movie that's playing in your head right now.

So let's wrap this up.

We started our journey with the blind men in V1.

Peering through tiny keyholes at simple lines and edges.

Just seeing the trunk and the leg.

We moved to V2 where the brain started to figure out who owns the borders separating figure from ground.

Then we went to the mid -level committee room where they shouted about gestalt rules, the laws of physics, and the importance of avoiding accidents.

And we used Bayes' theorem to place a bet on the most likely reality that could have created that image.

We sent that organized information down the temporal lobe, down the what pathway to the IT cortex, where a highly specialized sparse network of neurons finally identified Jennifer Aniston.

And we did all of that in a fraction of a second.

And we didn't even have to try.

That is the miracle of it.

All that heavy lifting, all that debate and calculation, is completely hidden from your conscious mind.

You just get the result.

You just see the world.

It really makes you appreciate just looking around the room.

Every single glance is a computational masterpiece.

Absolutely.

The world that comes in through your eyes is ambiguous and chaotic.

Your brain makes it definitive.

It really is.

And it leaves you with a final thought, doesn't it?

If so much of perception is about probability, about your brain making its best guess based on past experience and built -in rules, how much of what you see as objective reality is just your brain's most successful and consistent bet?

That's a deep one to end on.

Well, thank you for listening to this deep dive into Chapter 4.

A huge thank you from all of us here at the Last Minute Lecture Team.

Keep those eyes open, and we'll see you in the next deep dive.