Chapter 1: Introduction to Sensation & Perception

Welcome to Last Minute Lecture.

This free chapter overview is designed to help students review and understand key concepts.

These summaries supplement not replaced the original textbook and may not be redistributed or resold.

For complete coverage, always consult the official text.

Welcome back to the Deep Dive.

I want to start today by asking you to do something that might feel a little silly.

Wherever you are, unless you're driving, I want you to hold up your hand in front of your face.

Just, you know, look at it.

That's a classic move.

What do you see?

I see skin.

I see my fingerprints.

I see a little scar on my thumb and it feels, well, it feels immediate.

It feels like I'm just directly accessing reality, right?

Right.

My eyes are open.

The light hits my hand and boom, I see my hand.

It feels instant.

It feels unmediated.

But we have a text in front of us today, chapter one of sensation and perception, sixth edition.

And it is essentially here to tell you that you are completely wrong.

Completely wrong.

It's an illusion.

You aren't seeing your hand.

Not really.

What you are experiencing is a reconstruction,

a simulation, really running inside this dark sealed box that we call a skull.

So the reality that I feel so confident about is actually the end result of this massive, messy biological construction project.

Exactly.

And that is the mission for today.

We're taking a deep dive into this chapter to answer, I mean, maybe the most fundamental question there is, how do we know what we know?

How does the physics of the world, like light and sound waves, become the psychology of the mind?

It is the bedrock of everything.

I mean, psychology, neuroscience, biology,

even AI.

It all starts right here with this question.

So let's get into it.

The book opens with this really striking image, figure 1 .1, and it's just a hand holding a smartphone.

It's totally mundane, right?

But it's a trap, a philosophical trap.

It is.

It asks a couple of questions.

Does the phone feel the fingers holding it?

Does it hear your voice when you speak into it?

Well, let's play devil's advocate for a second.

My phone has a microphone.

It has a touch screen with all these pressure sensors.

So technically, it is capturing data.

It reacts.

Right.

If I touch the screen, an app opens.

So why isn't that perception?

Why can't we say the phone feels my touch?

That's what I thought at first.

I mean, it's sensing something, isn't it?

But the book draws this really hard line in the

between sensation and perception.

And that's the key.

Think about it this way.

The phone registers, what?

XY coordinates of pressure.

It registers sound waves as digital values.

But does the phone know why you're touching it?

No, of course not.

Does it feel the warmth of a hand?

I mean, if someone runs a finger down your back, you don't just register pressure moving south at two centimeters per second.

Oh, you feel something.

It could be affection.

It could be creepy.

It could just be a tickle.

You give it meaning.

Right.

And that is the difference.

Sensation is the raw data, the pressure, the photons, the sound wave.

Perception is the absolute miracle of giving that data meaning and purpose.

So the phone has sensors, but it has no internal life.

The book uses that phrase, private experience.

It has no private experience.

Exactly.

And that word prize it is so important.

And it leads us directly to this incredibly trippy thought experiment from the 1700s.

Oh, this is Etienne Bonnot de Condeac and his statue.

Figure 1 .2.

This is one of my favorite philosophical nightmares.

So Kondelac asks us to imagine a marble statue.

It has a body and inside and outside, but it has no senses, no eyes, no ears, no touch nerves, nothing.

Basically just floating in a void.

It's worse than a void.

A void implies, you know, blackness or silence.

But you need eyes to see blackness.

You need ears to hear silence.

This statue has, it has nothing.

It has no concept of self because it has no boundaries with the world.

Okay.

That is genuinely terrifying to think about.

And then Kondelac says, let's give it just one sense.

We'll unlock smell and we hold a rose under the statue's nose.

So now the statue smells a rose.

But here is the kicker.

The statue doesn't know it has a nose.

It doesn't know there's a rose out there in the world.

It doesn't even know it has a body.

All it has is this one singular sensation.

So to the statue, the entire universe is just scent of rose.

Kondelac argues that the statue becomes a scent of the rose.

There's no distinction between me and the smell.

They are one and the same thing.

Wow.

That's heavy.

It basically says that without these sensory inputs to place things in space and time and to distinguish self from other, we don't really exist as distinct entities.

We are just a collection of sensations.

Our whole mental life, our ego, our memories, our entire understanding of the world is completely dependent on these, I mean, kind of flimsy biological sensors.

Precisely.

If you cut the cables,

the you that you think you know just disappears.

So if sensation is the only thing tethering us to reality, we better figure out how it works.

But that's the hard part, isn't it?

Because my experience of red might be completely different from your experience of red.

How do you measure something so subjective, so private?

And that is the central challenge of psychophysics.

The chapter lays out this really clear road map of how we've tried to solve this problem.

It starts with history, you know, the dawn of trying to measure the mind.

Then it goes into the specific methods and the biology.

And finally it ends with computer models.

It's a journey from philosophy to physics, to biology, to AI, all in one chapter.

Let's start with the history, because we have to talk about Gustav Fechner.

Figure 1 .3 calls him the founder of experimental psychology.

But his origin story is, honestly, it sounds like a super villain backstory.

It really does.

So Fechner was operating in Leipzig in the mid -1800s.

He was a physicist, a brilliant guy, but he was completely obsessed with his philosophical problem.

How do mind and matter connect?

The text mentions he was a panpsychist.

What does that mean?

Panpsychism.

It's the belief that everything has a mind.

Not just humans and dogs, but plants, rocks, the planet itself.

It all has some form of consciousness.

So he thought the universe was conscious.

He did.

He actually wrote a book called Nana,

or Concerning the Mental Life of Plants.

He hated the idea of materialism that were just meat machines.

He wanted to prove that the spiritual and the physical were two sides of the same coin.

Okay.

And to do this research, specifically on vision, he made a very, very bad decision.

A catastrophically bad decision.

He stared at the sun.

For long periods of time.

Through colored glass, but yes.

He spent huge amounts of time staring at the sun to study afterimages.

A quick pro tip for our listeners.

Do not do this.

Please, please do not do this.

He severely damaged his retinas.

And this seems to have triggered a massive collapse.

He couldn't bear any light.

He had to resign his professorship at the university.

He painted the walls of his room black and lived in total isolation for three years.

A three -year breakdown.

And the text mentions his cure.

I have to read this directly.

He ate raw ham steeped in wine and lemon juice.

The spiced ham cure.

You can't make this stuff up.

And fruit.

That was it.

Spiced ham and wine.

Medicine in the 1840s was a wild west.

But here is the cinematic moment.

It's October 22, 1850.

They call it Fechner Day now.

He's lying in bed and suddenly he has this epiphany.

He realizes he could describe the relationship between the mental world and the physical world using mathematics.

But he didn't just invent it out of thin air though, right?

He was building on the work of another guy, Ernst Weber.

Right.

Weber was an anatomist and he was interested in touch.

He was doing these very simple experiments with weights.

Felser 1 .4 shows a setup like this.

He would have people lift a standard weight and then a slightly heavier one and just ask, can you feel a difference?

And this seems like it should be simple.

Either it's heavier or it isn't.

But Weber found a consistent pattern.

Let's say I give you a weight of 40 grams.

To feel a difference, I have to add one gram.

So 41 grams feels noticeably heavier than 40.

Okay.

That makes sense.

A one gram difference.

But then if I give you a weight of 400 grams and I add that same one gram,

you feel nothing.

You can't tell the difference at all.

Why not?

It's the same amount of added weight.

It's still one gram.

Because the background is so much heavier.

The initial stimulus is larger.

To feel the difference starting from 400 grams, I have to add 10 grams.

So it's a ratio.

It's not an absolute amount.

Exactly.

It's a constant proportion.

For weight, it's about one to 40.

This is what we call the just noticeable difference or sometimes called the difference threshold.

So Weber's big insight was that our sensitivity isn't absolute.

It's relative to the size of whatever we're sensing to begin with.

And that is Weber's law.

Fechner saw this and realized, this is it.

This is the mathematical key.

And he took this idea and turned it into a formal equation, which we now call Fechner's law.

The formula in the book is ZOS KRRDAI.

So sensation equals a constant times the

physical stimulus.

Right.

Okay.

Logarithm is a word that makes a lot of people's eyes glaze over.

Let's break down what that actually means for our daily life.

Okay.

Think about lighting a room.

Imagine you were in a pitch black cave.

Total.

Absolute darkness.

And I light one single candle.

The difference would be huge.

It would go from nothing to light.

It would feel blindingly bright compared to the darkness.

Right.

That one candle creates a massive jump in your psychological sensation of brightness.

Now let's change the scenario.

Imagine I already have a hundred candles burning in the room.

It's quite bright.

And now I light one more candle.

I probably wouldn't even notice the difference would be tiny.

Exactly.

Physically, I added the same amount of light energy one candle's worth.

But psychologically, the effect was negligible.

This is what the logarithm does in the equation.

It means that as the physical intensity grows and grows, our psychological experience of it grows much, much slower.

So the mind compresses the physical world.

It has to.

If it didn't, if our sensation grew one -to -one with the physical world, walking out into the sunlight on a bright day would probably be agonizingly painful.

It would just completely overload the system.

Fechner's law shows how our biology protects us from being overwhelmed.

Okay.

So Fechner gives us the math, but how do we actually measure these thresholds in a lab setting?

The chapter outlines three core methods and they are super practical.

Yeah.

These are the classic psychophysical methods.

And what we're usually looking for is the absolute threshold.

That's the minimum amount of stimulus energy a person can detect 50 % of the time.

And table 1 .1 lists some of these and they are just mind blowing.

For vision, it says we can detect a candle flame from 30 miles away on a dark, clear night.

It's incredible.

For smell,

one drop of perfume diffused through a three -room apartment.

For touch,

the wing of a bee falling on your cheek from a distance of one centimeter.

We are just unbelievably sensitive instruments, but to measure that bee wing reliably, we need a method.

So method one is the method of constant stimuli.

This is the gold standard for accuracy, but it is tedious.

So imagine I sit you down in a soundproof booth and I want to find the quietest sound you can possibly hear.

I pick a set of tones, some very loud, some totally silent and a bunch right on the edge of what I think you can hear.

And you have to go over again.

And for each one, you just say, yes, I heard it or no, I didn't.

And you'd expect that at a certain volume.

I just switch from always saying no to always saying yes, like flipping a light switch.

You would expect a sharp vertical line on the graph.

But if you look at figure 1 .6 in the text, it's not a vertical line at all.

It's a smooth S curve.

Why?

Why is it curvy?

Why isn't it a clean switch?

Because you are a biological machine and biology is messy.

Maybe on trial number five, you heard the tone perfectly.

But on trial number 20, the exact same tone, your stomach rumbled or a neuron in your brain misfired, or you're distracted thinking about lunch.

It's internal noise.

Exactly.

So the threshold isn't this hard cliff.

It's a probabilistic zone.

It's fuzzy.

So we define the absolute threshold as the point on that S curve where you say yes, 50 % of the time.

Okay, so that's super accurate, but it sounds like it would take forever.

Method two is the method of limits.

This feels a lot more like the hearing tests we get in school.

It is.

Here, we don't do random presentations.

We do runs.

I'll start with a sound that I know you can hear and I'll turn it down step by step until you say, stop, I can't hear it anymore.

That's a descending run.

And then you do the opposite, start from silence and go up.

Right.

Then I'll start with a tone that's too quiet to hear and turn it up step by step until you say, okay, now I hear it.

That's an ascending run.

And the threshold is just the average of all those crossover points where I switched my answer.

Correct.

It's much faster.

The problem here, though, is what the book calls overshoot.

People tend to get into a rhythm.

You might keep saying yes, yes, yes, even after the sound is gone, just out of habit.

Or you might keep saying no, no, no, when you can actually hear it, because you're anticipating it getting louder.

And finally, there's the method of adjustment.

This one seems like the easiest.

It's basically me with the volume knob.

Exactly.

You give the subject control.

Here's a dial.

Turn it until you can just barely hear the tone.

It's fast.

It's very intuitive.

But the book calls the data messy because people vary a lot in where they set that dial.

It's not usually used for really rigorous threshold testing.

So those methods find the limit, the kind of zero point of sensation.

But what about measuring the strength of an experience?

If I give you a spicy taco, I don't just want to know if you can detect the spice.

I want to know how spicy it is to you.

This is where we move from thresholds to scaling, specifically a technique called magnitude estimation developed by SS Stevens at Harvard.

And the task is pretty simple, right?

It is.

I give you a stimulus, let's say a moderately bright light, and I tell you this light has a brightness of 10.

Then I show you other lights.

If one feels twice as bright, you say 20.

If it feels half as bright, you say five.

You just assign numbers to your private experience.

And when Stevens did this, he found that Fechner wasn't 100 % right.

Not everything follows that logarithmic curve.

Right.

Fechner's law basically says that sensation always grows slower than the physical stimulus.

Stevens found that for some things like electric shock, it's the complete opposite.

The text mentions the exponent in his power law for shock is 3 .5.

What does that mean in practical terms?

It means sensation grows faster than the stimulus.

Much, much faster.

If I double the physical current of an electric shock, your experience of pain doesn't just double.

It goes up by a factor of more than 10.

It's an exponential explosion of sensation.

That sounds absolutely horrible.

It is.

But think about the evolutionary reason for it.

If you are touching something that is damaging your tissue, nature doesn't want you to have a nice compressed logarithmic experience.

It wants you to have an exaggerated experience.

It wants you to recoil immediately.

So pain needs to explode so you pay attention and get away from the danger.

Exactly.

Whereas for something like brightness, a compressed scale is better, so you're not blinded by the sun.

The math reflects the biological utility.

That makes total sense.

Now let's talk about taste.

Because there is a demonstration here involving something called cross -modality matching that reveals that we are, quite literally, living in different sensory worlds.

This is the PROP tasting experiment, and it is fascinating.

So cross -modality matching is when I ask you to match the intensity of one sense to the intensity of another.

For instance, adjust this light until it is as bright as the sound is loud.

Which sounds incredibly abstract, but the weird thing is that people are surprisingly consistent with it.

Until you introduce this chemical, PROP.

Propylthrosyl.

It's a bitter chemical.

But here's the thing.

For some people, it's almost tasteless.

For others, it is one of the most repulsive things they've ever tasted.

So they ask people to do this cross -modality matching with the bitterness of PROP.

Exactly.

So non -tasters, the people who can barely taste it, when asked to match the bitterness to another sensation, they match it to a whisper or the tick of a watch.

Yeah.

It's barely there for them.

Okay.

What about medium -tasters?

They match it to something like the smell of frying bacon, a noticeable, moderate sensation, but then you have the super -tasters.

And these are the people who have the highest density of taste buds on their tongue.

And when they're asked to match the bitterness of PROP, they match it to the brightness of looking directly at the sun or the most intense pain they have ever felt.

The brightness of the sun.

I mean, that is not just a preference.

That's not just being a picky eater.

That is a completely different reality.

It absolutely is.

It means that when a super -taster eats broccoli or drinks black coffee, they're experiencing a sensory assault that a non -taster can't even begin to imagine.

It changes how we think about diet, about food culture.

It's all rooted in this fundamental biological difference.

We don't all live in the same world.

Okay.

So we've measured thresholds.

We've measured the magnitude of sensation, but then the chapter pivots to something that feels very modern, very different.

Signal Detection Theory.

STT.

Yeah, this is where perception stops being just about biology and starts being about, well, about gambling.

Camping.

It's about making decisions under uncertainty.

In the real world, you almost never get a pure, clean signal.

There is always noise.

And noise in this context isn't just sound, right?

It can be anything.

Anything that confuses the signal.

The textbook uses the example of a radiologist looking at a mammogram, which is a perfect case.

The signal you're looking for is a tumor.

The noise is all the healthy tissue, the shadows, the grain of the image itself.

And the radiologist has to make a decision.

Is that little white spot cancer, which is signal plus noise, or is it just a bit of dense tissue, which is noise alone?

And because those two things can look very, very similar, the distributions overlap, they can never be 100 % sure.

They have to make a bet.

The text also uses the shower ringtone example, which is way more relatable for me.

Let's walk through it.

It's a great example.

You're in the shower.

The water's blasting.

That is the noise.

You're waiting for a super important phone call.

That is the signal you're trying to detect.

And suddenly through the sound of the water, you think you hear a ring.

But did you?

The sound of the water splashing against the curtain can create frequencies that sound an awful lot like a phone ring.

So you have a decision to make.

Do you jump out of the shower?

And this is where the idea of the criterion comes in.

It's like this internal line in the sand you draw.

Exactly.

You mentally set a criterion level of evidence.

If the sound is louder or more ring -like than X, I'm going to decide it's the phone.

If it's quieter, I'll decide it's just water.

But I can move that line based on my motivation, right?

So if I'm waiting for a job offer, I am desperate not to miss that call.

So what do you do with your criterion?

You shift it to the left, you become liberal, you lower your standard of evidence, you'll decide it's the phone, even for very faint, ambiguous sounds.

The good thing is I'll catch every single ring.

I'll have a very high hit rate.

The downside.

I'll have a ton of false alarms.

I'll be jumping out of the shower every time a pipe squeaks.

I'll be standing there naked and wet and cold looking at a perfectly silent phone.

Which is annoying.

So let's flip the scenario.

You're trying to relax on a Sunday morning.

You don't want to be bothered by anyone.

So I shift my criterion way to the right.

I become conservative.

I set a really high bar for what I'll accept as a phone call.

The benefit there is you will almost never have a false alarm.

You will enjoy your shower in peace.

Your correct rejection rate will be very high.

I might miss a real call.

And that's the trade off.

You increase correct rejections, but you also increase misses.

SCT's biggest contribution is showing us that there's no such thing as a pure absolute threshold.

Your perception is always a combination of two things.

Your actual sensitivity to the signal and your decision bias or your criterion.

At least everything, doesn't it?

Airport security, looking for weapons, a juror deciding guilt or innocence, even dating.

It's all signal versus noise.

It's an incredibly powerful framework for understanding the world.

All right.

Let's go deeper into the machine.

Let's get to the hardware.

Section five is sensory neuroscience.

Time for some biology.

And we have to start with this foundational concept, the doctrine of specific nerve energies, which was proposed by Johannes Müller in the 1800s.

It sounds like a really dense academic title, but the concept itself is actually kind of spooky when you think about it.

It is.

Müller's big realization was that the brain is locked away in a dark silent vault.

It never sees the world directly.

It never hears the world directly.

All it ever gets are electrical impulses from nerves.

So how on earth does it know the electrical spikes?

Müller said it's all about which nerve is firing.

It's the who, not the what.

If the optic nerve fires, the brain just says light.

If the auditory nerve fires, the brain says sound.

It doesn't matter how you stimulate the nerve.

The text suggests a little demo for this.

It says to gently press on the corner of your closed eye in a dark room.

If you do that, you'll see a little spot of light on the opposite side.

We call it a phosphine.

Now there's no light entering your eye.

You are just poking it with your finger.

But because you are mechanically stimulating the optic nerve, your brain's only way of interpreting that signal is to call it a visual flash.

That's incredible.

So we are basically prisoners of our wiring.

If you could somehow plug my auditory nerd into your visual cortex, I would see Mozart.

Theoretically, yes.

I mean, that's the fundamental principle behind things like cochlear implants for the deaf or retinal prosthetics for the blind.

We are just hacking the doctrine of specific nerve energies.

Speaking of nerves, the book has this great diagram of the cranial nerves.

There are 12 pairs of them coming directly out of the brain.

And the text highlights the big sensory ones for us.

The olfactory nerve, which is cranial nerve for smell,

the optic nerve number two for vision,

and the vestibulocochlear number eight for hearing and balance.

And then there are three whole separate nerves just dedicated to moving the

oculomotor, the trochlear, and the abducens.

It's amazing how much biological hardware is dedicated just to looking at things.

And another one of the giants of the field, Hermann von Helmholtz, he figured out that this hardware has a speed limit.

Right.

He measured the speed of nerve impulses in frogs.

Before him, people thought it was basically instant.

They thought it was as fast as light or electricity in a wire.

He showed that no, nerve signals are actually pretty slow.

They travel at about 30 meters per second, which is roughly 90 feet per second.

Which is fast, but it's not instant, not even close.

No, it means there is a lag,

a measurable delay.

When you stub your toe, the pain signal takes a noticeable fraction of a second to travel all the way up your leg and spinal cord to your brain.

So we are technically living in the past.

We are always living in the very recent past.

By the time you perceive the now, it has already happened and is on.

We are constantly lagging just a little bit behind reality.

That is deeply unsettling.

Let's zoom in even further.

From the whole nerve to the single neuron, the book has these beautiful drawings from Santiago Ramon y Cal.

The Holy Trinity of the Neuron.

So Cajal's beautiful drawings showed us that neurons are individual cells.

They aren't all connected in a big net.

Then Sir Charles Sherrington came along and named the tiny gap between them, the synapse.

And then Otto Lewey discovered that they communicate across that gap using chemicals, neurotransmitters, not just electrical sparks.

Right.

But the signal that travels down the neuron itself, that's electrical.

And that's the action potential.

Okay, this is crucial, but it's also where the technical language can get a bit dry.

Depolarization, ion channels.

Can we use an analogy to break this down?

Let's use the nightclub analogy.

I like this one.

Imagine the neuron is an exclusive nightclub.

Okay, I'm with you.

The neuron is a nightclub.

Outside the club, there's a huge, massive crowd of people desperate to get in.

Those are sodium ions, NEA+.

They're all positively charged and they're super hyped up.

They want in, the party's inside.

But the bouncers, which are the ion channels in the cell membrane, are keeping the doors closed.

Now, inside the club, you have some other people just kind of more negative chill vibe compared to the hyped up positive crowd outside.

Exactly.

That is the neuron's resting potential.

It's about negative 70 millivolts.

But then a signal comes from another neuron, a text message saying the party is on and the bouncers fling open the doors.

And all that sodium from the outside rushes in.

It floods in.

And because all that positive charge rushes in, the vibe of the club instantly spikes.

It goes from negative 70 to positive 40 in a flash.

That is the action potential.

That's the spike.

But you can't leave the doors open forever.

The club will get too crowded.

No, the bouncers panic.

They have to restore the balance.

So what do they do?

They open the back doors and kick the potassium ions out to bring the positive charge back down.

And this whole process, this party and riot happens in how long?

One thousandth of a second,

one millisecond, and then it resets ready for the next party.

And it's the of this firing that codes information.

The text shows these graphs called tuning curves in figure 1 .27.

Right.

A neuron isn't a generalist.

It has a specialty.

An auditory neuron, for example, might be specifically tuned to a frequency of exactly one thousand hertz.

That's its characteristic frequency.

That's like its favorite song.

It's its favorite song.

It requires very little energy, a very quiet sound to get that neuron to fire at that pitch.

It's like a lock that opens really for one specific key.

But if I play a different tone, say two thousand hertz, the neuron might still fire, but you have to shout.

The sound has to be much, much louder to get a reaction.

That U shaped curve on the graph shows us the neurons sweet spot and how it gets less sensitive to frequencies further away from that spot.

OK, we've looked at the single neuron.

Now let's look at the whole brain at once.

Section seven is neuroimaging.

This is the stuff that used to be but is now a reality in every university.

It's the alphabet soup of neuroscience.

EEG, MEG, MRI, FMRI, BTT.

Let's break them down.

If I want to know exactly when something happens in the brain with millisecond precision, what do I use?

You use EEG, electroencephalography.

You put a cap with electrodes on the scalp and it picks up the combined electrical noise of millions of neurons firing.

It is incredibly fast.

It gives you those squiggle line graphs you always see.

There's a downside.

A big one.

It's incredibly blurry.

It has terrible spatial resolution.

It tells you when the brain reacted, but it can't tell you exactly where.

The book uses the analogy of standing outside a football stadium.

You hear the crowd roar the instant a goal is scored, so your timing is perfect.

But you have no idea which fan and which seat started the cheer.

Precisely.

That's EEG.

OK, so if I want to know where, if I want a precise map of the activity, I should use FMRI.

Functional MRI, yes.

This uses a giant magnet to track the flow of oxygenated blood in the brain.

How does blood oxygen tell us about brain activity?

It's called the BLD signal blood oxygen level dependent.

The logic is simple.

Active neurons are hungry.

They burn fuel.

So if a part of your brain starts working hard on a task, the body's plumbing system rushes more oxygen rich blood to that specific spot.

So FMRI is really just tracking the plumbing, not the neurons directly.

Exactly.

It's a blood flow map, which is an indirect measure of activity.

The spatial resolution is amazing.

We can see these tiny specific areas lighting up, but blood flow is slow.

It takes a few seconds for that rush of blood to arrive.

So FMRI has terrible timing.

It has a significant lag.

So you have this fundamental trade -off.

EEG is fast but blurry.

FMRI is sharp but slow.

We use different tools for different research questions.

It's still incredible that we can do this at all, but the chapter ends with a final pivot.

We've been studying the biological brain.

Now we're trying to build an artificial one, computational models.

The mind is what the brain does.

If that statement is true, then we should in theory be able to write computer code that does the same thing.

And one of the first ideas here is efficient coding.

This is a really elegant idea that helps explain why our brains are built the way they are.

Just look at the world around you.

It isn't random static like a TV screen with no signal.

It has structure.

It has redundancy.

Figure 1 .33 uses the example of pixels in a photograph.

Right.

If I take a photo of a green forest and I look at one pixel and it's green, what is the probability that the pixel right next to it is also green?

Very, very high.

Exactly.

Nature is predictable and redundant.

Efficient coding theory says the brain shouldn't waste precious energy analyzing and reporting every single green pixel.

It should only focus on the changes, the edges, the surprises, the unpredictable information.

So we are wired to ignore the predictable to save energy.

Which leads us directly to Bayesian models.

This is the very popular idea of the Bayesian brain.

It says the perception isn't just a passive process of taking in data from the senses.

It's an active process of using priors.

And priors are our past experiences and expectations.

Yes.

Your brain is a gambler.

It is constantly calculating the probability of what it's seeing based on everything it has seen before.

If you see a dark, tall shape in the corner of your bedroom at night, your sensory data from your eyes is weak and ambiguous.

Right.

It's just a blob.

But your prior knowledge tells you it's usually the coat rack that I left there.

So you perceive a coat rack even with bad data.

But if I just watched a horror movie, my prior might change to it's a monster.

Exactly.

And you might actually perceive it as a monster for a split second.

Perception is a combination of the current sensory evidence and the probability from your past experience.

And finally, this brings us to deep learning.

Artificial neural networks.

These are the models that are powering the AI revolution happening right now.

And this brings us completely full circle.

These networks are built with layers of artificial nodes and weighted connections,

explicitly mimicking neurons and synapses.

And they're getting incredibly good at perception.

They can recognize faces, drive cars, diagnose tumors better than some doctors.

But the text mentions a huge problem.

The black box problem.

It is the great humbling irony of our time.

We built these deep neural networks to help us understand how perception works.

But because they have millions or even billions of connections and they learn on their own from massive amounts of data, we often have no idea how they're doing what they do.

Wait, we built them, but we don't know how they work.

We know the code.

We know the math.

We know the architecture.

But we often don't know what specific features the AI is actually using to make its decision.

When it identifies a cat, is it looking at the pointy ears, the texture of the fur,

the specific curve of the tail, or some combination of pixels that a human would never even notice?

It's opaque.

So we started this whole journey by trying to understand the black box of the human brain.

We decided to build a computer brain to help us figure it out.

And now we have a second black box that we also don't understand.

We have successfully replicated the mystery.

That is a wild thought to end on.

From a marble statue smelling a rose in the 1700s, to a supercomputer hallucinating a cat in the 21st century.

The fundamental question hasn't changed.

How do we know what is real?

It's a puzzle we're still solving.

And this is just chapter one.

We have really only just unpacked the toolbox.

That's right.

Next time, we're going to start using these tools to look at the specific senses one by one.

Thank you for listening to this deep dive into sensation and perception.

This is the Last Minute Lecture Team signing off.

Stay curious.