Starting Your Research

1:04is from cognitive psychologist Daniel Simon's famous invisible gorilla experiment. And you know, about half of the people who watched that video for the very first time, they just never see the gorilla. Their brains completely edited out. Well, welcome to the deep dive, everyone. Today, we are exploring the invisible gorillas hiding in how we understand the world. We are pulling apart chapter four of the textbook research methods from theory to practice.

1:29And while the text is, it's this foundational guide for new researchers, what it really offers is this incredible masterclass in how human beings go from just like noticing weird things in the world to actually proving how reality works. Right. We're looking at the hidden architecture of scientific discovery. Yeah. And as a starting point, I think we really need to look at the big picture they present early in the chapter, like that visual flow chart.

1:53Oh, yeah. The overarching theme here is that science isn't just a collection of facts, you know, it is a highly structured, interconnected loop. A loop, yeah. Because we often think of research as a straight line, like you have an idea, you test it, you get an answer, boom, done. But the reality is far more cyclical. You start with a question, you dive into what has already been written, you generate a testable hypothesis, you carefully choose a method to measure it, you get your ethical approval from an institutional review board or an IRB to make sure you aren't violating anyone's rights, and then you collect the data, analyze it, and finally publish.

2:29And the second you publish, another researcher reads your paper, finds a flaw or a new angle, and the entire loop just fires up all over again. Exactly. But, I mean, looking at that flow chart in the book, it can be incredibly intimidating for a novice. It's basically like staring at the blueprints of a skyscraper when you've only ever built a Lego house. That is a perfect way to put it.

2:50Getting into that loop seems daunting. Yeah, because if I want to research, say, human memory, I'm looking at literally a century of accumulated science, how do you even find a starting point without drowning in millions of PDFs? Well, the trap most people fall into is what the text calls brute force searching. Oh, like just typing stuff in. Right. You go to Google Scholar or something, you type in human memory, and you are instantly hit with three million results.

3:17And you just end up aimlessly scrolling, hoping to, like, stumble across something useful. It is wildly inefficient. It's like walking into a massive library blindfolded and just grabbing books off the shelf at random. Yes. The smarter approach is to step back and look at the ecosystem of the information. For instance, if you want to understand the broad trends of a field, you don't start with hyper -specific granular studies.

3:42You look for comprehensive reviews. Like getting the lay of the land first. Exactly. Journals like the Annual Review of Psychology or Current Directions in Psychology, they do the heavy lifting for you by summarizing the major developments over the last year or decade. So once you have that broad view, then you can use specialized databases, right? Yeah. Like Psycho, NFO for psychology. Right. But even then, the real magic isn't just typing in keywords.

4:07The database has advanced features. You can filter by peer -reviewed status or filter for empirical studies versus meta -analyses, which are statistical summaries of multiple studies, or even filter by specific age groups. Wow. So you can get super targeted. You can. And then you use a strategy called reference chaining. Reference chaining. Yeah. If you find one absolutely perfect foundational paper on your topic, you don't need to keep guessing keywords.

4:34You can just look backward at their bibliography to see the intellectual DNA of their idea, like who they cited. That makes so much sense. It's like following a breadcrumb trail left by the experts themselves. And furthermore, you can look forward. Databases now allow you to do a forward search, which shows you every single paper that has cited your perfect article since the day it was published. Oh, wow.

4:56So you aren't just searching for topics. You are mapping the lineage of an idea across time. Okay, so reference chaining gets you the right papers. But here is the next hurdle. Yeah. Right? Once you pull that thread and gather a stack of relevant research papers, you have to actually consume them. And reading a scientific paper front to back like a mystery novel is a terrible idea. Yeah, the chapter actually warns that reading a research paper is an entirely different skill than reading a regular book.

5:23It requires a skill called non -linear reading. You are essentially dismantling the paper to extract the architecture of the study. You skip around. Right. Like most readers start with the abstract, which is that 150 to 300 words summary at the top. Right. And it's great for screening to see if the paper is worth your time, but it really lacks the nuance needed to truly understand the work.

5:44So from there, you skim the introduction or literature review, which sets up the why and ends with the researcher's specific hypotheses. Then you have the method section. This is the how. It's divided into participants, materials, and procedures. And there is a critical concept here. Variables must be operationalized. Operationalized, meaning what exactly? It means translating a vague concept like happiness into a strictly measurable term, like the number of times a subject smiles per minute.

6:16Oh, okay. Making it concrete. Yes. Operationalization is the bedrock of the scientific method because it allows for replication. If you don't explicitly define how you measured a variable, no other scientist can recreate your exact experiment to see if your findings hold up. Because if the system can't replicate, the system breaks down. Precisely. Okay. So after the method comes the results section, which is just this wall of raw statistics and math.

6:44And then finally, you get the discussion section where the researchers interpret their findings, talk about limitations, future directions, all that. Right. Now, I have to push back here because the chapter says not to skip the results section. But if I'm reading a dense paper and the math in the results section looks like alien hieroglyphics, why shouldn't I just skip straight to the discussion? Like, why not just read the author's plain English summary of what they found?

7:06That is a super common question. But you shouldn't skip it because the discussion section is essentially the spin room of science. The spin room. Yeah. The results section gives you the cold, unfiltered mathematical facts. The discussion section is where the authors apply their own interpretive lens to those facts. Oh, so they get to shape the narrative. Right. Let's say a drug improves memory by a statistically significant but practically tiny margin, say, 2%.

7:34Okay. The results section just says 2%. But the discussion section might frame that as a promising groundbreaking trend in cognitive enhancement. Reading the raw results allows you to evaluate the reality of the data before you are influencing by the spin the researchers want to put on it. You want the raw facts, not the PR campaign for the facts. That makes total sense. But, you know, spending all your time deconstructing literature in a library doesn't actually produce new ideas on its own.

8:02Which brings us back to our invisible gorilla. Oh, right. The text emphasizes that incredible research questions often start with just actively observing the real world and getting your hands dirty in the lab. Yeah, Daniel Simons didn't discover inattentional blindness by just reading old papers. He observed how human attention actually functions in real time. Inattentional blindness is this profound error of perception, right? Right. Where we fail to see a completely unexpected object simply because our attention is entirely focused on something else, like counting basketball passes.

8:37Right. And what's fascinating about Simon's work is that when it caught fire in the mainstream media, he didn't just step back. He actively worked with reporters. He guided them. Yeah. He wanted to make sure the media didn't exaggerate the findings into some sweeping claim like human eyes are completely unreliable, you know. He ensured the public understood the specific mechanism of attention. The text actually gives this great practical advice for students, too, to get involved in ongoing projects like the Summer Research Opportunity Program.

9:06There's this really enthusiastic quote from author Martha Arterberry about the thrill of research. She says, it's a day to day. Which is such a great mindset to have. It is. But it brings up a bridging question for me. So our everyday common sense observations can spark a brilliant study like the invisible gorilla, but can our common sense also lead us completely astray? Oh, absolutely. And that leads right to the heart of the chapter.

9:32There is a sharp distinction between the naive theories we hold in our heads and the formal scientific theories that should actually guide research. Okay. Naive versus formal. Naive theories are the implicit, everyday beliefs we all carry around about how the world works. Formal scientific theories are strictly grounded in empirical, tested data. Can you give an example of a naive theory causing problems? Sure. Look at the work of psychologist Carol Dweck on how people view intelligence.

10:00Many of us walk around with an entity theory of intelligence. This is the naive belief that intelligence is a fixed, stable trait. You're either born smart or you aren't. Right. You either have it or you don't. Exactly. But conversely, an incremental theory assumes intelligence is malleable and grows through struggle and effort. So if you hold the naive entity theory, failing a test is devastating because it means you fundamentally lack intelligence.

10:25Right. But if you hold the incremental theory, failing just means you need to study harder. Precisely. And these naive theories don't just affect individuals, they shape entire cultures. The text dives into this incredible study by Leslie and colleagues regarding the massive gender imbalances across different academic fields. Oh, right. Yeah. They surveyed academics across disciplines and found a clear pattern. In fields where the prevailing culture holds the naive theory, that success requires innate, unteachable raw brilliance.

10:56Like disciplines like physics, math, and philosophy. Fields of prized genius. Right. In those fields, women make up only about 30 % or less of the PhDs. Wow. And the text says that's driven by societal stereotypes about brilliance, right? Societal stereotypes incorrectly assume that this elusive raw brilliance is a male trait. But when you look at fields like psychology or education, where success is stereotypically viewed as the result of hard work, empathy, and incremental growth rather than innate genius.

11:23What happens to the numbers there? The percentage of female PhDs skyrockets to around 70%. So the naive theories we hold about what it takes to succeed actively build the demographic realities of our institutions. Which is exactly why researchers must rely on formal tested theories when building a study rather than their own common sense assumptions. Yes. And the text actually breaks formal theory down into two architectural layers.

11:50You have framework theories and specific theories. Let's try an analogy here. Okay. Let's hear it. A framework theory is like deciding on the broad architectural style of a house, say mid -century modern. It dictates the overall aesthetic and rules. While a specific theory is the exact detailed blueprint for the kitchen plumbing within that house. That maps perfectly. A framework theory gives you a broad perspective. For example, developmental psychopathology is a framework.

12:16Or say evolutionary psychology, which assumes behaviors exist because they helped our ancestors survive. A specific theory within that framework might predict exactly how humans react to the sound of a rustling bush in the dark. Or how expressed emotion impacts schizophrenia. The overarching architectural style fundamentally changes the exact questions you ask. Right. The book contrasts clinical psychology with positive psychology to show this. Clinical psychology operates on a framework focused on treating psychological distress and pathology.

12:47Right. While positive psychology operates on a framework focused on human strengths and thriving. They can look at the exact same human behaviors, but they ask fundamentally different questions. One asks, what is causing your depression? The other asks, what coping mechanisms can we build to buffer you against depression? The theory acts as a lens. And that lens narrows the scope of inquiry and can lead to completely counterintuitive discoveries.

13:12Counterintuitive how? Well, take the classic debate of nature versus nurture. McLean G. conducts a study using the theory of domain -specific expertise. The common sense, naive assumption is that adults simply have better, more mature brains than children. So an adult will always beat a child in a memory test. Naturally. But she tested child chess experts against college -age novices. Wait, I remember this. She had them look at a chessboard, took it away, and asked them to recreate the positions of the pieces from memory.

13:40And the kids completely destroyed the college students. Yes. Because the theory of domain -specific expertise suggests that deep knowledge in one specific area overrides general cognitive maturation. Oh, wow. And when she switched the test to a traditional generic number recall task, the adults won. The theory allowed her to isolate and prove that expertise isn't just about getting older, it's about the deep structuring of specific knowledge. That is so cool.

14:08Theories really shape the hypotheses we make. And Leon Festinger's theory of cognitive dissonance is a perfect example of a theory shaping a really counterintuitive hypothesis. Oh, Festinger, yes. The theory states that human beings experience profound mental discomfort when we hold two conflicting thoughts, or when our beliefs clash with our actions. We hate that friction so much that we will go to absurd lengths to resolve it. Right.

14:32So Festinger set up an experiment where participants had to complete a mind -numbingly boring, repetitive task for an hour? Sounds awful. It was. Afterward, he asked them to lie to the next participant in the waiting room and tell them the task was actually incredibly fun. And here's the twist. He paid half the participants a large sum of money to lie and the other half a very tiny amount.

14:58So common sense dictates that the people paid more would rate the task more favorably because they were rewarded, right? Exactly. But the theory of cognitive dissonance predicted the exact opposite. Wait, really? Yeah. The people who were paid a lot had an easy internal justification. The task was awful, but I got paid well, so who cares? No mental friction. Right. They justified the wasted time with the money.

15:20But the people paid almost nothing, experienced massive dissonance. The task was awful, I just lied to someone about it, and I didn't even get paid well. I must be a fool. So to resolve that mental pain, the brains actually tricked them into believing the task wasn't that bad. When surveyed later, the underpaid group rated the boring task as significantly more enjoyable and important. That is wild. But I have to ask though, regarding methodology, if cognitive dissonance is all about reducing mental discomfort, does the method used to test it have to be specifically designed to annoy people?

15:55Well, it doesn't have to be annoying, but it absolutely must generate psychological friction. You have to trap the participant's mind in a contradiction. And this highlights a massive point in the text, the intimate, unbreakable link between method and analysis. You cannot separate what you're studying from how you measure it. Let's look at how theory dictates methodology with babies. Oh, I love the baby studies. You want to study infant cognition, but babies can't talk, they can't fill out surveys, they can't use a keyboard.

16:23How do you know what's happening in their heads? Researchers use the habituation or preferential -looking paradigm. Basically measuring how long a baby stares at something. Exactly. Researcher Elizabeth Spelk used this to test if babies understand the laws of physics. She would show a baby a normal event, like a ball dropping onto a solid floor. She'd do it over and over until the baby got bored and looked away.

16:44That's habituation. Okay. Then she introduces an impossible event. Using a trap door or mirrors, it looks like the ball drops completely through the solid floor. And the baby stares significantly longer at the impossible event because they know it's weird. Right. And another researcher, Karen Nguyen, did this with math. She'd place a toy behind a screen, then visibly place a second toy behind the screen. When she drops the screen, if there are two toys, the baby barely looks.

17:13But if she secretly removed a toy, leaving only one. The baby stares intently because the math is wrong. One plus one does not equal one. Exactly. But think about the underlying theory making this method possible. The entire preferential -looking methodology rests on the theoretical assumption that babies are born smart, with an expectation of how the physical world works. Right. If a researcher held the naive theory that babies are just blank slates, they would never think to use a stopwatch to measure a baby's stare as a proxy for intelligence.

17:45The theory builds the method. And we see the exact same architecture in behavioral genetics with twin studies. Researchers compare monozygotic twins, identical twins, who share 100 % of their DNA with dizygotic fraternal twins, who share about 50%. So if identical twins share a behavioral trait more often than fraternal twins, it points to a genetic link. Yes. But again, the method of comparing twins is entirely built on the theoretical assumption that traits are genetically determined.

18:13Okay, so if we follow this chain reaction. Theory dictates the method, the method generates the data, and then the data dictates the statistical analysis. Exactly. Moving from method to analysis, the text brings up the t -test, which is a statistical tool used to compare the differences between the means and variances of two different groups. Like comparing the average -looking time of babies who saw the impossible event versus a control group.

18:37Yes. And when you run these statistics, you inevitably collide with figure 4 .1 in the book. The normal distribution, the famous bell curve. Visually, imagine a bell, right? The huge, swollen little of the bell represents the average. The vast majority of data falls right around the mean. Right. The percentages the book gives are 34 .1 % just on either side of the mean. Then moving outward, 13 .6%, then 2 .1%, and finally just 0 .1 % in the extreme tails.

19:10Those long, razor -thin tails represent the highly unusual outcomes. And understanding the shape of that curve is how researchers decide between a one -tailed or a two -tailed hypothesis test. Right. Imagine you test a drug that you believe increases intelligence. A one -tailed test is, well, it's statistically arrogant. It assumes your drug will only push scores higher into that top tail of the bell curve. It presumes the direction of the outcome.

19:34Yes. A two -tailed test is more conservative and intellectually humble. It doesn't presume the direction. It splits the statistical probability to account for the fact that your drug might make people smarter or it might actually make them dumber, pushing their scores into the bottom tail. Oh, I see. Now, there is a debate the book mentions about a theoretical analysis or data mining. Some people argue we shouldn't rely on region theories to dictate our analysis at all.

20:01Oh, the data miners. They use powerful computers to scan massive oceans of data looking for any correlations, arguing that this is how serendipitous discoveries happen. They point to Ivan Pavlov's dogs as a great serendipitous discovery. Yes, but the critics of data mining are fierce. The textbook highlights Paul Meehl's critique. He warned about the crud factor. The crud factor. I love that name. Meehl argued that everything correlates with everything else to some tiny degree simply by random chance.

20:29To prove this, in 1966, Meehl analyzed a massive data set of 57 ,000 children from four different Lutheran synods. 57 ,000. Yeah. And he found statistically sound, mathematically perfect differences. Like what? Well, the data proved that Missouri Synod children were more likely to play a musical instrument, and Wisconsin Synod children were less likely to have siblings. That is hilarious. The math was flawless, but the findings were completely meaningless.

20:57There is no theological doctrine driving Lutherans to play the clarinet. Exactly. So isn't data mining essentially just a giant fishing expedition? Like if you throw a big enough net into a lake of data, eventually you'll catch an old boot and call it a fish. That is the exact hazard of the crud factor. Without a guiding theory to explain why two things should be connected, you can't tell a profound discovery from a random illusion.

21:22However, having a rigid theory to avoid fishing expeditions comes with its own major risk. Which is? Confirmation bias. Becoming totally blind to data that proves you wrong. Ah, confirmation bias. Eagerly accepting anything that supports your theory and ignoring anything that contradicts it. The authors illustrate this beautifully with two cartoons. Figure 4 .2 is a cartoon by Tim O 'Brien. It shows a researcher standing in a blown up smoking lab, holding a flask.

21:51He looks at the disaster and confidently says, I totally meant to do that. He sees what he wants to see. Right. And figure 4 .3 is from The New Yorker. It shows medieval knights in full armor playing baseball. One knight says to the other, let's walk him and pitch to the bishop. Because their pre -existing medieval theory completely alters how they interpret the raw data of the batter.

22:11Exactly. Our theories alter our perception of reality. Which brings us to the chapter's final advice on avoiding common beginner problems. Okay, what's the advice? First, don't try to win a Nobel Prize on your first study. Science is about extending existing research. Second, avoid purely incremental research. Like, just changing the age group in an old study without a theoretical reason why it should matter. And lastly. And most importantly, don't try to measure too many variables at once.

22:37It leads to uninterpretable results. Keep it simple. It really feels like the entire chapter is essentially telling students to walk a tightrope. Like be original, but not too original. Let theory guide you, but don't let it blind you. It is a continuous balance. Well, let's tie it all together for the listener. We've navigated from the daunting task of picking a topic. Using targeted searches and reference chaining rather than brute force.

23:04We've decoded the art of nonlinear reading, realizing you shouldn't skip the results section. Right. We looked at the invisible gorillas of observation, and how our naive theories, like raw brilliance, can actively damage fields. And finally, we explored the inseparable deep connections between theory, methodology, and statistical analysis. It's all connected. So, we'll leave you with this provocative thought to explore on your own. If our theories shape what we see, and our methodological tools shape our theories.

23:33What invisible gorillas are currently walking right through the middle of our most accepted scientific fields, completely unnoticed, simply because we haven't built the right theory to look for them yet? That is a great question to end on. Thank you so much for joining us on this deep dive. On behalf of the Last Minute Lecture team, keep questioning the framework, and we will catch you next time.