Chapter 8: Quantitative Methods

Welcome to Last Minute Lecture.

This free chapter overview is designed to help students review and understand key concepts.

These summaries supplement not replace the original textbook and may not be redistributed or resold.

For complete coverage, always consult the official text.

Imagine a researcher, right?

And they wanna prove that playing violent video games causes kids to become aggressive.

Okay, classic example.

Right, so they go to a local playground, they observe the kids, tally up the data, and publish a study that makes national headlines.

The media completely runs with it.

Video games cause violence.

Oh, I've seen that headline a million times.

Exactly, but behind the scenes, their methodology was secretly flawed, they used the wrong statistical test, and their core assumptions were entirely backward.

Basically rendering the whole study totally meaningless.

Welcome to the deep dive.

Today, we are opening up the hood on quantitative methods.

And hey, if you are a college student staring down a massive research method syllabus right now, consider this your supportive last minute tutoring session.

We got your back.

We really do.

Our mission today is to break down the mechanics of quantitative design, specifically drawing from chapter eight of Creswell and Gettermann's research design.

We want you to actually understand how good research is built.

And how to spot when it completely falls apart.

So we're gonna get through it together, step by step, without getting buried in the jargon.

Because honestly, the foundational concept here starts with a word that sounds incredibly intimidating.

Determinism.

Oh yeah, that's a heavy one.

Right, it's this post -positivist philosophical assumption.

But when I was looking through the chapter, I realized,

I mean, what are we actually talking about here?

It sounds like we're saying free will doesn't exist.

Yeah, it does sound like a massive philosophical crisis.

But its application in research is actually really practical.

Determinism, at least in this context, is just the belief that certain factors determine or heavily influence other factors.

Okay, so it's not about destiny.

No, no.

Quantitative research is just built on this premise.

You're looking for cause and effect.

Or at the very least, predictable associations.

You're examining the relationships between variables.

So going back to that video game example, if we operate under determinism, we're basically saying that the variable of playing the game influences the variable of playground aggression.

Precisely.

But how you investigate that influence,

that changes everything.

If you just wanna see if the two things are linked, meaning kids who play more games tend to get into more fights, you use a correlational design.

Like surveys.

Like a survey, yeah.

But if you want to definitively prove that the games actually cause the aggression, you have to cross that line from correlation to causation.

And to do that, you have to run a true experiment.

Got it.

And that distinction between a survey and an experiment seems to shape the entire blueprint of a study.

It's the core of it, yeah.

Because looking at the methodology roadmap in the chapter, it seems like whether you're doing a survey or an experiment, your method section always needs four primary elements.

Let's hear them.

First, who are you studying?

Second, how will you formally test your hypotheses?

Third, what instruments will you use?

And fourth, how will you analyze the data?

That is the architectural blueprint of quantitative research right there.

So if we follow that blueprint, we usually start with the most common approach to gathering data when we're looking for relationships, the survey method.

Surveys basically provide quantitative descriptions of trends, attitudes, or test associations in a specific population.

And they generally answer three types of questions.

Yeah, what did you find in the chapter?

First, descriptive questions.

What percentage of a company's employees support hiring women of color in leadership?

Perfect example.

Second, questions about relationships.

So is there an association between the number of diverse executives and overall company satisfaction?

And third,

predictive questions, which are usually in a longitudinal study.

Does hiring those executives this year predict greater satisfaction three years from now?

Exactly, those are the distinct goals of a survey.

But I have to ask, if experiments are the gold standard because they can actually prove cause and effect, why do researchers even bother with surveys?

Why not just experiment on everything to get definitive answers?

I mean, it comes down to two massive constraints in the real world, practicality and ethics.

Ah, ethics, always getting in the way of mad science.

Right, consider a study on nurse burnout, which the chapter brings up.

Let's say you wanna know if working, grueling, 14 -hour overtime shifts causes emergency room nurses to experience severe burnout.

Okay.

To run a true experiment,

you would have to randomly assign a group of nurses to work dangerous, exhausting hours just to see if they emotionally and physically break down.

Oh, wow, yeah.

An institutional review board, the ARB, would immediately reject that.

You can't ethically torture people for data.

Exactly, you cannot ethically manipulate that independent variable.

So instead, you use a survey design.

You survey nurses in the real world to see if the number of overtime hours they're naturally working predicts higher burnout symptoms.

That makes a lot of sense.

And you can collect that data at one point in time, which is called a cross -sectional study.

Or you can survey them multiple times over a year, which is a longitudinal study.

And there's actually a great checklist for this in the text.

Table 8 .1 acts as the ultimate survey study checklist for stating your purpose, your rationale, and figuring out that cross -sectional versus longitudinal data collection.

It's a lifesaver for students writing proposals.

For sure, but a survey is only as good as the people taking it.

You can write the most brilliant questions in the world, but if you ask the wrong people, your data is useless.

Which brings us to sampling.

Yes, population and sampling strategy is where a lot of studies just live or die.

A study population is the giant overarching group you wanna make inferences about.

So for example, all the emergency room nurses in the entire country.

But obviously, you can't email every single one of them.

Right, so you pull a study sample, which is the realistic subgroup you actually survey.

And getting that sample requires a specific mechanism.

Like, if you magically had a master list of every nurse in the country, you could use a single -stage sampling procedure and just randomly pick names.

Exactly.

But since that list doesn't exist, you use multi -stage sampling or clustering.

How would you visualize that clustering process for our listeners?

It's basically like zooming in on a map.

You can't get a list of every nurse, but you can get a list of hospitals.

So your first stage is randomly picking, say, 50 hospitals.

Then your second stage is getting the staff list for just those 50 hospitals and randomly picking nurses from there.

You cluster them to make the sampling mathematically possible.

That's spot on.

And the goal is to get a truly random sample where everyone has an equal chance of being picked.

But sometimes pure randomness just isn't enough, which is why researchers use population stratification.

Oh, I love the analogy for this.

The best way I can think to explain stratification is like baking a mathematically perfect cake.

Okay, a cake.

I'm listening.

So imagine you're making a cake for a crowd and you know the population eating it consists of 60 % chocolate lovers and 40 % vanilla lovers.

Right.

If you just randomly grab ingredients from your pantry blindfolded, you might end up baking a 100 % strawberry cake.

Total disaster.

Absolute disaster.

So stratification means you deliberately sort your ingredients into two separate buckets first.

One for chocolate, one for vanilla.

And you measure out exactly 60 % from the chocolate bucket and 40 % from the vanilla bucket.

It ensures your sample cake perfectly mirrors the specific demographic ratios of your broader population.

That is an excellent analogy.

Stratification guarantees proportional representation.

But once you have your stratified buckets, how many people do you actually need to pull?

Historically, some researchers would literally just guess or say,

ah, let's survey 10 % of the hospital.

Which sounds like a terrible idea if you're trying to do rigorous science.

It is.

Today, you don't guess.

You use a power analysis.

Okay, what is that?

Before you collect any data, you use specialized software like a free program called G -Power to mathematically calculate the exact sample size you need.

Oh, wow.

Yeah, the software looks at the statistical test you plan to use and the size of the relationship you expect to find.

And it tells you, hey, you need exactly 214 people to detect a meaningful effect.

So no guessing.

None.

If you survey fewer people than the power analysis demands,

your study might be too mathematically weak to find a relationship, even if one actually exists in the real world.

Okay, so we've used our software.

We know exactly how many people we need and we've stratified our sample.

Now we need the actual questionnaire.

And a survey is considered an instrument, right?

Yes, absolutely.

So it has to be calibrated just like a physical tool.

It has to pass two big tests, validity and reliability.

Let's break down the mechanics of those two because students get them confused all the time.

Please do.

Validity is fundamentally asking,

does this instrument measure the thing it claims to measure?

Okay.

If you designed a survey to measure, let's say, leadership potential, construct validity asks if your questions truly capture that abstract concept or if you're accidentally just measuring extraversion.

Oh, interesting.

You also look for a concurrent or criterion validity, meaning do the scores on your new survey strongly associate with existing gold standard measures of leadership?

So validity is, are we measuring the right thing?

Reliability then is, are we measuring it consistently?

Like if I take your survey today and then take it again in a month, my scores shouldn't wildly fluctuate if my life hasn't changed.

That's retest reliability.

You've got it.

And even more critically, there is internal consistency.

This is usually measured using a statistic called Cronbach's alpha, which ranges from zero to one.

An optimal score is between 0 .7 and 0 .9.

Wait, why that specific range?

What is Cronbach's alpha actually doing behind the scenes?

Think of it as checking if all the questions on your survey are pulling in the exact same direction.

Okay.

If you have 10 questions designed to measure workplace stress,

a person who is highly stressed should answer strongly agree to most of them.

Right, that makes sense.

If their answers are completely erratic, like they agree they feel overwhelmed, but disagree that they feel pressure, the questions are probably confusing or poorly written.

A Cronbach's alpha of 0 .8 tells you that the items are mathematically hanging together and consistently measuring the same underlying vibe.

Which is exactly why researchers do pilot testing.

You don't just write a survey and immediately blast it out to 5 ,000 people.

No, please don't do that.

You field test it with a small group first to catch those confusing questions, see if the survey takes too long and causes participant fatigue, and ensure that Cronbach's alpha is actually solid.

And once the instrument is perfectly calibrated, you have to actually get people to fill it out.

Yeah, that's the hard part.

Yes, this is where a lot of research fails.

People just ignore the email.

To combat this, researchers use a multi -phase administration process, most famously the Dillman method.

The Dillman method is fascinating because it's essentially a psychological campaign.

It's not just one email.

No, it's very strategic.

You start with an advance notice letter, like hey, an important survey is coming.

A week later, you send the actual survey, then a postcard follow -up thanking them or reminding them.

And finally, a personalized letter with a replacement questionnaire for the holdouts.

It builds a sense of obligation and importance, which dramatically boosts the response rate.

It's highly effective.

But once that campaign concludes, you're left with a mountain of raw data, thousands of checked boxes.

The next conceptual leap is transforming those check marks into actual variables you can analyze.

This mapping process is really the bridge between collecting data and analyzing it.

Table 8 .2 in the book lays this out beautifully.

So imagine you're trying to see if a graduate student's early success predicts their future career.

Your predictor variable might be the number of publications they had in grad school.

Your outcome variable is the number of research grants they secure in their first seven years as a faculty member.

But you can't just look at those two variables in isolation.

Right, because you have to account for a control variable, like severe life stress.

Exactly.

A highly stressed person might get fewer grants regardless of how many papers they published in grad school.

So you map each of these conceptual variables directly to specific questions on your survey.

And once everything is mapped, you import that data into statistical software like R, SPSS, or Excel.

But before you run any exciting formulas, you have to check for response bias.

Uh oh.

Yeah, this is the terrifying possibility that the people who didn't respond to your survey are fundamentally different from the people who did, which would mean your results are entirely skewed.

But wait, how do you analyze the answers of people who didn't give you any answers?

That seems basically impossible.

It's a clever workaround called wave analysis.

You look at the people who responded in the very final weeks of your Dillman campaign, the ones who only answered after that fourth personalized letter.

Oh, I see where this is going.

The methodological logic here is that these extremely late responders are behaviorally almost identical to the non -respondents.

They just finally caved in.

Oh, wow.

If the answers of the late responders are wildly different from the people who answered on day one, you likely have a massive response bias problem.

That is such a smart way to audit the data.

So assuming your waves look good, you move on to descriptive statistics, calculating the means, the standard deviations, and figuring out how to handle missing data where someone skipped a question.

And then the main event,

inferential statistics.

This is where we determine if the relationships we see in the data actually mean anything.

I'm looking at table 8 .3, the list of inferential tests, t -tests, ANAVAs, Pearson correlations.

And honestly, to a student, this just looks like alphabet soup.

How do you know which one to actually use?

Think of it like a toolbox.

You don't use a hammer on a screw.

You choose the test based entirely on the shape of your variables.

Okay, let's walk through the mechanisms.

Sure.

If you are comparing two distinct independent groups like part -time nurses versus full -time nurses to see who has a higher average burnout score, use a t -test.

Two groups, t -test, got it.

The t -test mathematically compares the means of those two specific groups against the spread of their data.

But if you have three or more groups, say, day shift, night shift, and rotating shift nurses, a t -test breaks down.

So what's the tool for that?

You have to use ANOVA, or Analysis of Variance, which analyzes how the variance within each group compares to the variance between the groups.

Okay, so t -test is for two groups.

ANOVA is for three or more.

But what about that control variable we mentioned earlier, like life stress?

That requires an ANCOVA analysis of covariance.

Think of ANCOVA as putting a statistical blindfold on your data.

It mathematically removes the influence of the life stress variable, holding it constant across everyone so you can see the pure, unpolluted relationship between publications and grants.

That is amazing.

And what if you aren't comparing groups at all?

If you just wanna see if two continuous numerical scales move together, like age and income, you use a Pearson correlation.

Okay, that makes it so much more approachable.

When you run these tests in the software, it spits out a report.

And the number everyone immediately looks for is the p -value to see if the result is statistically significant.

Ah, the infamous p -value.

Now, the common explanation I always hear is that a p -value of less than .05

means there is a less than 5 % chance that your results were a fluke.

Is that exactly right?

That is actually one of the most common and frankly dangerous fallacies in science.

A p -value does not tell you the probability that your results are due to chance or the probability that your hypothesis is right.

Wait, really?

Then what is it actually measuring?

It operates in reverse.

Yeah.

A p -value is the probability of obtaining your exact data or something more extreme, assuming the null hypothesis is completely true.

Assuming the null hypothesis is true.

Meaning?

We're not assuming.

Nothing has happened.

Exactly.

Meaning, if we assume there's absolutely zero relationship between violent video games and aggression in the real world, how weird or surprising is the data we just collected?

Uh.

If the p -value is .01, it means our data would be highly bizarre.

Only a 1 % probability of looking like this if there were truly no relationship.

Because the data is so weird under that assumption, we reject the assumption.

Wow, okay.

So it's a measure of how surprising the evidence is against the baseline assumption of nothing is happening.

That completely reframes it.

It's a huge shift in perspective.

But even if you have a great p -value, the methodology rules state you also have to report confidence intervals and effect sizes.

Why?

Because a p -value only tells you if an effect exists, not if it matters.

A confidence interval gives you the range where you are 95 % certain the true population average actually lives.

And the effect size.

The effect size tells you the practical magnitude.

You could have a highly statistically significant result showing that a new tutoring program improves test scores, but the effect size might reveal it only improves scores by half a point.

Statistically real, but practically useless.

That's a great distinction.

So surveys and statistics are incredibly powerful for mapping these relationships and predictions.

But let's pivot.

Let's do it.

If we are tired of just predicting things and we wanna draw a hard line in the sand and say treatment A definitively caused outcome B, we have to leave the survey behind.

We have to design a true experiment.

And the absolute heart of an experimental method.

The mechanism that separates it from everything else is random assignment.

Wait, I'm confused here.

Didn't we just spend a bunch of time talking at random sampling, pulling names out of buckets?

Is random assignment just a different word for the same thing?

It is a critical distinction that trips up a lot of people.

Random sampling is how you get people into your study from the outside world.

Random assignment happens once they are already inside the building.

Okay, inside the building.

It is the process of randomly sorting your existing participants into different experimental conditions.

Ah, so random assignment is flipping a coin at the classroom door.

Heads, you go into room A and get the new math tutor.

Tails, you go into room B and get no tutor.

Exactly.

And the mechanism behind why we do this is vital.

It eliminates systematic bias.

Because it's a coin flip.

Right, because you flipped a coin.

The highly motivated students, the tired students, the math geniuses, they are all equally distributed between room A and room B.

The two groups are perfectly equal before you do anything.

So if room A scores higher at the end, the only logical explanation is the tutor.

And if you can't randomly assign them, say you are forced to use pre -existing classrooms.

Then it is no longer a true experiment.

It is called a quasi -experiment.

And your ability to prove causation drops significantly.

That is a massive distinction.

And in these experiments, the language shifts, right?

The thing you manipulate, like the math tutor, is the independent variable.

The outcome you measure, the test score, is the dependent variable, because it depends on the manipulation.

And you must rigorously control for confounding variables, those noisy variables that mess up your data.

To do that, researchers often use a manipulation check.

Let's dig into that, because the mechanics here are brilliant.

Say your experiment involves trying to induce low self -esteem in participants to see how it affects their puzzle -solving skills.

Okay, evil scientist.

Right, you do this by giving them brutally fake, negative feedback on an initial test.

You can't just assume your insult worked.

You need a manipulation check, a quick hidden questionnaire immediately after the feedback to verify their self -esteem actually dropped.

If their self -esteem didn't budge, your independent variable failed, and the experiment is broken before they even get to the puzzles.

Furthermore, if participants know you are trying to lower their self -esteem, they will act unnaturally.

So researchers employ a cover story,

a plausible but untrue explanation of the study's purpose.

But isn't that lying?

It's deception, yes.

And the moment you use deception, ethics take over.

You're required to have an extensive debriefing process afterward to explain the true purpose and undo any psychological harm.

And the entire protocol must be rigorously audited and approved by the IRB.

So we have our randomly assigned participants, our variables, and our cover story approved.

How do we actually sequence the events in the lab?

Are we testing everyone at once or bringing them back multiple times?

That is your experimental design structure.

You could use a between -subjects design where a participant only experiences one condition, they get the tutor or they don't.

Or a within -subjects design where they experience all conditions.

They take a pre -test, get the tutor, and take a post -test.

Or a mixed design combining both.

To visualize this, the chapter introduces a classic notation system developed by Campbell and Stanley.

And when you look at it, it honestly looks exactly like a football coach drawing up a play on a chalkboard.

It's all X's and O's.

It functions exactly like a playbook.

Let's decode the symbols.

Okay, so X is the play.

That's the experimental treatment or the independent variable.

O is the observation.

That's when you measure the dependent variable.

And R is making sure the teams are fair before the game starts, random assignment.

Perfect.

So let's look at the first play in the book.

Example 8 .1, the pre -experimental one -shot case study.

On the chalkboard, it's just X followed by O.

A group gets a treatment and you measure them.

No control group, no pre -test.

That seems incredibly weak for proving causation.

You have nothing to compare it to.

It is very weak.

So we turn to example 8 .2, the quasi -experimental design.

Here, you have an experimental group and a control group.

Both get a pre -test, an O, the experimental group gets the treatment, an X, and both get a post -test, another O.

But crucially, there is no R, no random assignment.

Right, you used pre -existing groups.

So while it's better, you still can't be sure the groups were equal to begin with.

Which brings us to the Super Bowl play.

Example 8 .3, the true experiment.

Specifically, the post -test only control group design.

Let's draw it up.

Group A gets R, then X, then O.

They are randomly assigned, they get the treatment, they get measured.

Group B gets R, then just O.

Randomly assigned, no treatment, measured.

Because of that powerful R, any difference in the final O has to be caused by the X.

And occasionally, you'll see example 8 .4, a single -subject design, like an ABA design, where you observe a baseline behavior, A, give a treatment, B, and then actually withdraw the treatment to see if the behavior returns to the baseline back to A.

So you've drawn up the perfect play on the chalkboard.

But in the real world, the field is muddy, players trip, things go wrong.

These are your threats to validity.

Yes, internal validity threats from table 8 .5 are the things that actively destroy your conclusion that your treatment caused the outcome.

Let's look at the mechanisms of how these ruin a study.

First is history.

Imagine you are testing a new financial literacy curriculum and halfway through the semester, the national stock market collapses.

Oh, wow.

That external event is a history threat.

At the end of the study, did your curriculum change the student's spending habits or did the evening news terrify them?

You have no idea.

Then there's maturation.

If you are testing a year -long reading intervention on kindergartners, you have to remember that five -year -olds naturally develop rapidly over a year.

Are they reading better because of your specific program or literally just because their brains grew up?

Right.

And mortality, which sounds grim, but in research usually just means attrition.

If your new math program is so frustrating that all the struggling students drop out, leaving only the math geniuses, your final average score will look artificially high.

Not because the program worked, but because the people who were failing left the room.

Exactly.

There's also diffusion of treatment,

where participants in the control group talk to the experimental group in the waiting room, learn the secrets of the treatment, and contaminate their own data.

They just share the playbook.

Yep.

And testing, where simply taking the pre -test acts as practice, making them score higher on the post -test even if the treatment was useless.

And beyond internal threats, you have external validity threats in table 8 .6.

This basically asks, can you generalize this outside the lab?

Generalizability is huge.

Like, if your experiment worked perfectly on 18 -year -old college freshmen in a quiet, temperature -controlled laboratory, can you confidently say this treatment will work for 55 -year -old factory workers on a noisy floor?

That's the interaction of selection and setting.

But honestly, looking at this massive list of threats, it feels hopeless.

How can we trust any published study if there are a dozen ways it can secretly fail?

It is a vital question.

But this is exactly why experimental methodology is so rigid.

We trust good studies because a well -designed experiment actively neutralizes these threats.

How so?

Through control groups and blinding.

If a stock market crash, so history, happens, it happens to both the experimental and the control group equally.

If participants age over a year, maturation, both groups age equally.

The threats cancel each other out across the two groups, leaving only the true effect of the independent variable.

By blinding the experimenter so they don't know who is in which group, you prevent the researcher from suddenly influencing the participants.

It's a beautifully closed system when done right.

And the chapter wraps this entire methodology up with a brilliant real -world example,

Creswell's value affirmation study.

Undergraduates were randomly assigned, there's our R, to either a value affirmation group or a control group.

Then they were all subjected to a highly stressful laboratory task.

The dependent variable they measured wasn't a survey answer, it was the actual stress hormone cortisol measured in their saliva.

And because they used random assignment, a strict control group, and perfectly timed laboratory procedures to block out external solidity threats, they could definitively conclude whether the psychological value affirmation actively buffered the biological stress response.

It shows the entire methodological engine working in perfect harmony.

So let's look at the roadmap we just traveled.

We started with the philosophical assumption of determinism, seeking relationships.

We explored how to uncover those relationships by sampling populations, calibrating survey instruments, and running inferential statistics.

And finally, we learned how to isolate variables and prove direct causality using the rigorous, tightly controlled playbook of the TRUE experiment.

It is a comprehensive framework for discovering truth in a noisy world.

But before we finish, there is one final, modern concept mentioned in the chapter that is fundamentally reshaping quantitative research.

Pre -registration.

Ah, yes.

Pre -registration is a revolution.

It requires a researcher to publicly publish their exact hypothesis, their sample size power analysis, and their data analysis plan in an independent registry before they collect a single piece of data.

Why is that mechanism so crucial to modern science?

Because it stops researchers from cheating.

In the past, someone might collect the data, realize their original idea failed, but notice some weird anomaly in the numbers.

They would then go back,

rewrite their introduction, make it look like they predicted that anomaly all along, and publish it as a statistically significant breakthrough.

Basically painting the bullseye around the arrow after they shot it.

Precisely.

Pre -registration locks your predictions in stone.

You can't move the goalposts after the game is over.

Which leaves you with a deeply provocative thought.

If pre -registration is the strict rule today, how many of the classic famous textbook studies from the 20th century do you think would actually survive if they were held to today's quantitative rules?

It is a genuinely fascinating question, one that should encourage every student to look at older research with a sharp analytical eye.

Absolutely.

Well, if you are that student staring down your syllabus right now, take a deep breath.

You now know how the engine works.

On behalf of the last minute lecture team, thank you for joining us on this deep dive.

You've got this.