Chapter 10: The Nonexperimental and Quasi-Experimental Strategies: Nonequivalent Group, Pre–Post, and Developmental Designs

Welcome to Last Minute Lecture.

This free chapter overview is designed to help students review and understand key concepts.

These summaries supplement not replaced the original textbook and may not be redistributed or resold.

For complete coverage, always consult the official text.

Welcome back to the Deep Dive.

You know, we got a question recently that really made me think.

It was about this study suggesting people with moderate adversity might actually have, well, better mental health than those with very little or tons of it.

Yeah, that finding really stops you in your tracks.

It's counterintuitive almost.

It is that maybe some struggle is actually

beneficial.

But then you immediately think, okay, how do researchers even study something like adversity?

Exactly.

That's the methodological puzzle, isn't it?

You can't ethically, you know, assign people to high trauma versus low trauma groups.

No way.

People come with their life experiences already baked in, so they're already in these groups.

Right.

Adversity isn't something you can just dial up or down in a lab setting.

So if the gold standard, the true experiment, needs that kind of manipulation and control,

especially random assignment, to nail down cause and effect, how do we tackle these really important real world questions?

Well, that's precisely what we're digging into today.

We're diving deep into chapter 10 of Research Methods for the Behavioral Sciences, the sixth edition.

Ah, okay.

So this chapter tackles the alternatives.

It does.

It focuses squarely on what are called non -experimental and quasi -experimental research strategies,

designs for when you can't do a true experiment.

Okay, so our mission for this deep dive is to really unpack chapter 10.

We want to figure out what these designs actually are, why you'd use them, and crucially, what their weaknesses are, especially when it comes to saying X caused Y.

And how to spot them, because you see these kinds of studies reported all the time.

Definitely.

Let's untack this.

So just to set the stage, remember, the book lays out five basic research strategies overall.

Experimental is one, but there are others.

Right.

We've spent a lot of time on the experimental one, manipulate one thing, control everything else, look for the effect, the cause and effect powerhouse.

Exactly.

But like we said with adversity, or think about studying, say, different teaching methods in actual schools, or therapies in clinics, the real world often just doesn't allow that perfect control.

You can't always randomly assign students to classrooms or patients to therapies, practical limits.

Precisely.

So you end up needing to compare groups that already exist, or maybe track changes in one group over time without having full control.

That's the constraint we're working with here.

So if you're comparing groups, but you didn't create the difference between the groups, what defines them?

That's the key difference for non -experimental and quasi -experimental studies.

They still compare groups of scores, so they look a bit like experiments sometimes.

Okay.

But the variable that defines the groups isn't manipulated by the researcher.

It's a non -manipulated variable.

It could be a participant characteristic.

They walk in the door with like their age, their gender, maybe their occupation, or like we said, their

Or it could be time itself.

Or time, exactly.

Comparing scores before some event or treatment versus after it.

And this is where we hit that really big flashing warning sign, right?

The limitation.

Absolutely.

This is the non -negotiable takeaway.

Because you lack that manipulation, and crucially that control and random assignment, these designs cannot establish unambiguous cause and effect relationships.

Period.

There's always some other potential explanation lurking.

A confound.

Always an inherent confounding variable or a major threat to internal validity that you just can't eliminate by the design alone.

Okay.

Now, the chapter makes a distinction between non -experimental and quasi -experimental.

What's the difference there?

It sounds subtle.

It is a bit subtle, but important.

It really comes down to the degree to which the design tries to minimize those threats to internal validity we just talked about.

Okay.

A non -experimental design makes frankly little or no attempt to control for those potential confounds.

It might just be describing differences or looking for relationships.

Some more descriptive.

Yeah, often.

A quasi -experimental design, on the other hand, incorporates some features, some elements that try to reduce those threats.

Maybe adding a comparison group that's similar or taking measurements before and after.

It gets closer to the rigor of an experiment, but it doesn't quite make it all the way.

So it's like they're trying harder to rule out alternatives, but can't quite seal the deal like a true experiment.

That's a good way to put it.

They make some attempt.

Okay.

So structurally, you said they often look like experiments because they compare groups of scores.

Right.

The setup can seem familiar, but the groups aren't formed by manipulation.

They're defined by, what was it, a participant variable or time.

Correct.

And that leads to two broad categories, mirroring the structure of experiments.

You have between -subjects types, where you compare different groups of people.

These are often called non -equivalent group designs.

Because the groups aren't equivalent from the start.

Precisely.

And then you have within -subjects types, often called pre -post designs, where you compare scores from the same group of people measured at different times.

Okay.

Between groups and within groups, but with a twist because of how groups or conditions are defined.

Exactly.

And since we don't have a true manipulated independent variable, the chapter gives a specific name to the variable that does define the groups or the time points being compared.

Right.

What was that term again?

It's the quasi -independent variable.

That's learning objective 12, by the way.

Quasi -independent.

Okay.

So that's the stand in for the independent variable like age or time before after.

Yes.

It's the variable that differentiates the groups or scores and the outcome you measure.

That's still just the dependent variable.

Same as always.

So back to our adversity example, the level of adversity,

none, moderate, high, that would be the quasi -independent variable.

Correct.

And the mental health score is the dependent variable.

Perfect.

You're comparing groups defined by adversity, not manipulating adversity itself.

Got it.

Okay.

Let's dive into that first category.

Between -subjects, non -equivalent group designs.

All right.

So the defining feature here, LO2, is that the groups you're comparing are pre -existing.

They existed before you even showed up to do the study.

Like comparing kids from different schools or people with different jobs.

Exactly.

And the critical point is the researcher has no control over who is in which group.

No random assignment, no matching people across groups, none of that.

Which is why they're called non -equivalent.

Yeah.

You have to assume they might be different from the Yes.

Assume baseline differences.

Use the example from the chapter about a new foam policy in high schools.

You want to compare a school with the policy to one without.

You can't just randomly assign students to attend school A or school B.

Right.

You study the kids already there.

And that leads directly to the major threat to internal validity for these designs, LO3, assignment bias due to individual differences between the groups.

Because you didn't make them Right.

Any difference you find in, say, student focus or grades could be because of the foam policy.

Or it could be because the students in school A were already different from the students in school B in ways that matter, maybe higher motivation, different socioeconomic backgrounds, you name it.

So those pre -existing differences are confounded with the variable you're interested in, like the foam policy.

Exactly.

And that threat is always there in non -equivalent group designs.

It's built in.

Okay.

So what's the simplest non -experimental version of this?

That would be the differential research design, LO4.

It's very straightforward.

You simply compare two or more groups that are already defined by some existing participant characteristic.

Like comparing men and women or different age groups or personality types.

Yes.

Or people from single parent versus two parent households, like one example.

Or even, as the chapter amusingly notes,

comparing accident rates for people with different astrological signs.

Seriously.

Okay.

Definitely pre -existing groups there.

Absolutely.

The goal is just to see if differences exist between these pre -defined groups.

It's often called ex post facto research, meaning after the fact, because you're studying characteristics people already have.

So our adversity study, comparing the mental health of groups based on their past adversity, that sounds like differential research.

It certainly could be framed that way.

Yes.

You identify the groups based on history, then measure and compare their current mental health.

No manipulation.

And the book makes a point, box 10 .1, about how this differs slightly from correlational research, even though both are non -experimental.

Right.

It's about how you structure the data and the question.

In differential research, you use that participant characteristic, say household type, to explicitly create groups.

Then you compare the average score on the dependent variable, say self -esteem, between those groups.

You're looking for a mean difference.

Like in figure 10 .4a.

Exactly.

Whereas in correlational research, you treat everyone as one big group.

Yeah.

You measure both variables for each individual, and then you look for a pattern, an association, a correlation between those two variables across the whole sample, like in figure 10 .4b.

So comparing group means versus looking for a relationship pattern.

Correct.

But the bottom line for both, neither can establish cause and effect.

That's the crucial similarity.

Okay.

What's the next non -experimental, non -equivalent group design?

That's the post -test only non -equivalent control group design, LO4.

This one often pops up when evaluating some kind of treatment or program when you can't use random assignment.

How does it work?

You have one pre -existing group that gets the treatment or intervention, let's call them the treatment group.

You compare them to another pre -existing non -equivalent group that doesn't get the treatment, the control group.

And here's the key.

You only measure both groups after the treatment has happened, just a post -test.

So using the book's notation, it's XO for the treatment group, treatment, then observation.

Right.

And just O for the control group, observation only.

No pre -test, no random assignment.

Exactly.

The chapter uses an example of introducing street parks in one neighborhood, X, and then measuring social interaction, O.

You compare that to social interaction, O, in another neighborhood without the new parks.

Or maybe students who took an optional ethics course versus those who didn't, comparing their ethics scores afterwards.

Perfect example.

And why is it non -experimental?

Because you have no idea if the groups were comparable before the intervention.

The neighborhood that got the parks might have already been more social.

The students who chose the ethics course might have already had higher ethical reasoning.

Precisely.

They're non -equivalent.

The observed difference could be the treatment, or it could just be those pre -existing differences.

This design doesn't help you sort that out.

Okay.

So that seems pretty weak in terms of ruling out alternatives.

How do you make it stronger to move towards quasi -experimental?

The key improvement is adding a pre -test measurement.

That gives us the pre -test, post -test, non -equivalent control group design, L04, L05.

Okay.

So you measure both groups before the treatment, then one group gets the treatment, then you measure both groups again afterwards.

Exactly.

So the schematic becomes OXO for the treatment group and OO for the non -equivalent control group.

Pre -test, treatment, post -test for one.

Pre -test, no treatment, post -test for the other.

Yes.

And that added pre -test, L05, is what significantly boosts the internal validity.

Why?

Because it lets you see if the groups were similar or different on your dependent variable before the intervention even started.

It gives you a baseline comparison.

Right.

If the two groups scored similarly on the pre -test but differently on the post -test, you have much stronger evidence, not definitive proof, but stronger evidence that the treatment, the X, actually caused the change.

You've somewhat accounted for those initial group differences on the measured variable.

It helps rule out the explanation that they were just different groups to begin with, at least on the thing you measured.

Exactly.

It reduces that threat.

And this design also helps control for many time -related threats, things like history effects, outside events, or maturation, people changing naturally over time.

Because both groups are measured over the same time period.

So presumably, any major outside event or natural developmental trend should affect both groups similarly.

If one group changes significantly more than the other, it's less likely due to just general history or maturation.

Okay, that makes sense.

But there's still a catch, isn't there?

Why is it still only quasi -experimental?

The big remaining threat is something called differential history.

Differential history.

Meaning history affects the groups differently.

Yes, precisely.

Because the groups are non -equivalent and often exist in different locations or contexts, like two different schools, two different clinics, some external event might happen to one group, but not the other during the study period.

Like the chapters example.

Maybe one school has a winning sports season that boosts morale, while the other has a losing season.

That could affect student outcomes differently, totally separate from the teaching method being studied.

Exactly.

Or maybe a local factory closes near one community health clinic, but not the other.

That differential event becomes a confound.

You can't be sure if the treatment or the unique local event caused the difference between groups.

That potential for differential time related effects is why it remains quasi -experimental.

Got it.

Even with the pre -test, you can't guarantee the groups experience time in exactly the same way.

Correct.

Okay, let's switch gears to the other main category.

Within subjects pre -posed designs.

What's the setup here?

So here, LO6, instead of comparing different groups, you're tracking the same group of individuals over time.

You make a series of observations on this single group.

And the goal?

Usually it's to see if some intervening event or treatment influenced their scores by comparing the observations made before the event to those made after it.

So one group acts as its own control or comparison across time.

No separate control group needed like in the non -equivalent designs we just discussed.

Right.

And because you're tracking the same people over time, the main threats to internal validity, LO7, are those time related factors we know from Chapter 9.

Let me guess.

History, maturation, instrumentation.

Testing effects or order effects from being measured multiple times.

Yeah.

And statistical regression towards the mean.

Yep.

All five are potential culprits.

And you can't use counterbalancing here like in a typical within subjects experiment because before has to come before after.

Absolutely.

Time only flows one way.

So these designs have to inherently grapple with those time related confounds.

Okay.

So what's the most basic, the non -experimental version of this pre -post approach?

That's the straightforward pre -test, post -test design.

LO8.

Simplest possible version.

One observation, then the intervention or event, then one more observation.

Schematic, OXO.

Exactly.

Like measuring voter attitude,

O, showing them a political ad, X, then measuring their attitude again, O.

Or maybe measuring depression scores, giving a week of therapy, X, measuring again.

Yes.

And it's non -experimental LO8 because it makes absolutely no attempt to control for any of those time related threats.

If the attitude changes after the ad or depression lessons after therapy.

You can't be sure it was the ad or the therapy.

Maybe some major news event happened between measurements, history.

Maybe the person was just having a better week anyway, maturation.

Maybe just being measured the first time changed how they responded the second time.

Testing effect.

You simply cannot isolate the effect of X.

Too many other possibilities.

So how do you strengthen this?

How do you make it quasi -experimental?

You add more observations, lots more.

This leads to the time series design, LO8, LO9.

Okay.

So instead of just OXO, it's more like.

O, O, O, X, O, O, O.

A series of observations before the event, X, and a series of observations after the event.

And that X, the intervention,

could be something the researcher introduces, like a new workplace policy.

Yes.

Or it could be a naturally occurring event that the researcher didn't control at all, like a change in law, an economic downturn, or even a natural disaster like an earthquake.

That specific case is called an interrupted time series design.

You're looking at data before and after some interruption you didn't cause.

Like tracking crime rates for months before and after a new policing strategy is implemented.

Perfect example.

Or maybe looking at worker productivity before and after a company reorganizes.

Often uses archival data.

So how does having that series of observations, the O, O, O parts,

help control the time threats, LO9?

This is the clever part.

That series of observations before the X is crucial.

It establishes a baseline trend.

It lets you see if things were already changing before your intervention happened.

Ah, so you can spot pre -existing trends.

Exactly.

Look at figure 10 .6 in the chapter.

If the data show a clear trend upwards or downwards before X, and that trend just continues after X, then X probably didn't have much effect.

But if the data are stable before X and then show a clear jump or drop right after X, you have much better evidence that X caused the change.

You've visually ruled out maturation or other gradual trends.

It helps distinguish the intervention's impact from ongoing changes.

Right.

And what about the history threat?

Is that still a problem?

It's much less of a problem unless some other outside event happens at the exact same time as your intervention X, LO9, figure 10 .5.

Like the example of starting depression therapy just as the weather dramatically improves.

Yeah.

If they happen simultaneously, you can't tell if improvement was therapy or sunshine.

But if the other event happens clearly before X or clearly after X, the time series graph will usually show its separate effect, distinct from the change associated with X.

Okay.

So coincidence is the main historical threat here.

Pretty much.

And the series of observations after X is also useful.

It shows you if the effect of X was immediate, if it was gradual, if it wore off quickly, or if it was sustained long term.

Figure 10 .6 again gives you a richer picture of the impact.

Makes sense.

And these can be used for just one person.

Or a whole organization.

Yes.

They're very flexible.

Used for single cases, groups, organizations, even cities or states looking at policy impacts.

Classified as quasi -experimental because that series of observations provides significant control over those time -related threats even without a separate control group.

Okay.

Brilliant.

That covers the main non -experimental and quasi -experimental structures based on comparing groups or time points.

The chapter then discusses a specific application area.

Studying changes related to age.

Right.

Developmental research designs.

LO10.

These are specifically focused on understanding how behavior, thoughts, or feelings change as people get older.

And since you absolutely cannot manipulate or randomly assign age.

These are inherently non -experimental designs.

Their goal is primarily descriptive to describe the relationship between age and other variables.

This is the first type.

The cross -sectional design.

LO10.

This is basically a between -subjects approach applied to age.

How does it work?

You select different groups of participants where each group represents a different age.

For instance, you might recruit a group of 40 -year -olds, a separate group of 60 -year -olds, and a third group of 80 -year -olds.

Then you measure all of them on your variable of interest, say memory performance, at roughly the same point in time, like in figure 10 .7.

So it's like a differential design using age to define the pre -existing groups.

Essentially, yes.

It's differential research focused on age differences.

What are the advantages?

The big pluses LO10 are speed and efficiency.

You can collect data on different age groups relatively quickly all at once.

You don't have to wait 20 or 40 years for people to age.

And participants only need to commit to one session.

Seems practical.

But what's the major downside?

The huge weakness LO10 is the potential for cohort effects, sometimes called generation effects.

Ah, right.

Explain that again.

Differences you observe between your 40, 60, and 80 -year -old groups might not be due to the biological or psychological process of aging itself.

They might be due to the fact that these different generations grew up in vastly different historical contexts, with different educational systems, nutrition, technology, cultural norms, major world events.

So maybe the 80 -year -olds score lower on a tech -related task, not because of age -related decline, but because they didn't grow up with computers like the 40 -year -olds did.

Exactly.

Their life experiences are different.

The effect of age is confounded tangled up with the effect of the generation or cohort they belong to.

You can't separate those influences with a purely cross -sectional design.

That seems like a massive problem for studying aging.

So how do you get around cohort effects?

You follow the same people over time.

That's the longitudinal design, LO10.

Okay, so this is a within -subjects approach.

Yes.

You select one group of participants, often people who are around the same age, to start a single cohort.

And you measure them repeatedly on your variable of interest over a significant period as they age.

Like testing the IQ of a group of 40 -year -olds, then testing the same people again when they're 60, and again when they're 80, like figure 10 .8.

Precisely.

It's kind of like a very long -term pre -post design, where aging is the process happening between measurements.

What's the main advantage here?

The biggest strength, LO10, is that it eliminates cohort effects as an explanation for observed changes, because you're only studying one generation.

Any changes you see are more likely related to aging itself.

Plus, you can actually track individual development, how specific people change over time, not just group averages.

That sounds much better for understanding how people actually age.

But I sense some major drawbacks coming.

Oh, yes.

Huge practical drawbacks, LO10.

First, they are incredibly time -consuming.

We're talking studies that can last decades.

Wow, okay.

Which also means they are extremely expensive, maintaining funding, research staff, contact with participants for that long.

It's a massive undertaking.

And keeping the participants involved must be hard.

That's the other killer problem.

Participant attrition or dropout.

Over decades, people move, they lose interest, they get sick.

Sadly, some pass away.

It's inevitable you'll lose participants.

Why is that such a threat?

Because the people who drop out might be systematically different from the people who stay in the study.

Maybe the less healthy or less motivated or lower -income participants are more likely to drop out.

If that happens, the sample you're left with at the end is biased.

It's not representative of the original group.

So any changes you see might be due to aging or they might just reflect the characteristics of the survivors who stuck with the study, another confound.

Exactly.

Attrition bias can seriously threaten the validity.

And you also have potential issues with testing effects, people getting better or bored just from taking the same test repeatedly over years.

Man, studying development sounds tough either way.

Cross -sectional is fast, but has cohort effects.

Longitudinal avoids cohort effects, but is slow and plagued by attrition.

It's a classic trade -off in developmental research.

The chapter also briefly mentions hybrid designs, like cross -sectional longitudinal designs.

What are those?

They're often used more in sociology or public health, maybe tracking attitudes or behaviors in different age cohorts over time, but usually with different samples at each time point, like surveying college students' droids every five years.

You see how trends change in that demographic, but you're not tracking the same individuals' aging.

Less about individual development, more about societal shifts across age groups.

Okay, got it.

So we've covered the major non -experimental, quasi -experimental, and developmental designs described in the chapter.

What about analyzing the data from these studies?

Right, statistical analysis, LOFN, generally follows the logic of the design type, often using similar tools as experimental research.

So for the between -subject designs, like the non -equivalent group comparisons or cross -sectional studies?

You're typically comparing the mean scores of the different groups.

So you'd use things like independent samples t -tests, if there are two groups, or ANOVA, analysis of variance, if there are more than two groups.

If your dependent variable is non -numerical, like categories, you might use a chi -score test.

Makes sense, compare the group averages.

And for the within -subjects designs, like pre -post, or time series, or longitudinal?

There, you're comparing scores from the same people measured at different times.

So you'd use repeated measures t -tests, or repeated measures ANOVA.

Okay, what about that pre -test -post -test, non -equivalent control group design, that had both different groups and measurements over time?

Ah, good catch, that's a mixed design.

It has a between -subjects factor, treatment versus control group, and a within -subjects factor, pre -test versus post -test time.

So you'd typically analyze that using a mixed design ANOVA, though the chapter just notes this without going into the analytical details.

Right, and just to circle back on the terminology one last time, LO12 recap.

The key terms are the quasi -independent variable, that non -manipulated variable, like age, gender, time, group membership, that defines the conditions being compared.

Distinct from a true manipulated independent variable.

Exactly, and the dependent variable, which is the outcome you measure, to see if it differs across those conditions or time points.

And the chapter mentions table 10 .3 as a handy summary of all these designs we've discussed.

Yes, that table is a great quick reference for the structures, threats, and whether they're non - or quasi -experimental.

Okay, I feel like we've really covered the ground here.

We've done a thorough deep dive into chapter 10, hitting all the key strategies outside of true experiment.

Definitely.

We looked at why you'd need non -experimental and quasi -experimental designs, those practical and ethical constraints.

We clarified the crucial difference between them based on the attempt to control threats.

And we hammered home the main limitation, difficulty establishing clear cause and effect.

Yep, we explored the two main structures, non -equivalent group designs.

Like differential research and post -test only, and the stronger pre -test, post -test, non -equivalent control group design.

And the within subjects pre -post designs.

From the simple non -experimental pre -test, post -test, to the more robust quasi -experimental time series design.

And we can't forget the specific challenges of developmental research using cross -sectional and longitudinal approaches.

We touched on the typical statistical analyses and recapped the key terminology.

I think we've genuinely hit every major section and concept in this chapter.

Agreed.

We've laid out the landscape of these important alternative research methods.

So the big takeaway, really, is about understanding the trade -offs, isn't it?

These designs let us study things we couldn't otherwise.

But we have to be incredibly cautious about jumping to causal conclusions.

Quasi -experimental designs try to mitigate the risks, but confounds like differential history or cohort effects can still be major issues.

Absolutely.

It's all about recognizing the design's inherent limitations.

Okay, so here's a final thought for you, the listener.

Next time you read about a study, maybe comparing smokers and non -smokers, or looking at the impact of a new law, or even revisiting that finding about moderate adversity and well -being, ask yourself, could the researchers have possibly randomly assigned people to these groups or conditions?

If the answer is likely no, then chances are you're looking at a non -experimental or quasi -experimental design.

And your next question should be, what are the groups being compared?

How are they formed?

If it's a study over time, what else might have happened during that period?

What are the potential confounds, the alternative explanations that this particular design might not rule out?

Developing that critical eye, being able to recognize the structure and the inherent limitations of these designs.

That's a huge step towards becoming a much sharper, more informed consumer of the research you encounter every day.