Chapter 10: Mixed Methods Procedures

Welcome to Last Minute Lecture.

This free chapter overview is designed to help students review and understand key concepts.

These summaries supplement not replaced the original textbook and may not be redistributed or resold.

For complete coverage, always consult the official text.

Imagine you're trying to truly understand a massive bustling city, right?

And it's your very first time there.

Okay, I'm picturing it.

So you could send up a drone.

You get that really high altitude zoomed out shot.

And from up there, you'd see all the traffic patterns, the population density across the different boroughs, maybe the exact number of parks, right?

Which is incredibly precise.

I mean, it's totally quantifiable.

Exactly.

But it's also completely silent, like it gives you the scale of the city, but it keeps you entirely distant from the actual, you know, the experience of living there.

Yeah.

To really know the city, you got to land the drone.

You have to walk down to the street level.

You got to smell the garlic roasting at the food carts, actually talk to the locals about what their neighborhood feels like.

You need their stories.

Yes, you need the stories.

And today, we're looking at the ultimate researcher's dilemma.

How do you merge that high altitude drone shot with the street level interview?

It's the big question.

Welcome to this deep dive.

Today our mission is to tackle Chapter 10 from Creswell and Creswell's research design text.

We're unpacking mixed methods procedures.

Which can sound pretty daunting if you're a college student staring down a research methods syllabus.

Oh, totally.

Yeah.

But our goal here is to show you exactly how top tier researchers hack both numbers and narratives together.

We want to make sure your own findings are absolutely bulletproof to any skeptic.

It is a critical skill to have because, well, mixed methods is actually officially recognized as the third methodological movement.

Third movement.

Yeah.

So for a really long time, researchers just lived in these strict silos.

You had your pure quantitative research.

So the numbers.

Right.

And then your pure qualitative research, the narratives.

But then emerging roughly in the late 1980s, people realized that neither approach by itself could capture the full complexity of the human experience.

Makes sense.

So they carved out this third distinct space that sits between them and actively incorporates both.

OK.

So let's figure out what actually makes this so powerful.

According to the text,

there are six defining characteristics that separate true mixed methods from just like a messy pile of data.

Yeah.

They're very specific about this.

First, you're collecting both qualitative, open -ended data and quantitative closed -ended data.

But you don't just dump them into the same report.

No, absolutely not.

You connect them.

You integrate them using really specific, deliberate design procedures.

Right.

You analyze them together, often visually.

You draw what the book calls metinferences from that combination.

And finally, you frame the entire study within your specific worldview or theoretical lens.

And the justification for going through all this extra effort is fascinating, really.

I mean, we do it to mine the data further.

Right.

The authors use the pandemic as a very relatable example here.

During that time, public health officials were constantly looking at the quantitative statistics, right?

New cases, vaccination rates.

Just refreshing the dashboards every day.

Exactly.

But those numbers hit a wall.

To truly understand the problem of vaccine hesitancy, researchers had to shift gears.

They had to hear the qualitative voices of individuals who were resisting the mandates.

Because you need the numbers to see the scale of the crisis, but you need the stories to uncover the underlying reasons.

Precisely.

Well, let me stop you there and just pose a hypothetical.

Sure.

Say I'm a market researcher trying to figure out why a new flavor of spicy Doritos is failing.

Okay.

Tragic.

So I send out a multiple choice survey to a thousand people.

And then the next week, I go to a grocery store and do a long form interview with five completely different people about their favorite snack chips.

Is that mixed methods?

The textbook is very strict on this point.

No.

No, it is not.

Really?

Yeah.

If you are just gathering multiple sources of data and keeping them completely separate, or if you fail to actually blend the findings, that's just called multi -method research.

True mixed methods must cross the boundary between closed -ended numbers and open -ended words.

Crucially, it must integrate them.

Integration is the absolute core mechanism here.

Okay.

So since integration is the engine driving this whole thing, let's open up our vocabulary toolkit.

We need to define what these pieces actually look like in practice before we start building any research blueprints.

Good idea.

First, we have open -ended versus closed -ended data.

Right, which is foundational.

Closed -ended is your typical survey scale.

Like rate this from strongly agree to strongly disagree.

And open -ended is when the participant just shares their views in their own unprompted words.

But keep in mind, sometimes a single source can provide both of those simultaneously.

Wait, really?

How does that work?

A patient's medical record is a great example of this.

It contains closed -ended objective scores, like your blood pressure or cholesterol tests.

Oh, sure.

But it also contains open -ended written narrative notes from the attending physician about the patient's family history, or maybe their emotional state.

Oh, wow.

Yeah, that makes perfect sense.

Next up is the research design itself, which the book says isn't just a survey form.

It's the entire procedural architecture.

It stretches from your broad philosophical assumptions about how knowledge is created all the way down to how you actually interpret the final data.

It's the whole blueprint.

Yeah.

And integration, as we just established, is both the intent to combine and the actual

of combining those two databases.

Now, the text talks heavily about presenting this integration in what they call a joint display.

Yes, that's a big one.

And I want to avoid painting a mental picture of confusing textbook tables with X and Y axes.

So is a joint display basically just a split screen on your TV?

I love that analogy.

Like, on the left side, you've got the raw numbers showing plummeting student test scores, And on the right side, you've got the exact quotes from frustrated teachers, and you're just forcing them to talk to each other.

That is the perfect way to visualize it.

You're forcing the numbers and the words into the exact same room to see how they interact.

It's the literal visual representation of your integration.

Okay, awesome.

And then there's metanfrances.

Ah, yes.

Which honestly sounds like a made -up academic word designed just to secure grant funding.

Kind of does.

What does that actually mean in plain English?

Is a metanference basically the idea that one plus one equals three?

Like an insight you could only possibly get by smashing that split screen data together.

That is exactly what it is.

A metanference is the unique higher level conclusion you draw specifically from examining that joint display.

You look at the quantitative inference, you look at the qualitative inference, and you deduce a new integrated truth that neither database could have yielded on its own.

Wow.

Okay, so now that we have our toolkit, we can look at the first of the three foundational blueprints for putting these pieces together.

The core designs.

Right.

Core design number one is the convergent design.

So this is a one -phase design.

The intent here is to collect both your quantitative and qualitative data simultaneously.

At the exact same time.

Exactly.

And the goal is to compare, merge, or validate the results against each other.

But there's a massive underlying assumption you have to get right for this to work.

Fix that.

You must measure the exact same concepts or variables in both phases.

If your survey is measuring a community's level of trust in local police, your interviews also have to be asking about trust in local police.

You can't compare apples to completely unrelated oranges.

Let's bring back our city metaphor.

I picture the convergent design like this.

Your drone measures a massive traffic gridlock at a specific intersection at 2 p .m.

That's your quantitative data.

At the exact same time, your street team is standing at that intersection interviewing the drivers about why they're stuck.

That's your qualitative data.

You merge their testimonies side by side on that split screen to see if they converge.

So if the drone shows the traffic is stopped because of a roadblock and the drivers say they're waiting for a parade to pass, the data converges perfectly.

It does.

But this introduces a major validity threat.

Which is?

What happens when they diverge?

What if the drone shows traffic moving at a perfectly normal speed, but every driver you interview furiously claims it took them an hour to move two blocks?

Oh, right.

That's a problem.

A fatal flaw in this design is just ignoring those divergent results.

You cannot sweep that mismatch under the rug.

You have to explain it.

Like maybe the drone was measuring the wrong lane, or maybe the driver's perception of time was altered by road rage.

Exactly.

And another common threat here is the issue of unequal sample sizes.

Your drone might track 5 ,000 cars, but your street team only interviewed 10 people.

You have to be really careful not to let the sheer volume of the quantitative data drown out those qualitative voices.

To see this in action, look at the class and study on older driver safety that's mentioned in the chapter.

Oh, that's a classic convergent design.

Yeah, the researchers took a massive national quantitative data set of crash statistics involving elderly drivers.

But numbers alone don't prevent accidents.

No, they don't.

So simultaneously, they conducted qualitative interviews with the stakeholders, the older drivers themselves, their families, and driving practitioners.

And then they built that joint display.

Right.

Table 10 .2 in the book shows how you can map this out.

They put the hard statistics of intersection crashes right next to the personal stories of declining vision or delayed reaction times.

The numbers showed them where the danger was, and the interviews showed them the human mechanism causing it.

It's a brilliant use of the design.

It really is.

But what happens if you shoot that drone up and you spot a neighborhood where the power usage is absolutely zero?

Uh -oh.

You didn't expect to find that.

You have no street team deployed there because you didn't even know it was an issue.

You have to wait, analyze that weird number, and then specifically send a team to investigate.

That timeline completely breaks the convergent model.

It does, which necessitates our second blueprint, Core Design 2, the explanatory sequential design.

Okay, explanatory sequential.

Notice the word sequential.

This is a two -phase design, and the flow is strictly chronological.

Quantitative data collection happens first.

The numbers come first.

Right.

You analyze those numbers, find the significant or surprising results, and then you follow up with qualitative data collections specifically to explain those numbers in more detail.

So the joint display for this one reads left to right, just like the study itself.

Like in table 10 .3.

Exactly.

Quantitative scores on the left,

qualitative follow -up themes in the middle,

and the mini -inferences on the right explaining how the themes explain the scores.

Yes.

But the book highlights a massive non -negotiable sampling rule here.

The qualitative sample must be a subset of the participants from the quantitative sample.

And that rule trips up a lot of new researchers.

Well, let me push back on that because in a lot of research, having overlapping samples is considered a bad thing, right?

It introduces bias.

Usually, yes.

If I'm testing my failed spicy Doritos and I survey a thousand people,

shouldn't my follow -up interviews be with fresh, unbiased people who haven't been primed by my survey questions?

Your instinct about bias is normally correct, but you have to remember the core intent of this specific design.

The entire point is to explain specific quantitative results.

Okay.

Let's stick with your Doritos example.

You survey a thousand people and fifty of them rate the chip a zero, claiming it tastes like literal dirt.

Gross.

Right.

That's a massive unexpected anomaly.

If you go out and find fifty random new people to interview about spicy food, they might actually like the chip.

They wouldn't share the characteristics of your anomaly.

Oh, I see.

To understand why those specific fifty people tasted dirt, you have to go back and interview those exact fifty people.

Selecting a new group would completely ruin the validity of the explanation.

That makes perfect sense.

And the text highlights Banyard's Childhood Abuse Recovery Study to demonstrate this.

Yes, a very powerful study.

They used an explanatory sequential design by gathering longitudinal statistical surveys first.

They were measuring the resilience of survivors across several years.

And the numbers showed a clear measurable trend of recovery for certain individuals.

But a survey can't capture the deep emotional nuance of how someone rebuilds their life.

So they followed up with in -depth interviews with a subset of those exact same resilient women.

The statistical survey identified who was resilient, and the targeted interviews explained the psychological mechanisms behind it.

The design moves linearly from numbers to words.

But let's slip the script again.

What if you're studying a community that is so completely isolated, or a phenomenon so incredibly new, that you don't even know what numbers to measure yet?

You're flying blind.

Exactly.

You don't know what questions to put on the survey, and you don't know what variables matter.

The explanatory design breaks down because you have no starting numbers.

How do researchers handle the complete unknown?

They reverse the order.

That brings us to Core Design 3, the exploratory sequential design.

This is a three -phase design, and it's built for venturing into the unknown.

The flow goes like this.

Qualitative exploration happens first.

You just go into the community and talk to people, gathering raw themes and quotes.

Okay, that's phase one.

Phase two is a design phase.

You use those qualitative findings to build a brand new, culturally sensitive, quantitative instrument like a custom survey.

And finally, phase three is the quantitative test, where you deploy that newly built survey to a much larger population to see if those initial themes generalize.

That is exactly like an architect visiting an uncharted neighborhood.

You don't just show up and start pouring concrete.

No, that would be a disaster.

Right.

First, you sit and interview the locals to see how they actually live, what their family structures look like, and how they move around.

That's the qualitative exploration.

Then, you sit at your drafting table and draw up a blueprint based specifically on their quoted needs.

That's the design phase.

Finally, you build a massive housing complex based on those blueprints and track if the broader population actually uses it correctly.

That's the quantitative test.

And the book shows this flow in table 10 .4, mapping the quotes directly to the new survey items.

And here's where we hit another critical sampling rule, which is the exact opposite of the previous design.

Oh, really?

Yes.

In the exploratory sequential design, your qualitative sample and your quantitative sample must be different individuals, though drawn from the same overall population.

Wait, why do they have to be different this time?

Because if you use the same people who helped you build the survey to then test the survey, you introduce massive confounding factors.

They're already intimately familiar with the questions.

I mean, they helped write them.

Oh, of course.

You need a completely fresh group of individuals to prove that your new instrument actually works accurately at scale.

The ultimate proof of why this design is so vital is Beaton Court's Rwandan HIV -affected children's study.

Yes, an absolute masterclass in the exploratory sequential design.

Imagine trying to measure the mental health of children in Rwanda who have been affected by HIV.

You cannot just take a standard Western mental health survey developed in a clinic in Boston, translate it into the local language, and drop it into Rwanda.

It would be entirely culturally tone deaf.

It wouldn't measure the right idioms of distress, the localized manifestations of trauma, or the community's unique resilience factors.

The numbers would be useless.

The researchers knew that.

So they started with the qualitative phase.

They conducted open -ended interviews with the children and caregivers to unpack locally relevant indicators of mental health.

They listened first.

Right.

Then during the design phase, they took those specific quotes, codes, and themes and used them to build a culturally localized quantitative instrument.

The qualitative words literally became the blueprint for the quantitative variables.

That's incredible.

Finally, they tested that newly minted instrument in a broader quantitative intervention.

The qualitative built the quantitative.

Okay, so these three core designs, convergent, explanatory, and exploratory, they are your basic Lego bricks.

Good way to look at it.

But in real world research,

like testing a massive multi -year public health initiative across a whole state,

you often have to snap these basic bricks together into a much larger, more complex structure.

Right.

And that requires leveling up to what the text calls complex designs.

Complex design.

Then complex designs simply embed those core Lego bricks into larger overarching frameworks.

The chapter outlines four main types.

Okay, lay them on me.

First is the mixed methods experimental or intervention design, where you attach qualitative

during or after a traditional clinical trial to understand the participant's actual experiences with the treatment.

Got it.

Second is the case study design, which uses a convergent approach to build deeply detailed profiles of specific cases so you can compare them.

Third is the evaluation design, commonly used to develop, implement, and refine large scale social programs over time.

Let's focus deeply on the fourth one, though, because it really shows how methodology can

The participatory social justice design.

Oh, this one is vital.

Because this isn't just about collecting data from a detached academic distance.

This is where researchers embed mixed methods into community collaboration frameworks.

Exactly.

In a social justice design, the community members aren't just subjects.

They are co -researchers.

Wow.

You might use an explanatory sequential design gathering survey data on community health and then following up with interviews.

But the entire process is driven by an underlying theoretical lens of empowering a marginalized group and creating actionable social change.

Hodgkin's study on social capital is a great example of this.

It is.

They use surveys to map broad trends among men and women, but intentionally use the follow -up qualitative interviews specifically with women to highlight gender inequality and lift up historically disenfranchised voices.

The framework dictated the focus.

And the chapter gives a really helpful five -step process for drawing a visual diagram of these complex designs.

With the flowcharts.

Right.

First, you identify your qualitative and quantitative data.

Second, you draw boxes representing the larger overarching framework, like the stages of your social justice initiative.

Okay.

Third, you spot exactly where both data types occur within that framework.

Fourth, you determine if they are being merged convergently or connected sequentially.

And fifth, you draft the final flowchart models.

So to make sure I'm grasping this, a complex design isn't a replacement for the core designs.

No, not at all.

It's basically the chassis of a car.

The overarching experiment or social justice initiative is the chassis, and you just drop a convergent or sequential engine into it wherever you need the power to process data.

That analogy is flawless.

The chassis gives the car its shape and purpose, but the core mixed methods design is the engine making it run.

And drawing that detailed flowchart is absolutely essential.

Why is that?

Because whether you are defending a dissertation or pitching a proposal to a grant committee, you have to prove to skeptics that your findings are bulletproof.

That visual diagram shows them exactly where the engine is installed and how the gears connect.

With an entire menu of core and complex designs available, the next practical hurdle is figuring out how to actually choose the right one for your specific study and how to defend that choice.

The textbook boils this decision down to two main factors, intent and procedures.

Intent and procedures.

For intent, ask yourself what you were trying to achieve.

If your intent is to compare two databases to see if they match, use a convergent design.

If you want to build a new instrument by moving from qualitative to quantitative, use exploratory.

If you want to explain strange quantitative results with deeper qualitative context, use explanatory.

And then procedures.

Then look at your procedures.

Are you merging the data side by side concurrently?

Or are you connecting them in a chronological sequence?

Once you lock that in, you have to write what the text calls an integration statement.

Yes, very important.

This is a crucial declarative sentence that goes right into your methodology section.

It explicitly states your exact intent and procedure.

For example, integration involved explaining the initial quantitative survey results by following up with a qualitative interview phase.

Boom.

It is clear, it is defensible, and it leaves no room for confusion.

You should also consider the secondary logistical reasons for your choice, right?

Like what methodology is most common or respected in your specific academic field?

Exactly.

Or are you a solo grad student who needs the manageable step -by -step pacing of a sequential design?

Or are you part of a massive, well -funded research team that has the personnel to handle the chaotic multitasking of a concurrent complex design?

Speaking of academic traditions, I've seen older research papers where the authors capitalize the words quell or queonan in all caps, or use lowercase letters to visually show which database was prioritized or considered more important.

Uh, yes.

Is that still a best practice?

It is not, and the text explicitly updates this standard.

The latest edition of the book completely eliminated that capitalization notation.

Oh, interesting.

Why?

Because assigning priority implies an inherent authority and dominance of one data type over the other.

The field has evolved.

Modern mixed methods research views both quantitative and qualitative data as equally legitimate, vital partners in the search for truth.

Neither is the boss.

They are collaborators.

So let's bring it all back to where we started.

What is the ultimate takeaway from Chapter 10?

It's all about integration.

Right.

Mixed methods research isn't just about doing double the work to make your methodology section look thicker.

It is fundamentally about integration.

It is about actively constructing those magical maintainances, the one plus one equals three, that neither the high altitude drone footage nor the street level interviews could ever reveal on their isolated own.

Well said.

You need the scale of the quantitative and the soul of the qualitative to truly know the city.

And that leads to a really important thought to carry with you beyond just writing research proposals.

What's that?

The next time you read a stark numeric statistic in the news,

a plunging economic indicator, a surprising political poll, a new health metric,

ask yourself, what is the hidden qualitative human story behind that number?

And how might knowing that story completely change the way you interpret the data?

That is a fascinating question to leave off on.

A warm thank you from the Last Minute Lecture team for joining us on this Deep Dives.

Keep questioning the numbers, keep listening to the narratives, and we will tent you on the next Deep Dive.