Chapter 6: Research Strategies and Validity

Welcome to Last Minute Lecture.

This free chapter overview is designed to help students review and understand key concepts.

These summaries supplement not replaced the original textbook and may not be redistributed or resold.

For complete coverage, always consult the official text.

Think about all the information that comes your way every single day.

News headlines shouting about the latest study, claims about the best diet, tips on how to boost your memory.

Mm -hmm.

That's a lot.

Yeah.

We make decisions, right?

Big ones, small ones, based on what we hear, what we read, but how do we actually sort through it all?

How do we really know if a finding is, well, trustworthy,

if it applies to us?

That's the critical question, isn't it?

So today, we're going to dive deep into the basic mechanics of how research actually works.

We're looking at the different blueprints researchers use to ask questions.

And maybe even more importantly, how we can evaluate the quality, the trustworthiness of their answers.

Exactly.

Our guide for this is a chapter from a textbook, Research Methods for the Behavioral Sciences, the sixth edition.

It gives a really structured way to think about these core ideas.

It's a solid foundation.

It is.

So our mission today,

to kind of unpack this chapter for you, pull out the key insights about these research strategies and this idea of validity.

So you can better understand the studies you come across.

Right.

And feel more confident about what conclusions you can reasonably draw.

We'll cover the different ways researchers tackle questions, these really crucial concepts,

external and internal validity.

Which basically boil down to, do the results generalize and do they actually mean what the researcher claims they mean?

You got it.

So let's start at the beginning.

Research starts with a question.

Fundamentally, yes.

And what's really fascinating, and what the chapter emphasizes, is that the type of question you ask dictates the whole approach, the whole plan for finding an answer.

The research strategy.

That's the term they use.

Yeah.

The research strategy.

Okay.

Let's unpack that.

Strategies.

The chapter lays out five main ones.

You can think of them like different tools, maybe, for different jobs.

That's a perfect analogy.

You wouldn't use a hammer for a screw.

The question tells you which tool, which strategy is appropriate.

And it's not just some technical choice the researcher makes in a vacuum.

Not at all.

It's driven entirely by what they want to know and the specific kind of answer they're looking for.

Five strategies.

Let's walk through them.

Where does the chapter start?

It starts with probably the most fundamental approach,

the descriptive strategy.

Descriptive.

Okay.

Sounds straightforward.

It is.

The goal is simple.

Just describe individual variables as they exist naturally.

You're not looking at connections or relationships yet.

Just getting a picture of one variable.

So, like, finding out the average number of hours students study, or how much time people typically spend on social media.

Exactly.

Or average sleep time, percentage of people voting for someone.

The data you end up with is basically a list of scores for each person on that single variable.

It gives you a snapshot.

Precisely.

A snapshot.

And it's often the crucial first step, you know?

Before you can figure out how variables relate, you need to understand what those variables actually look like on their own in a particular group.

Sets the stage.

But, as you said, most research doesn't stop there.

Right.

Usually we want to know how things connect, how variables relate to each other.

Yeah, we're looking for relationships.

Consistent patterns where, you know, as one thing changes, another thing tends to change in a somewhat predictable way.

The chapter mentions different shapes these relationships can take.

It does.

It talks about linear relationships, like a straight line pattern or curvilinear, a curved pattern.

And direction, too.

Positive.

Where both variables go up together.

Or negative, where one goes up as the other goes down.

Like maybe more study time goes with higher grades?

That'd be positive.

Potentially.

Yeah.

Though the chapter also wisely points out that not all relationships fit neatly into these simple categories.

Real life is messy.

True.

And it highlights a really key distinction when you're looking at relationships, how you structure the data.

Ah, yes.

This is important.

You can either measure two or more things for each person in a single group.

Or...

Or you can compare scores on one variable between different groups of people.

And that difference in data structure separates the next few strategies.

It does.

Let's take the first approach, measuring two things for each person in one group.

That leads us to the correlational strategy.

Okay, correlational.

We hear this term a lot.

We do.

The goal here is specifically to describe the relationship between two variables.

You measure two scores for each individual and everyone's basically in the same boat, the same single group.

Like that Facebook time and GPA example they mentioned.

Measure Facebook hours and GPA for a bunch of students.

Exactly.

You get two numbers for each student.

Then you can plot that data, often on a scatter plot, and see if there's a pattern.

Maybe, you know, higher Facebook time tends to appear alongside lower GPAs.

So you see a pattern and association.

You see an association.

But...

And this is probably the single most important takeaway about this strategy, the one that helps you cut through so much misleading information.

Don't say it.

Correlation is not causation.

Bingo.

The correlational strategy does not try to explain why those variables are related.

It can't.

So finding that link between Facebook time and GPA doesn't mean, from that study, that using Facebook causes grades to drop.

Correct.

There could be other factors, right?

Maybe certain personality types are drawn to both more social media and less studying.

Maybe stress leads to both.

You just don't know from a correlational study alone.

And this is huge, because you see correlational findings reported all the time, as if they imply cause.

All the time.

Headlines love it.

Study links X to Y.

Understanding the limits of the correlational strategy is your shield against jumping to those causal conclusions.

Association is just association.

Okay.

So that's one way to look at relationships.

What are the other data structure comparing groups?

Right.

This brings us to a set of strategies.

Experimental, quasi -experimental, and non -experimental.

The chapter groups them because they all involve comparing scores from different groups.

Like comparing grades between kids from high income versus low income families, maybe?

That's a perfect example.

One variable income level, in this case, defines the groups.

Then you measure another variable grades for everyone and see if the average score differs between those groups.

And the chapter kind of ranks these strategies based on how much they can explain.

Yeah, based on their explanatory power, especially regarding cause and effect.

At the very top, the strategy designed specifically to pin down cause and effect.

That's the experimental strategy.

The gold standard for causation.

Pretty much.

The goal is an unambiguous cause and effect explanation.

Here, researchers aren't just observing.

They're actively intervening.

They do something.

They manipulate one variable, the independent variable, creating different conditions or treatments.

Then they measure a second variable, the dependent variable, to see if the manipulation caused a change.

Like the note -taking study.

Laptop versus pen and paper.

Exactly.

The researchers assigned students to use either a laptop or pen and paper.

They controlled that.

Then they measured learning outcomes.

Because they control the situation so carefully.

They could be pretty confident that the method of note -taking caused the difference in learning they observed.

Precisely.

That rigorous control, often involving random assignment to conditions, is what allows for strong causal claims in an experiment.

Okay, stepping down a bit from that level of certainty.

Quasi -experimental strategy.

Right.

Quasi means sort of, or resembling.

So this strategy tries to get at cause and effect, but it always has some kind of built -in flaw that prevents a completely unambiguous conclusion.

It still compares groups, though.

Maybe like that smoking cessation example.

One group gets a program, another doesn't.

Compare quitting rates.

Yes.

It involves group comparisons, often before and after an intervention for one group versus a comparison group.

But the key difference, the flaw, usually lies in how people end up in the groups.

So unlike a true experiment, the researcher doesn't fully control who gets what.

Often no.

They might use pre -existing groups, like employees in different office locations for that smoking study, or maybe people self -select into a treatment.

So if the smokers in office A who got the program quit more often than smokers in office B who didn't, you can't be sure it was the program.

Exactly.

Maybe the people in office A were already more motivated to quit, or maybe they differed in age or health consciousness before the study started.

That lack of control over assignment creates ambiguity.

You can't isolate the treatment as the sole cause.

Gotcha.

And the last one in this group comparison category,

non -experimental strategy.

This one's goal is similar to the correlational strategy.

Just demonstrate that a relationship exists, often by showing a difference between groups.

But crucially, it does not attempt to explain why.

So showing association,

like correlational, but the data looks different comparing groups instead of two measures on one group.

You got it.

The data involves comparing scores on one variable between two or more distinct groups, like the example of comparing verbal skill scores for girls versus boys.

Or maybe measuring one group at two different times, like anxiety before and after an exam period.

Yes, that fits too.

The key is that it lacks the manipulation and rigorous control of an experiment.

So you could show, maybe, that girls score higher on verbal skills in your sample.

Right.

You can demonstrate the difference, the relationship exists in your data, but the non -experimental strategy doesn't allow you to say why that difference occurs.

It doesn't explain the cause.

And the chapter makes a point to distinguish non -experimental from correlational again

Same basic goal, describe a relationship, don't explain cause, but different data structures.

Correlational,

one group, two or more variables measured per person, non -experimental, two or more groups compared on one variable.

And all these strategies naturally lead to different kinds of statistical analysis, right?

Absolutely.

The strategy determines the data structure, and the data structure determines the appropriate stats.

Descriptive might use means, percentages, correlational uses, well, correlations.

Comparing groups often involves t -tests or ANOVAs.

The stats match the strategy's goal.

So thinking of the big picture, just knowing which of these five strategies the study used gives you a huge head start.

It really does.

It's like a filter.

You hear about a study, you ask, okay, what kind of question were they asking?

Descriptive, correlational,

experimental, and boom, you immediately know the kind of conclusion, just association or potential cause that the study could possibly legitimately support.

Okay, so strategy is the overall plan, the goal, but there's more to planning a study than that, right?

Oh, definitely.

The chapter brings in two more layers that are important to distinguish, and sometimes people mix them up.

Research design and research procedure.

Strategy design procedure, how do they fit together?

Okay, so research strategy is the big picture approach and goal, like we just discussed, descriptive, correlational, experimental, et cetera.

What you want to accomplish.

Exactly.

Then research design is about how you're going to implement that strategy.

It involves making specific choices about the study's framework.

Like what kind of choices?

Like will you study a large group to generalize or focus intensely on a single individual?

Will you track the same individuals over time or compare different individuals at one point in time?

How many variables are you going to include?

It's the general blueprint for executing the strategy.

Okay, so if the strategy is we're going to try and show cause and effect, experimental.

The design might be we're going to use two different groups of participants, manipulate one variable between them, and measure one outcome variable.

That's a specific design choice implementing the experimental strategy.

Got it.

Blueprint level.

Yeah.

And then research procedure.

That's the nitty gritty.

The exact step -by -step, blow -by -blow details of how this specific study is carried out.

Like super specific.

Super specific.

Exactly how variables are manipulated or measured.

The precise number of participants, how they're recruited, the instructions they receive, how they move through the study sequence, the room they're in, the time of day.

All the unique details of one particular execution of a design and strategy.

Okay, that clarifies the layers really well.

It's not just I'm doing an experiment.

I'm using an experimental strategy with a specific design comparing different groups.

And here's the incredibly detailed procedure for how I'm actually going to run it.

You nailed it.

Strategy, design, procedure, layers of planning.

All right, so you can have the best strategy, design, and procedure lined up.

But how do we know if the results we get at the end are, well, real,

accurate, meaningful?

Now we get to the heart of evaluating research.

This is where the chapter brings in the concept of validity.

This sounds really important.

It is.

Validity is basically the standard, the criterion researchers use to judge the quality, the truthfulness, the accuracy of a study, and its conclusions.

It asks,

is this finding legitimate?

And the chapter distinguishes this from measurement validity, right?

Like whether a specific test measures what it says it measures.

Correct.

Measurement validity, which they likely covered earlier, is about the tool.

Study validity is about the truth and accuracy of the conclusions drawn from the entire study.

OK, so what's the core question validity asks?

It's essentially asking, can we confidently believe both what happened in the study and what the researchers claim it means?

Anything that casts doubt on either part is considered a threat to validity.

And there are two main types the chapter focuses on.

Yes, external validity and internal validity.

Two crucial pillars.

Let's tackle external validity first.

What's the gist?

External validity is all about generalizability.

To what extent can the results of this specific study be generalized beyond the unique circumstances of the study itself?

So can we apply the findings to other people, other settings, other times, maybe even slightly different ways of measuring things?

Exactly.

Every study happens in its own little bubble, specific people, specific place, specific time.

External validity asks how far the findings can reliably stretch outside that bubble.

We want research to matter in the broader world, right?

Definitely.

So a threat to external validity.

Is any characteristic of the study that limits that generalization something that makes you say, OK, maybe it worked here, but would it work there or for those people?

And the chapter breaks down generalization into a few categories.

Three main kinds.

First, generalizing from the sample used in the study to the larger population it's supposed to represent.

Like if you only study psychology undergraduates, can you really say the findings apply to all adults?

That's the classic example.

A major threat here is selection bias.

If the way you pick your participants systematically excludes certain types of people, your sample isn't representative and you can't generalize confidently.

Convenience sampling, just grabbing whoever is easy to find, is a big culprit here.

Huge.

The chapter specifically calls out the heavy reliance on college students in psychological research, noting they can differ significantly from the general adult population in ways that might matter.

Self -concept, conformity, peer relationships.

And volunteer bias, too.

People who sign up for studies might be different from those who don't.

Yes.

Research shows volunteers tend to be, for instance, more educated, have higher social class, maybe higher intelligence, more motivated by approval.

Systematic differences that can limit generalization.

Participant characteristics and generally demographics.

Absolutely.

If a study only uses participants of a specific age range, gender, race, socioeconomic status, etc., the findings might only apply to people with those characteristics.

The chapter gives a funny example.

Results from suburban Republican preschoolers might not generalize very far.

Fair point.

Even generalizing across species, from lab rats to humans, needs careful consideration.

Right.

You have to ask if the underlying processes are truly comparable.

Okay, so that's generalizing across people.

What else?

Second, generalizing across the features of the study itself.

Will you get the same results if you replicate the study slightly differently?

Or do the findings only hold up under the exact peculiar conditions of the original study?

Things like the novelty effect.

Exactly.

Participants might act differently just because being in a study is new or unusual for them.

Maybe they're excited, maybe anxious.

That artificial behavior might not reflect how they'd act normally, limiting generalization to real -world settings.

What about multiple treatment interference?

Yeah.

If participants go through several different conditions or treatments within the same study, the experience of an earlier condition can affect their performance in a later one.

Maybe they get tired, or maybe they learn and get better.

Practice effects.

So the results might only apply to people who experience that specific sequence.

Right.

And even experimental characteristics can be a threat.

If the results depend heavily on the specific personality or demographic of the person running They might not generalize when a different experimenter runs it.

Okay.

And the third area of generalization.

Generalizing across the features of the measures.

Do the findings depend critically on the specific way the variables were measured?

How could that happen?

One way is through sensitization, sometimes called assessment reactivity or pre -test sensitization.

The very act of measuring something can change the participant.

Like giving a pre -test on attitudes might make people think about the topic differently, so they react differently to a later persuasion attempt.

Precisely.

Or the example of self -monitoring just tracking your own behavior, like how often you feel depressed, might actually reduce the depression separate from any treatment.

The results then might only generalize to situations where people are also being assessed or are self -monitoring.

Got it.

What else under measures?

Generality across response measures.

You might measure fear using heart rate, but would you get the same result if you measured it by observing fearful behavior or by asking people how scared they feel?

The finding might be specific to one type of measurement.

And time of measurement makes a difference too.

Hugely.

Think about a program designed to help people quit smoking.

Does it work?

Well, the success rate might look great right after the program ends, but much lower six months later.

The timing of the measurement drastically affects the conclusion and its generalizability.

Okay.

So, external validity is really about asking,

how far does this finding travel?

Does it hold up with different people in different settings, using different measures at different times?

Does it apply outside the lab?

Yeah.

In you to me.

That's the essence of it.

Now flip the coin.

If external validity is about generalizing outward, internal validity is about looking inward.

Okay.

What's the focus here?

Internal validity applies specifically to studies that are trying to establish a cause

primarily experimental and to some extent quasi -experimental studies, it asks.

Is there one and only one unambiguous explanation for the results observed?

Can we be sure that the variable the researcher manipulated or the factor defining the groups is really the thing that caused the change in the outcome?

Exactly.

High internal validity means you can confidently rule out other possible explanations.

A threat to internal validity is any factor that allows for an alternative explanation.

The chapter introduces the idea of extraneous variables first.

Right.

These are simply all the other variables present in the study environment or within the participants that aren't the main variables of interest, the independent and dependent variables.

Like the room temperature, the lighting, the time of day, the participant's mood, their IQ.

The experimenter's tone of voice.

There are potentially thousands in any given study.

Most are harmless background noise.

But some extraneous variables can become problematic.

Yes.

They become confounding variables.

This is a critical concept for internal validity.

What makes an extraneous variable a confound?

A confounding variable is an extraneous variable that changes systematically along with the two variables you are studying, for example, your independent and dependent variables.

And because it varies in sync with your independent variable, it provides a plausible alternative explanation for any observed change in your dependent variable.

Okay, let's use that music study example again.

If the pleasant music condition always happened in a bright, cheerful room and the unpleasant music condition always happened in a dim, drab room.

And you find that performance is better in the pleasant music condition.

You don't know if it was the music or the room environment that caused the difference.

The room environment, an extraneous variable, changed systematically with the music type, the independent variable.

So the room environment is a confounding variable.

It confuses the interpretation.

Precisely.

It confounds the effect of the independent variable.

The chapter also uses that classic Coca -Cola versus Pepsi example where the cups were labeled Q &M.

If people preferred the soda in cup M, was it the soda or the letter M?

The letter is confounded with the type of soda.

So the core principle for achieving internal validity is?

To ensure that the only thing that differs systematically between your treatment conditions or the groups being compared is the independent variable itself.

Everything else should either be held constant or randomized across conditions.

Got it.

Control the potential confounds.

The chapter groups these confounding variables into categories too, right?

Helps organize the threats.

First, environmental variables.

These are characteristics of the study environment itself that might differ systematically across conditions.

Like the room size, time of day, maybe the gender of the experimenter if different experimenters run different conditions.

The room color and soda label examples fit here?

These can be threats in almost any study if not carefully controlled.

Second category.

Participant variables, also known as individual differences.

These are pre -existing characteristics that participants bring with them to this.

Things like age, gender, IQ, personality, prior experience, motivation.

And these become threats when you're comparing different groups of people.

Yes, especially if the groups weren't formed using random assignment.

If the participants in one group are, on average, systematically different from the participants in another group on some characteristic before the study even begins, that difference is a confound.

Like if one group just happens to be smarter or older or more motivated than the other group.

Right.

Then any difference you see in the outcome could be due to that pre -existing difference, not your treatment or the variable defining the groups.

This is the fundamental weakness of many non -experimental and quasi -experimental designs that use pre -existing groups, like comparing different classrooms or different office locations.

You can't be sure the groups were equivalent to start with.

Exactly.

Third category.

Time -related variables.

These are threats specifically for studies that measure the same group of participants at different points in time, like before and after an intervention, or across multiple treatment conditions.

What kind of things change over time?

Anything that happens between the measurement points can potentially be a confound if it affects participants' scores.

This could be things internal to the participant,

like getting tired, fatigue, getting better at a task through repetition, practice effects, changes in mood, or just natural maturation or development.

Or things happening outside the study.

Yes.

History affects major external events happening during the study, like a pandemic, an election, even just a change in weather that could influence the outcome variable.

If you're measuring anxiety in September and then again in December, the approaching holidays or end -of -semester stress could be a time -related confound.

So internal validity is all about rigorous control and designed to isolate the intended cause and rule out all of these alternative explanations, environmental, participant differences, time -related factors.

You got it.

It's about achieving that clear, unambiguous explanation for the results, particularly when claiming cause and effect.

Now the chapter points out something really practical.

There's often a tension between internal and external validity.

A classic trade -off.

It's a fundamental challenge in research design.

Oh, so.

Well, think about it.

To get high internal validity, especially for cause and effect claims, you need really tight control.

You want to eliminate or neutralize all those potential confounding variables.

Which often means conducting the study in a very controlled, maybe artificial laboratory setting.

Right.

But that very artificiality, that high level of control, can sometimes make the situation so different from the real world that you start to wonder if the results would actually generalize.

It can hurt external validity.

And conversely.

If you prioritize external validity, you want your study setting and conditions to be as natural and realistic as possible.

Maybe conduct it in the field.

But the real world is messy, full of uncontrolled, extraneous variables.

Exactly.

So making it more realistic often means sacrificing control, which opens the door for confounding variables and potentially weakens internal validity.

So researchers have to make a choice or find a balance.

They do.

The decision often hinges on the primary goal of the research.

Is it more important to demonstrate that a cause and effect relationship can exist under pristine conditions,

prioritizing internal validity?

Or is it more important to show what typically does happen in a complex, real world setting, prioritizing external validity?

Both goals are valuable.

The chapter also revisits the idea of artifacts.

What are those again?

Artifacts are sort of external factors accidentally introduced into the research situation that can distort the results.

They're problematic because they can threaten both internal and external validity simultaneously.

Okay.

Examples.

Two major ones are experimenter bias and demand characteristics participant reactivity.

Experimenter bias.

Yeah.

That's when the researcher's own expectations influence the outcome.

Yes, often unconsciously.

They might treat participants in different conditions slightly differently or interpret ambiguous data in a way that fits their hypothesis.

How does that threaten both validities?

It threatens external validity because the results might only be obtainable when that biased experimenter runs the study.

It threatens internal validity because the experimenter's influence provides an alternative explanation for the results.

It looks like the treatment worked, but maybe it was just the experimenter's subtle nudges.

And using double -blind procedures where neither the participant nor the researcher interacting with them knows the condition assignments helps minimize this.

It's a key strategy, yes.

Okay.

And the other artifact,

demand characteristics and reactivity.

Demand characteristics are any cues or features of the study that might inadvertently suggest to the participants what the purpose or hypothesis is, hinting at how they're expected to behave.

And reactivity is the broader phenomenon where participants simply act differently because they know they're being observed because they're in a study.

Right.

They might try to be a good participant and help confirm the hypothesis or a negative participant and try to mess things up or just act unnaturally self -conscious.

How does this affect validity?

Well, it hurts external validity because if people are acting artificially due to being studied, the results might not reflect how they behave in real life.

And it hurts internal validity because their reactive behavior rather than the independent variable could be the reason for the observed changes or differences.

This seems like a bigger risk in a lab setting than out in the field.

Generally yes, though reactivity can occur in field settings too if people realize they're being watched.

Researchers try various techniques to minimize reactivity like unobtrusive observation or ensuring anonymity.

The chapter also briefly mentions researchers sometimes using exaggerated variables.

Yeah, like deliberately making the difference between conditions very large, maybe comparing the effects of a quiet room versus an extremely loud one just to maximize the chances of finding any effect.

That's the downside.

The results might be real, but they might only apply to those extreme conditions.

It could limit the external validity when trying to generalize to situations with more subtle differences.

So if we bring it all back together,

understanding validity really helps us see the strengths and weaknesses of the different research strategies we started with.

Absolutely.

Descriptive, correlational, and non -experimental strategies, because they often occur in more natural settings and exert less control, tend to have potentially higher external validity.

You're seeing things as they are.

But they generally have lower internal validity because that lack of control means lots of potential confounds.

And they aren't designed to provide unambiguous cause and effect explanations anyway.

Where is the experimental strategy?

It's designed for high internal validity through rigorous control and manipulation, giving the best shot at establishing cause and effect.

But that very control can sometimes create artificiality, potentially limiting external validity.

And quasi -experimental falls somewhere in between.

Often yes.

It tries for causal explanation, but has known internal validity flaws, while its setting might be more naturalistic than a pure lab experiment.

So when you encounter research, it's crucial to ask, what was the goal here?

Were they aiming for maximum control to isolate a cause, even if artificial?

Or were they aiming for real -world relevance, even if the causal picture is murky?

Knowing that helps you interpret the findings appropriately.

Neither approach is inherently better.

They just answer different kinds of questions and have different limitations.

Wow.

Okay, we've definitely taken a deep dive here.

We've gone through the main research strategies, descriptive, correlational, experimental,

quasi -experimental, non -experimental, how they differ in their goals and data.

And then we really unpacked the critical concepts of external validity.

Can the results generalize?

And internal validity is the causal explanation, clear and unambiguous.

Plus, all the various threats that researchers have to constantly watch out for selection bias, confounds, reactivity, experiment or bias.

It's a lot to manage.

It is.

Doing good research is challenging.

But understanding this stuff is just incredibly powerful for us, for anyone consuming information.

Yeah.

It gives you a lens, a framework, to look past the headlines and evaluate claims more critically.

Absolutely.

You're not just passively accepting conclusions anymore.

So the final thought here, the takeaway for you listening, what does this all mean day to day?

Well, maybe next time you see that news report or that ad citing a study, or even hear a friend talk about something proven by research.

Pause for a second.

Ask yourself, what kind of question was the study really trying to answer?

What was the strategy?

Was it set up to show cause or just association?

And think about validity.

How representative were the participants?

What might have been confounded?

How artificial was the setting?

Give it all that.

How much confidence should I really place in this finding, applying broadly or being the only explanation?

Asking those questions shifts you from being a passive recipient to an active critical thinker about the research you encounter.

And with that, we've covered the core concepts from this chapter on research strategies of validity, the different approaches, the definitions of external and internal validity, the numerous threats and how these ideas connect.

A really solid foundation for understanding research methods.