Chapter 11: Factorial Designs

Welcome to Last Minute Lecture.

This free chapter overview is designed to help students review and understand key concepts.

These summaries supplement, not replace, the original textbook and may not be redistributed or resold.

For complete coverage, always consult the official text.

You ever ask a question, maybe about research, and you get that feeling the real answer is just,

well, it depends.

Oh, absolutely.

Like, do violent video games cause aggression?

Right.

Does it?

Or, you know, does it affect everyone the same way in every situation?

Exactly.

And that's where things get interesting, because real life is rarely about just one single cause, is it?

Not at all.

It's usually a mix of things.

And trying to look at those things just one by one, you only get part of the picture.

So today we're diving into a really powerful tool researchers use to get handle on those it depends situations.

We are.

We're tackling factorial research designs.

Yeah, and our main guide here is the chapter, Factorial Designs from Research Methods for the Behavioral Sciences, the sixth edition.

It's a great chapter.

Really lays out how these designs let researchers look at multiple influences, multiple variables, all at the same time.

It gives you a much richer understanding.

So if you've ever felt a bit lost trying to figure out how different things, different factors combine to create an outcome, or maybe you're just curious how scientists untangle these complex relationships, then this deep dive is definitely for you.

We're going to break down the main ideas, the methods, look at some actual examples, and we'll try keep it clear and straightforward.

No dense jargon.

Yeah.

The mission today is really to get you comfortable with factorial designs.

What are they?

How do they work?

And why are they so incredibly useful for understanding behavior?

The bottom line seems to be moving beyond just if something has an effect to understanding how different factors might interact, how they combine to create unique results.

That's where the real story often is, in that it depends.

Okay, let's get started then.

In simple terms, what exactly is a factorial design?

Okay, simply put, it's a study looking at more than one independent variable.

Those are the factors at the same time.

Seeing how they affect the dependent variable, the outcome you're measuring.

And these factors, they can be things the researcher changes, right?

Manipulates.

Yep.

Those are experimental factors.

Or they can be things that already exist, like age or personality traits.

Those are quasi -independent variables.

Got it.

And you mentioned each factor has levels.

What does that mean?

Right.

Levels are just the different values or conditions within a factor.

So if you're studying caffeine's effect,

your caffeine factor might have levels like say zero milligrams, 50 milligrams, 100 milligrams.

Those different amounts are the levels.

Okay, that makes sense.

Now I've seen this shorthand, like two by two or two by three by two.

What's that notation telling us?

Ah, yeah, that's really handy.

Each number in that notation stands for one factor.

Okay.

And the value of the number tells you how many levels that particular factor has.

So a two by two design?

Needs two factors.

And each one has two levels.

And a two by three by two.

That would be three factors.

The first has two levels, the second has three, and the third has two again.

It gives you a quick blueprint of the study's structure.

Right, right.

So the number of numbers is the number of factors.

And the numbers themselves tell you the levels for each.

How do you work out the total number of different conditions in the study?

You just multiply the levels together.

Simple as that.

For a two by two, it's two times two, four conditions.

Four groups or situations.

Exactly.

Or four conditions each participant goes through within subjects design.

For that two by three by two example, it'd be two times three times two.

Twelve conditions.

Okay, that calculation seems pretty important for planning.

It really is.

It tells you how complex things are getting, how many participants you might need.

Now the chapter uses an example.

Studying from paper versus a screen.

And also whether study time was fixed or self -regulated.

That's a two by two, right?

That's right.

Ackerman and Goldsmith study.

Can you break down the four conditions using factor and level?

Sure.

So factor one is, let's call it presentation mode.

It is two levels.

Paper and screen.

Okay.

Factor two is study time control, also two levels.

Fixed time and self -regulated time.

So the four combinations are paper with fixed time.

Yep.

Then screen with fixed time.

Paper with self -regulated.

And screen with self -regulated.

Those are your four treatment conditions.

Got it.

And a big point the chapter makes is that these designs often feel more like the real world.

Why is that considered an advantage?

Well because things in the real world rarely have just one single cause.

Factorial designs let us mirror that complexity a bit better.

We can see how multiple factors work together, how they interact.

It just gives a more ecologically valid picture.

More true to life.

Right.

Like whether reading on paper or a screen is better might depend on how much time I have maybe.

Exactly.

Or the type of material.

It's not always just one thing in isolation.

So that leads to the question, why not just run separate studies?

One for paper versus screen, another for fixed versus self -regulated time.

Why the complexity?

That's a really crucial point.

It might seem simpler, but you lose something huge if you do separate studies.

What's that?

You lose the ability to see the interaction between the factors.

The interaction.

Yeah.

You can't see if the effect of say paper versus screen changes depending on whether the time was fixed or self -regulated.

You miss those unique combinations.

Okay.

Bring that back to the paper screen example.

What insight about that combined effect would we miss?

Well, maybe paper is only slightly better with fixed time, but much better when people have to regulate their own time, especially on screen.

The original study found something like the self -regulation challenge was bigger on screen.

A factorial design lets you see that specific pattern, that interaction.

Separate studies would likely just show small overall effects and miss that crucial nuance.

Okay.

So it's about seeing how the factors work together, not just on their own.

That perfectly sets up talking about main effects and interactions.

Let's start with main effect.

What's that?

A main effect is basically the overall average effect of one single factor on your outcome variable.

You kind of average across all the conditions of the other factors.

So in the paper screen study, a main effect for presentation mode would be like, overall did people do better on paper than screen, regardless of the time condition?

Exactly.

And likewise, a main effect for time control would be asking,

overall did fixed time lead to different scores than self -regulated time, averaging across both paper and screen?

Think of it as the broad impact of each factor by itself.

And how do researchers usually spot these in the data?

Is it just looking at averages?

Pretty much.

They often calculate the average score for each level of a factor.

So the average score for everyone who used paper versus everyone who used screen or the average for fixed time versus self -regulated time.

If those overall averages are different enough, that suggests a main effect.

Gotcha.

So main effects are the big picture for each factor.

Now interactions, this feels like where the real power is.

What's an interaction?

An interaction is when the effect of one factor is different depending on the level of the other factor.

Basically, the influence of one variable changes across the conditions of another variable.

It's not consistent.

It's that it depends situation we talked about.

The combined effect is different than just adding up the main effects.

So back to paper screen and study time.

An interaction means the difference between paper and screen performance isn't the same for fixed time as it is for self -regulated time.

Precisely.

Maybe paper has a big advantage only when time is self -regulated, but there's no difference when time is fixed.

That pattern is the interaction.

The effect of presentation mode depends on time control.

The book used a drug interaction analogy.

That really clicked for me.

One drug changing how another one works.

That's a perfect analogy.

It's not just additive.

And graphs are really helpful here too, right?

How do they show interactions?

Yeah, line graphs are great for this.

You usually put levels of one factor on the horizontal axis, the outcome on the vertical axis, and then you draw separate lines for each level of the other factor.

And if those lines are parallel, basically running alongside each other?

Then probably no interaction.

The effect is consistent.

But if the lines are not parallel, if they cross or spread apart or pinch together.

That's your visual cue for an interaction.

The changing gap between the lines shows the effect isn't consistent.

Exactly.

Non -parallel lines scream interaction.

You can also see it in the data tables by comparing the differences between means across rows or columns.

If those differences change, that points to an interaction too.

Now, there's a really important warning in the chapter.

If you find a significant interaction,

be careful about interpreting the main effects by themselves.

Why is that so critical?

Because the interaction tells you that the main effect is, well, conditional.

It doesn't apply uniformly.

The overall average effect, the main effect, might be misleading because the effect could be strong in one condition, but weak, absent, or even reversed in another condition of the other factor.

The interaction qualifies the main effect, gives you the necessary context.

So the interaction is the more specific, more accurate story in that case?

Definitely.

Relying only on main effects when an interaction is present can really obscure what's actually happening.

And just to be clear, main effects and interactions are independent, right?

You can have one without the other.

Absolutely.

You can have, say, one main effect and no interaction,

or main effects for both factors, but still no interaction.

Or even like that TV example in the book, no main effects but a significant interaction.

Exactly.

Where maybe overall viewing time didn't matter, but what kids watched interacted with how much they watched to affect grades.

Any combination is possible.

They tell you different things about the data.

This is much clearer now.

The chapter then talks about different types of factorial designs, like between subjects versus within subjects.

Let's unpack those.

What's the difference?

OK.

So in a between -subjects factorial design, you have different participants in each condition, completely separate groups.

So for our two -by -two paper screen example, that would mean four different groups of students.

Yep.

One group gets paper -fixed time, another gets screen -fixed time, and so on.

Each person experiences only one combination.

The advantage there is no order affects like getting better or tired from doing multiple conditions.

Right.

But the downside is you generally need more people.

And you have to worry about whether the groups were truly equivalent to start with.

Exactly.

Individual differences between groups could be an issue.

Now contrast that with a within -subjects factorial design.

That's where the same people do everything.

Yes.

The same group of participants experiences all the conditions.

So each student would do paper -fixed, screen -fixed, paper self -regulated, and screen self -regulated.

The big win there is you need fewer people.

And you control for individual differences because you're comparing people to themselves.

Uh -huh.

Huge advantages.

But the major headache is order affects, right?

The order they do things in might mess up the results.

Fatigue, practice.

Precisely.

Order affects are the main concern.

And people might also drop out if it gets too long or taxing.

So trade -offs.

But what if you want a bit of both?

That's where mixed factorial designs come in.

You have at least one between -subjects factor and at least one within -subjects factor in the same study.

Can you give an example?

Sure.

The chapter mentions a study on acetaminophen, you know, Tylenol, and how it affects perceiving pain and pleasure.

They had one factor that was between subjects.

Some people got acetaminophen, others got a placebo.

Different groups.

Right.

But then they had a within -subjects factor.

Everyone, regardless of which pill they took, viewed and rated both positive images and negative images.

Ah.

So the drug was between subjects, but the image type was within subjects.

Clever.

Exactly.

It combines features of both approaches.

Okay.

Besides between and within -subjects, designs can also be experimental, non -experimental, or mixed in terms of the variables themselves.

Right.

So a purely experimental factorial design is one where the researcher manipulates all the independent variables, all the factors.

Like the original paper screen study, if the researchers controlled who got paper versus screen A and D, who got fixed versus self -regulated time, maybe by random assignment.

Exactly.

You're actively controlling and manipulating everything you think is a cause.

This lets you make stronger causal claims.

But sometimes you can't manipulate the factors, right?

Like studying age or gender differences.

Correct.

When all your factors are pre -existing characteristics or groupings, things you measure but don't manipulate, that's a non -experimental or quasi -experimental factorial design.

Like the study mentioned, looking at college class level and Facebook use patterns.

Perfect example.

They didn't assign students to be freshmen or sophomores or tell them how to use Facebook.

They just looked at existing groups and patterns.

You're looking for relationships, not direct causation.

And then you can mix these two.

Some manipulated factors, some measured ones?

Yes, that's very common.

A combined strategy.

Often researchers manipulate a situation or treatment,

experimental factor, and also include a participant characteristic like gender, personality, or experience level,

quasi -independent factor.

Like the video game violence study, they manipulated the game's violence level.

But they also included gender, which they obviously did manipulate, to see if the game's effect was different for males and females.

Those are sometimes called person by environment designs.

Exactly, P by E or person by situation.

How do different types of people react to different types of situations?

It's a very common and informative approach.

The chapter also mentions using time as a factor.

How does that fit in?

Time is usually a quasi -independent factor.

You don't manipulate time itself, but you measure your outcome at different points in time.

Like tracking learning gains before, during, and after an intervention.

Precisely.

Time becomes a factor with levels like pre -test, post -test, follow -up.

You see how the effects of your other factors change or evolve over time.

And the standard pre -test, post -test control group design.

The book says that's actually a two -factor mixed design.

It is.

When you think about it, you have one between subjects factor,

the group, treatment versus control.

Different people.

Right.

And you have one within subjects factor, time of measurement, pre -test versus post -test.

Same people measure twice.

Exactly.

So it fits that mixed design structure.

And whether it's quasi -experimental or truly experimental depends on if you use random assignment to the groups.

We focused mostly on two -factor designs.

What about higher order designs with three or more factors?

Yeah.

You can definitely have more than two factors.

A three -factor design, for example, might look at teaching method, factor A, student prior knowledge, factor B, and classroom noise level, factor C.

So maybe a two -by -two -by -two design.

Could be.

And the added complexity allows you to look for higher order interactions, not just two -way interactions like AXB or BXC.

But a three -way interaction, AXBXC, whoa.

Exactly.

A three -way interaction means that the two -way interaction between two factors is different depending on the level of the third factor.

So the way teaching method interacts with prior knowledge might be different in noisy classrooms compared to quiet ones.

Precisely that.

It gets layered.

Well, you can have four -way or five -way interactions.

It sounds incredibly complicated to interpret.

It really does.

The practical value often diminishes because explaining those super complex interactions becomes very difficult.

Most factorial research sticks to two or three factors for that reason.

You're looking for even more specific, it depends conditions.

Okay.

So how do researchers analyze all this data?

The chapter points to ANOVA.

Yes.

Analysis of Variance, ANOVA, is the workhorse statistic for factorial designs.

And the type of ANOVA depends on the design.

Right.

If all factors are between subjects, you typically use an independent measure's two -factor ANOVA.

If it's a mixed design, a mixed design ANOVA.

If all factors are within subjects, a repeated measure's ANOVA.

And what does the ANOVA tell you?

For two -factor ANOVA, it gives you statistical tests, usually F -ratios and P -values, for three things.

The main effect effector A, the main effect effector B, and the interaction effect,

AXB.

So it tests whether those apparent differences in means are likely real or just due to chance.

Exactly.

A significant result suggests the effect is statistically reliable.

One last design choice.

Using just two levels versus multiple levels for a factor.

What's the thinking there?

Good question.

Two levels is simpler.

Easier to run, easier to interpret the results.

You're just comparing A versus B.

But...

But it might oversimplify reality.

If the relationship isn't just a straight line, two levels might miss the curve or the peak.

Ah, okay.

Using multiple levels, say three or four, can give you a more detailed picture of the relationship.

You can see if it's linear, curved, et cetera.

But that adds complexity, right?

More conditions, more participants.

Definitely.

And interpreting interactions gets trickier with more levels.

So it's a trade -off.

Yeah.

Simplicity versus completeness of the picture.

Okay.

We've got the structure, the types, the analysis.

Now let's talk applications.

Why are these designs so darn useful in practice?

Oh, they have tons of uses.

The chapter highlights three big ones.

First, they're fantastic for expanding on previous research.

How so?

Well, say a study found an effect for a certain treatment.

You might wonder, does it work the same way for older adults as it did for younger adults?

Or does it work differently depending on how it's delivered?

So you could take the original study's factor and add a new factor, like age group or delivery method.

Exactly.

You replicate the original finding potentially, but you also extend it by seeing if that new factor changes things, all within one efficient study.

Like the Bartholow and Anderson study adding gender to the video game violence research.

Perfect example.

They could see if the known effect for males also applied to females or if there was an interaction.

It builds knowledge systematically.

Okay.

That makes sense.

What's the second major application?

Reducing variance in between subjects' designs.

Sometimes differences between people just create a lot of noise in the data.

Making it hard to see if your treatment actually worked.

If you suspect a characteristic like, say, IQ or anxiety level is contributing a lot to this noise.

You could screen people out.

You could.

But that limits who your results apply to.

A better way is often to measure that characteristic and include it as a second factor in your factorial design.

Ah.

So you'd have your main treatment factor and then maybe high anxiety versus low anxiety as a second factor.

Precisely.

This groups people who are similar on that characteristic together within each treatment level.

It reduces the error variance within each group.

Making it easier to detect the treatment effect.

Yes.

And as a bonus, you also get to see if that characteristic itself has a main effect and, importantly, if it interacts with your treatment.

Does the treatment work better for high anxiety people, for instance?

That's really smart.

You count for the noise and learn more about it.

Okay.

What's the third application?

Evaluating order effects within subjects' designs.

We mentioned order effects practice.

Fatigue are a big concern when people do multiple conditions.

Factorial designs offer a way to check for them.

You use counterbalancing, have different participants do the conditions in different orders, and then you treat the order of conditions as a second between subjects factor in your analysis.

So factor one is the within subjects treatment, like condition A versus condition B.

And factor two is the between subjects order.

Group one gets A, then B.

Group two gets B, then A.

Exactly.

Then you look specifically at the interaction between the treatment factor and the order factor.

What does that interaction tell you?

It tells you if the effect of the treatment depends on the order it was presented in.

If there's no interaction, order didn't matter.

If there is an interaction, it reveals the presence and potentially the nature of the order effect, whether it's symmetrical, just a general practice or fatigue effect, or non -symmetrical.

One specific order has a unique impact.

So you can actually measure and account for those pesky order effects.

Yes, it allows you to separate the true treatment effects from the potential confounding influence of the sequence.

It's a very powerful diagnostic tool.

Wow.

Factorial designs really do seem incredibly flexible and insightful.

They really are.

They let us embrace complexity, move beyond simple questions to understand the nuances of how different factors interplay to shape outcomes.

It's much closer to how things work in the real world.

So for you listening, getting a handle on factorial designs really boosts your ability

to critically evaluate research you come across.

Absolutely.

When you see studies with multiple variables, you'll recognize the possibility of interactions, you'll appreciate that the answer might indeed be, it depends, and you'll look for how the researchers explored that.

It just makes you a more sophisticated consumer of information, whether that's scientific studies, reports at work, or even just news articles trying to explain complex issues.

Hopefully this deep dive has given you a solid grasp of what these designs are all about.

Yeah, their structure, their power, and their applications.

So here's a final thought to take away.

Next time you run into one of those questions where the answer feels complicated, maybe pause and think.

Could this be about interacting factors?

What different elements might be playing off each other here?

Factorial designs really remind us that simple A causes B stories are often incomplete.

Exploring those interactions, those it depends scenarios, is where so many fascinating insights come from.

So we encourage you look for these designs when you're reading research and just generally think about how multiple factors might be weaving together to create the results you see in the world.

And that wraps up our detailed look at factorial research designs, drawing heavily on that chapter from research methods for the behavioral sciences.

We've covered the core ideas, the different types, how they're analyzed, and why they're such valuable tools.