Welcome to Last Minute Lecture.
This free chapter overview is designed to help students review and understand key concepts.
These summaries supplement not replaced the original textbook and may not be redistributed or resold.
For complete coverage, always consult the official text.
So we spent all our time perfecting the tools of research right.
We're setting up observation studies, writing the perfect questionnaire, nailing the interview technique.
But what happens once the study is over and you are just staring at, you know, hundreds of pages of transcribed interviews, screen recordings, and these dense spreadsheets?
Yeah, and that's really the moment of truth in interaction design.
We're diving into that essential but often messy phase that comes after data collection,
analysis, interpretation, and presentation.
Our mission today is to give you the shortcut through that massive pile of raw data, teaching you how to extract meaningful findings, not just, not just summaries.
Absolutely.
Analysis is where we turn observation into actual actionable design requirements.
So to start, we have to recognize the basic ingredients we're working with, which is section 9 .2, qualitative versus quantitative data.
Right.
So quantitative data is all about magnitude, anything in the form of numbers or that can be, you know, easily turned into numbers.
Think how long a task took or the number of errors, a participant's age.
And qualitative data is the narrative, the descriptive stuff.
It's the words, the images, quotes from a user expressing frustration or notes describing their thought process.
It's the rich description of the nature of the experience.
Now, a common misconception, one we need to shut down immediately, is that your collection method dictates your data type.
Right.
Like if you do a questionnaire, it must be quantitative.
Exactly.
But it's wrong.
A single questionnaire can ask for a time measurement, which is quantitative, and then can ask for open -ended comments about the experience, which is qualitative.
They really exist side by side.
That makes perfect sense.
So if quantitative focuses on size, say, the average height is 5 feet 11 inches,
qualitative focuses on patterns and themes, like describing that average person as middle -aged or maybe having sophisticated taste in footwear.
Precisely.
And this leads us to the very first warning sign and analysis, which the text highlights in box 9 .1, the use and abuse of numbers.
Ah, yeah.
Just because you have numbers doesn't mean you can manipulate them carelessly.
This is where we see that, you know, classic trap of percentages.
You might hear a product manager proudly announce, 50 % of our early access users found the checkout flow confusing.
And that sounds like a crisis.
It does.
But if they only tested four people, saying two out of four users had problems, provides infinitely clearer context.
The numbers are identical, but the perception is completely different.
So what's the guideline?
The guideline here is stark.
Don't use percentages unless your data set has at least 10 data points, and you should always provide the raw numbers anyway.
Context is everything.
That is really actionable advice.
OK, so once we've gathered our massive data set and kept that percentage warning in mind, we can't just analyze it yet.
No.
We have to go through that painful phase of processing, the cleanup phase.
This initial processing, section 9 .2 .1, is just crucial.
If you ran interviews, your quick notes need to be expanded immediately while they're fresh in your mind.
And the recordings.
And if you recorded audio or video, you face transcription.
This is notoriously difficult because people speak so much faster than you can type, and the context is constantly shifting.
So is transcribing really worth the effort if it's so labor -intensive?
Oh, it absolutely is, especially for any kind of fine -grained analysis, because you need the exact words, the pauses, the cadence.
For questionnaires, we clean the database filtering results, maybe focusing only on people under 16, or looking at specific survey questions only.
And when dealing with complex observation studies, where you have notes and photos and screen logs all over the place.
You need to synchronize everything.
You have to weave those different logs and notes into one coherent narrative, the audio, the video of the user's face, the screen capture, so you can review the activity precisely as it unfolded.
Okay, so once we have clean, processed, quantitative data, we dive into the basics, which is 9 .3, and that means looking at averages.
But we need to be wary because there are three distinct averages, not just one.
Exactly, the three A's.
Tell us about them.
So you have the mean, the median, and the mode.
The mean is the standard average, we all know, add everything, divide by the count.
The median is the middle value once all your numbers are ranked in order.
And the mode is simply the most frequently occurring number.
Most of the time, they are pretty close.
And the mean is fine.
But let's look at that classic caution case.
Imagine you're reviewing compensation data and your figures are 2, 2, 2, 2, and then 450.
Right, in that data set, the median is 2, the mode is 2, but the mean suddenly jumps to 91 .6.
Wow.
So if you only report that the average compensation is 91 .6, you have wildly misrepresented the typical data point.
This is why you have to understand your data's distribution before you just pick an average to use.
That's such a powerful reminder that simple math can be deeply misleading and the structure of our original questions from Box 9 .2, that also heavily influences what kind of analysis we can even do.
Exactly.
If you ask an open -ended question like, what do you think of this new app?
You get rich, varied, qualitative data.
But if you switch to a fixed alternative or a closed -ended question, like in that memory mirror study.
Yes, asking if the virtual try -on was realistic, clunky, or distorted, then you're just generating tallies and percentages which are quantitative.
And let's not forget, Likert scales, that common 5 -point scale from strongly agree to strongly disagree, which changed the data type again, letting you quantify sentiment.
So when we dump all these numbers into a spreadsheet,
typically Excel, we are looking for two things.
Patterns and the outliers.
Outliers are the values that are significantly different from the majority.
The text shows this so vividly with that error rate data from a photo -sharing app.
A scatter graph immediately flags one user with an error rate of 9 when everyone else is under 5.
But wait, isn't finding and removing outliers just… just cheating?
Skewing the data to tell the story we want?
That's a great skeptical question.
It's not cheating, but it absolutely must be justified.
Outliers often indicate a special circumstance.
A user with technical difficulties, maybe they got an incorrect instruction or they had a totally unique behavior.
They shouldn't be blindly removed but investigated separately.
They often hold really unexpected and crucial insights.
Okay, moving on to section 2.
Basic qualitative analysis 9 .4.
Analyzing narrative images and transcribed feelings seems far harder than just running averages.
What are our core techniques here?
We have three basic approaches, identifying themes, categorizing data, and critical incident analysis.
The goal is always the same, find common features, and more importantly, the surprises.
And we use two analytic processes to achieve this, inductive versus deductive.
Right, inductive analysis means the themes and concepts sort of emerge from the data.
You don't know what you're looking for until the data tells you.
Deductive analysis, on the other hand, means you come in with a pre -existing theory or framework like an academic model and you use that to sort your data.
I think inductive analysis is best visualized with the affinity diagram technique in 9 .4 .1.
The affinity diagram is a fantastic technique.
You take all your individual thoughts and quotes, often written on literal sticky notes, even now in the digital age, and you inductively start grouping them based on similarity.
And that grouping process creates a hierarchy of themes.
Exactly.
It's widely used in contextual design, like for figuring out what users really need in a co -watching video app.
So if the affinity diagram is inductive, let's look at how categorizing data, 9 .4 .2, is often deductive.
This is where we apply codes based on a known framework.
That's it.
Take the Nestor Navigator study, which analyzed usability problems.
They developed a categorization scheme beforehand, dividing problems into broad buckets, like interface problems and content problems.
And then they coded the transcripts?
Yep.
They marked up transcribed excerpts with specific notations, like UP 1 .1, 1 .7, to quantify exactly how often and where usability problems were happening.
This is how you take messy qualitative transcription and start to make it quantifiable.
And finally, we have critical incident analysis in 9 .4 .3, which really focuses your effort.
This is all about isolating and studying the most pivotal or significant events, either highly desirable or highly undesirable, in a lot of detail.
The technique actually originated with the U .S.
Army Air Force's study of pilot performance back in the 1950s.
A great modern example is the Pokemon Go study.
Researchers used this technique to survey experienced players about key incidents.
And they found that beyond the expected increase in physical activity, the game spurred social interaction and genuine positive emotional change in users.
Contextual factors and emotional state are hugely powerful drivers that simple logs just miss.
Now we step up to depth dramatically with Section 3.
Advanced analytical frameworks?
That's 9 .5.
These are tools designed to drill down into the much finer points of human interaction.
We can start with Conversation Analysis, or CA.
This focuses on the micro -level mechanics of conversation turn -taking, pauses, syntax, to understand how people talk to each other and to their technology.
The analysis of the Amazon Echo and Alexa family fragment is incredibly revealing here.
Researchers looked at the syntax, where the overlaps happened, where the pauses were, and they realized the interaction with Alexa was constantly interwoven with other ongoing family activities.
Like telling a child to finish eating.
Exactly.
The insight was the punchline.
Based on the structure of the dialogue, the family was interacting with Alexa in a way more like instructing a helper than actually conversing with an entity.
The technology was just being folded into the domestic chaos.
That structural insight is so powerful.
Let's jump to the macro -level now with Grounded Theory, 9 .5 .5.
This is for when you want to develop a new robust theory, not just analyze existing data.
Right, Grounded Theory works by alternating between data collection and analysis.
You're letting the theory build itself up, grounded in the empirical evidence, it's highly iterative, and it's defined by three coding stages.
First, you have Open Coding, where you discover preliminary categories in the data.
Then Axial Coding, where you flesh out those categories and relate them systematically.
And finally, Selective Coding, where you integrate everything around one central theoretical backbone.
The idle game study is a perfect use case.
Researchers used open coding on observations of games like Cookie Clicker, constantly iterating until they developed a new taxonomy that defined these incremental games based on their underlying rules and structure.
Which was something previous taxonomies had just completely missed.
And for massive organizational projects, we use systems -based frameworks.
The Distributed Cognition of Teamwork, or DCOTE for example, is excellent for large -scale cognitive phenomena, like studying ICU nurses and infusion pumps, to understand how information flows and artifacts are used across an entire team.
That covers an immense amount of analytical depth, from a single word in a transcript to an entire hospital floor's workflow.
Let's wrap up with the tools that support this work in 9 .6, and the final act.
Presentation in 9 .7.
Software is your friend here.
For organizing, coding, and searching vast amounts of qualitative data, tools like in vivo and deduce are indispensable.
If you are doing serious number crunching, you move to statistical packages like SPSS or SAS.
When presenting findings, the audience absolutely dictates the format.
You might use structured notations like UML diagrams, which offer incredible precision, but if your audience isn't familiar with them, the meaning can be completely lost.
That's why we so often rely on stories, 9 .7 .2.
Stories are intuitive, they add authenticity, especially when you back them up with multimedia clips.
We can use participants' actual narratives, or we can construct powerful stories from repeated snippets of behavior found across the whole data set.
And a good story often communicates the why much faster than a chart communicates the how much.
It really does.
So to bring this all home, let's just revisit the critical importance of careful interpretation in 9 .7 .3 and avoiding the pitfalls we discussed at the start.
We have to ensure our presentation doesn't overemphasize selective results.
Remember those three quick, relatable mistakes.
First, misrepresenting a small sample size.
Don't say half the users don't use the feature if your end is only four.
Second, don't overgeneralize from a single event.
One designer walking for 10 minutes does not automatically mean significant time is wasted across the department.
And finally, context justifies your numbers.
If you log 1 ,000 hours of website activity and can state that the help files were used less than 1 % of the time, that tiny percentage is justified because the data set it's drawn from is massive and logged accurately.
Analysis determines what you know but the interpretation and presentation, well, they determine what others believe and what actually gets built.
Today, we journeyed from the raw material of qualitative and quantitative data through the mathematical cautions you need with percentages and the three A's to the inductive power of affinity diagrams and the rigorous structure of advanced frameworks like grounded theory and conversation analysis.
Ultimately, the way results are framed, choosing what percentage to highlight, which average to display, or using words like most or all is a powerful design choice and it has ethical implications.
So we leave you with this final thought.
If choosing the right visualization or narrative can drastically change the perception of a product's success,
how can we as informed consumers and future designers ensure that the data we rely on is being presented to inform rather than simply to persuade?