BBuckley - 1

CSc 238

Human Computer Interface Design

Chapter 2Understanding The Problem:

Design Research

ABOUT FACE

The Essentials of Interaction Design

Cooper, Reimann, Cronin, and Noessel

2

DilbertScott Adams

3

DilbertScott Adams

4

DilbertScott Adams

5

DilbertScott Adams

6

DilbertScott Adams

7

DilbertScott Adams

8

DilbertScott Adams

9

DilbertScott Adams

The Goal Directed Design Process

Six Phases

Chapter 2

10

Research Modelling Requirements Framework Refinement Supportusers and of users and definition of user, definition of of behaviors, development

the domain use context business, and design structure form, and needs

technical needs and flow content

Ch.2 Ch.3 Ch.4 Ch.5 Ch.5

11

Why research?

Good quote:

“If she (the designer) does not have clear and

detailed knowledge of the users she is

designing for, the constraints of the problem

and the business or organizational goals that are

driving the design activities, she will have little

chance of success.”

12

Qualitative Research

• Qualitative versus Quantitative Research

• What do you get?

Understanding!

Behaviors, attitudes & aptitudes

Technical, business & environmental contexts

Vocabulary & other social aspects of the domain

How existing products are used

13

Qualitative Research – there is more…

Helps the progress of design:

• Provides credibility & authority to the design team –

with design decisions can be traced to results

• Team has common understanding of domain issues &

user concerns

• Management makes more informed decisions –

instead of relying on anecdote, guesses, biases, etc.

14

Qualitative Research

Why wouldn’t you want to know:

• How the product fits into the broader context of

people’s lives

• The goals that motivate people to use the product &

the basic tasks help people accomplish these goals

• The experiences that people find compelling & how

these relate to the product being designed

• The problems people encounter with the current ways

of doing things

Spending the time to deeply understand the user population can

provide valuable business insights that are not revealed

through traditional market research.

Strengths & Limitations of Quantitative methods

Strengths:

Determine what motivates people to buy

Segmenting market to get at types of buyers

by age, gender, education, zip code

Data that executives need to convince executives

Remembering that a successful product must be

“desirable, buildable… and also viable”

“Understanding whether somebody wants to buy

something is not the same thing as understanding what

he or she might want to do with it after buying it.”

15

Quantitative research is needed!

Forecasting market acceptance

“… if you know X people might buy a product or

service for Y dollars, it helps in evaluating ROI.”

Usage analytics…

“If users are lingering in an area of your website or

are not visiting any other areas, that is critical

information to have before a redesign.”

Note: … the problem with User Centered Design!

16

Fig 2-1 The relationship between quantitative and

qualitative, Goal –Directed Research

17

Market Research Analytics

(Quantitative) (Qualitative)

Can inform

Goal Directed

Design Research

(Qualitative)

Drives

Behavioral Models

(Personas)

Market Sizing Studies

(Quantitative)

Can be used

to generate

Qualitative research

Tool of choice for

characterizing user

behaviors and needs

18

What is

Goal-Directed Research

Qualitative research activities:

1. Kickoff meeting

2. Literature review

3. Product / prototype & competitive audits

4. Stakeholder interviews

5. Subject Matter Expert (SME) interviews

6. User and customer interviews/Observation

Fig 2-2 An overview of the

Goal-Directed design

research process

19

Kickoff Meeting

Literature Review

Product / Prototype

and Competitive Audits

Stakeholder Interviews

SME Interviews

User and Customer

Interviews / Observation

Consumer

Products

1.

2.

3.

4.

5.

6.

1. Kickoff MeetingFirst, Identify the key stakeholders

Typically:

Executive

Managers

and

Representative

contributors from:

• Development

• Sales

• Product

management

• Marketing

• Customer support

• Design & Usability

20

“A stakeholder is anyone with authority and/or

responsibility for the product being deigned.”

Kickoff Meeting

Questions for key stakeholders:

• What is the product?

• Who will/does use it?

• What do your users need most?

• Which customers and users are the most important to

the business?

• What challenges do the design team and the business

face moving forward?

• Who do you see as your biggest competitors? Why?

• What internal and external literature should we look at

to familiarize ourselves with the product and/or business

and technical domain?

21

2. “Homework” – Literature Review

Review any references to the proposed product or its

domain of use.

References such as:

• Internal documents

• Industry reports

• Web searches

Use the information in developing questions to ask

stakeholders and SME in order to acquire domain

knowledge, and vocabulary.

22

3. Product/Prototypes and Competitive Audits

• Provides a sense of the “state of the art”

• Familiarizes the team with the strengths and

limitations of what is currently available

• Provides general idea of the product’s current

functional scope

• Note interaction design… good and bad

• Strengths and weaknesses in current products

23

24

4. Stakeholder interviews

• Who?

Anyone with authority/responsibility for the product being

designed.

• What information is needed from them?

– Preliminary Product Vision – perspective on what is being

designed

– Budget / schedule

– Technical constraints & opportunities for new approaches

– Business drivers – what is to be accomplished

– Stakeholders’ perceptions of the user

“… important in developing “buy-in” and a common

language and understanding about the design,

management, and engineering teams.”

25

5. Subject Matter Expert (SME) interviews

• Expert on the domain in which the product will operate.

• May or may NOT be expert users

If they are, beware of “invested” in existing designs!

• They are not designers

• Suggestions:

SMEs are necessary/mandatory for complex/specialized domains

SMEs should be consulted throughout the design process

26

6a. Customer interviews

• Customers will make the purchase decision, so

• What you need to learn from these interviews:

Their Goals for purchasing product

Their Current frustrations

Their decision process

Their role in installation, maintenance & managing

of the product

Other domain related issues & vocabulary

27

6b. User Interviews

Main focus of the Design work!

What you might learn… “the effect that experience with

the current version of a product… on how the user

behaves and thinks about its use.”

What you need to know:

– When, why & how the product will be used

– Domain knowledge – user’s perspective

– Current tasks and activities

– Goals & motivations for using the product

– “Mental Model:” how they think about their job… and

expectation about the product

– Problems & frustrations with current products

28

User observation

Users omit much when merely describing their

behaviors … to avoid seeming to be dumb,

incompetent, or impolite.

“You can talk to users about how they think they

behave, or you can observe their behavior first-hand.

The latter … provides superior results.”

But, do both if you can.

Interviewing & Observing Users

Contextual inquiry

Context, Partnership, Interpretation, Focus

Improving on contextual inquiry

Shorten the interview process, Use smaller design teams,

Identify goals first, Look beyond business contexts

Preparing for ethnographic interviews

Identifying candidates, putting together a plan

Conducting ethnographic interviews

Interview teams & timing, Phases of ethnographic interviews,

Basic methods, After the interviews

29

Contextual Inquiry

Asking questions of the user… as if they are the “master

craftsman” and you are the new apprentice.

Context: Observe the user in their normal work environment

Partnership: as a collaborative exploration… alternating

between observing work and discussion of its structure and

details

Interpretation: Analyze results to uncover design implications

Focus: Let the interview wander with subtle direction to capture

data relevant to design issues.

30

Improving this “Contextual Inquiry”

• Shorten the interview process

Shoot for one hour, max, for each interview

About 6 well-selected users for each hypothesized role or

type

• Use smaller design teams

The same 2 to 3 designers in each interview

• Identify goals first

Do this before determining the tasks that relate to the goals

• Look beyond business contexts

Typically assumes a business product and a corporate

environment, although

Ethnographic interviews are possible in consumer domains

31

32

Ethnographic Interviews - Summary

What do you do?

• Intense observation

• Directed interviews

What technique?

• Contextual inquiry

4 Principles:

1. Context

2. Collaborative tone

3. Interpretation of users’ behaviors, environs & what they say

4. Interview is focused –knowing what is needed

Suggestions:

• Short interview process

• Use smaller teams

• Identify goals first

• Look beyond business concepts

33

Preparing for ethnographic interviews

The Goal

To understand the behaviors and rituals of people interacting with

individual products

Determine the sorts of users & potential user to interview

Create a an initial hypothesis to ID who to interview

Persona Hypothesis

Roles in business & consumer domains

Behavioral & demographic variables

Domain expertise versus technical expertise

Degree of understanding/knowledge may vary

Environmental considerations

Putting together a plan

34

Preparing for ethnographic interviews

Identifying Candidates

Persona Hypothesis

First, try defining the different kinds of users/customers

Hypothetical Persona (see handout)

Addresses the following questions:

1. What different sorts of people might use this product?

2. How might their needs & behaviors vary?

3. What ranges of behavior & types of environments need to be

explored?

Identifying candidates - Roles

Roles in business and consumer domains

For business products:

Roles are common sets of tasks & information needs related

to the different classes of users

Roles often map to job descriptions

For consumer products:

Look for patterns from users’ attitudes & aptitudes, lifestyle

choices, stage of life… whatever can influence their

behaviors

35

Behavioral and Demographic Variables

• To differentiate different types of users

Example: behaviors of shoppers using an online store:

• Frequency of shopping

• Desire to shop (from loves to shop to hates to shop)

• Motivation to shop (from bargain hunting to searching for just

the right item)

• Demographic variables – from user market research

ages, locations, gender, income of target markets for the product

Domain expertise versus technical expertise

Subject area knowledge pertaining to a product versus digital technology

• Naïve users will use only a small subset of a product’s

functions… “care must be taken to support domain-naïve

behaviors!

36

Environmental considerations

… in the case of business products

• Small companies: workers have broader set of

responsibilities & more interpersonal contact

• Large companies have layers of bureaucracy, workers

tend to be specialized

• Examples of environmental variables

– Company size (small to multinational)

– Company location (Asia, North America, Europe, etc.)

– Industry sector (electronics manufacturing, consumer

packaged goods, etc.)

– IT presence (ad to draconian)

– Security level (lax to tight)

37

Putting an interview plan together

“… each identified role, behavioral variable,

demographic variable, and environmental variable

identified in the persona hypothesis should be explored

in 4 to 6 interviews…”

“Consumer products typically have much more

variation in behavior, so more interviews typically are

required to really delineate the differences.”

38

39

Conducting ethnographic interviews

Designers work with stakeholders who have access to

users… the best way to make interviews happen!

Interview teams and timing

Authors favor 2 designers per interview – switching roles

Moderator drives the interview and facilitator takes notes

Phases of ethnographic interviews

Early interviews: open-ended questioning, not into the details

Middle interviews: questions more focused

Late interviews: confirm what was learned, tying up loose

ends

Basic methods

After the interviews

40

Conducting ethnographic interviews

Basic methods:

– Interview where the interaction happens

– Avoid a fixed set of questions

– Assume the role of an apprentice, not an expert!

– Use open- and closed-ended questions to direct discussion

– Focus on goals first, tasks second

– Avoid making the user a designer

– Avoid discussions of technology

– Encourage storytelling

– Ask for a show and tell

– Avoid leading questions

41

What kinds of questions to ask?(avoid a fixed set of questions)

Goal-orientedWhat makes a good day? bad day?

What activities waste your time?

What is most important?

What helps you make decisions?

System-orientedWhat are the most common things

you do with the product?

What parts do you use the most?

What are your favorite parts?

What drives you crazy?

How do you work around problems?

What shortcuts do you use?

Workflow-orientedWhat did you do when you first came

in today? After that?

How often do you do this? What things do you do weekly or monthly, but not every day?

What’s a typical day? What would be an unusual event?

Attitude-orientedWhat do you see yourself dong five

years from now?

What would you prefer not to do? What do you procrastinate on?

What do you enjoy most about your job? What do you always do first?

Important “reminders”

• Assume the role of an apprentice!

• Use open-ended (encouraging detailed responses) and

closed-ended (encouraging brief responses) questions to

direct the discussion

• Focus on goals first and task afterwards

• Note… user are not designers!

• Do not discuss technology (functionality)!

• Encourage story telling (…day in the life)

• Ask for “show and tell”

• Avoid leading questions

Would feature X help you?

You like X, don’t you?

42

Do you think you’d use feature X if it

were available

Does X seem like a good idea to you?

After the interview

Immediately “After each interview, teams compare

notes and discuss any particularly interesting trends

observed or specific point brought up…”

Create a binder of notes, review any recordings and

print artifact images.

Access and visibility of the information that has been

gathered is essential.

43

44

Other types of research

“The Goal-Directed approach helps answer questions about

the product at both the big-picture and functional-detail

level with relatively small amount of effort and expense.”

1. Focus groups

2. Usability & user testing (to validate product

design)

3. Card sorting (exploring the user’s mental model)

4. Task analysis

Focus Groups

• Representative users (segments of larger market)

• Asked a structured set of questions with a structured

set of choices

• Useful in gauging reactions to a product’s form

• Good at identifying products users own or are willing

to purchase, but…

• Not so good at identifying what users actually do with

the products.

45

46

Usability & User Testing

What is tested?

Production software, clickable prototypes, or paper prototypes

Why? To validate the design

When? Late in the design cycle.

Focus? Assessing first-time use.

Effective in determining:

• Naming of all the stuff

• Organization

• First-time use and discoverability

• Effectiveness

Lots of good references… and some examples on YouTube.

47

Card Sorting

• Objective: To better understand how users organize information

and concepts

• Tool to uncover this aspect of the users’ mental model

• Users sort a deck of cards containing a piece of functionality or

information related to the product

• Tricky analyzing the results…

• Assumes that user’s sorting correlate with the way in which

they want to end up using the product.

48

Task Analysis

Details on how users currently perform specific tasks.

… to identify pain points and opportunities for improvement.

Note.

“The way people do things today is often merely the product of the obsolete systems and organizations they are forced to interact with, and typically bear little resemblance to the way they would like to do things or the way they would be most effective.”

Task Analysis Study

The Study should be focus on the following:

• Why the user is performing the task (the underlying goal)

• Frequency & importance of task

• Cues – what initiates or prompts execution of the task

• Dependencies – what must be in place to perform the task, as

well as what is dependent on the completion of the task.

• People who are involved (roles & responsibilities)

• Specific actions that are performed

• Decisions made

• Information that is used to support decisions

• What goes wrong – error and exception cases

• How errors and exceptions are corrected

49

50

JARED M. SPOOL

Founding principal of User Interface EngineeringLeading research, training, and consulting firm specializing in

website and product usability

Journey to the Center of Design Pt. 1http://www.youtube.com/watch?v=WCLGnMdBeW8

Journey to the Center of Design Pt. 2http://www.youtube.com/watch?v=Zpy7PKIhXgU&feature=SeriesPlayList&p

=0ABEEA4DEAFF1E9D