how to conduct usability studies: a librarian primer

42
How to Conduct Usability Studies A Librarian Primer Tao Zhang, PhD Digital User Experience Specialist Purdue University Libraries [email protected] Sponsored by ALA RUSA #RUSAUX Ilana Barnes Business Information Specialist Purdue University Libraries [email protected]

Post on 17-Oct-2014

592 views

Category:

Technology


0 download

DESCRIPTION

Slides used for RUSA webinar on September 4, 2013

TRANSCRIPT

Page 1: How to Conduct Usability Studies: A Librarian Primer

How to Conduct Usability Studies A Librarian Primer

Tao Zhang, PhD Digital User Experience Specialist Purdue University Libraries [email protected]

Sponsored by ALA RUSA #RUSAUX

Ilana Barnes Business Information Specialist Purdue University Libraries [email protected]

Page 2: How to Conduct Usability Studies: A Librarian Primer

Learning Objectives

2

•  Participants will be able to describe the usability concept and it's applications to libraries.

•  Participants will be aware of the basic work!ows and techniques needed when doing usability tests.

•  Participants will be aware of resources for usability studies and where to "nd out more.

Page 3: How to Conduct Usability Studies: A Librarian Primer

Agenda

3

•  What is a usability study? •  Usability study process •  Conducting the study •  Analyzing Results •  Communicating •  Where to "nd out more

Page 4: How to Conduct Usability Studies: A Librarian Primer

About Us

Ilana Barnes Tao Zhang

4

Page 5: How to Conduct Usability Studies: A Librarian Primer

Tell us about you!

What type of library do you work in? A)  Public B)  Academic C)  Other

How familiar are you with Usability Studies/Tests? A)  Very familiar B)  Somewhat familiar C)  Not familiar but excited to learn!

Chat box: Areas you especially want us to talk about

5

Page 6: How to Conduct Usability Studies: A Librarian Primer

What is User Experience?

6

Page 7: How to Conduct Usability Studies: A Librarian Primer

De"nition

Usability study refers to “evaluating a product or service by testing it with representative users. “

- http://www.usability.gov/how-to-and-tools/methods/usability-testing.html

7

Page 8: How to Conduct Usability Studies: A Librarian Primer

How UX "ts into Reference Services

•  Decisions that user experience research can help you with: •  What should our libraries front page look like? •  What do our undergraduates expect from the library? •  What would we like our undergraduates to expect from the library

that they do not know or understand or need?

•  Understanding your users aims and needs help you help them and show your impact at the same time.

8

Page 9: How to Conduct Usability Studies: A Librarian Primer

How to plan your usability study topic

•  What are my objectives? •  How much time do I have? •  How many teammates do I have? •  Who am I presenting this to? •  Is this for an existing product?

9

Page 10: How to Conduct Usability Studies: A Librarian Primer

When to Test

•  Early or middle stages of development •  Types:

Exploratory | Assessment | Comparison | Validation

•  Start preparing at least three weeks before actual test

•  DO pilot tests

10

Page 11: How to Conduct Usability Studies: A Librarian Primer

Usability Study Process

11

Recrui'ng   Preparing   Conduc'ng   Analyzing   Presen'ng  

Choosing Features Creating Tasks Preparing Test Script

Page 12: How to Conduct Usability Studies: A Librarian Primer

Recruiting

•  Determining the target audience (behavior) •  Finding representative members

–  Behavioral criteria, demographics –  Avoid sampling bias

•  Convincing participants to come –  Incentives –  Informed content (duration of study, data to collect)

•  Scheduling –  Reminding –  Deal with no shows

12

Page 13: How to Conduct Usability Studies: A Librarian Primer

Recruiting

Best people to invite: •  People who are going to need the service in the

near future •  People who have used a competing/previous

version service in the recent past Number of participants: •  5-8 •  Also consider representativeness and tasks

13

Page 14: How to Conduct Usability Studies: A Librarian Primer

Choosing Features

•  Input from: Development team | Stakeholders | Users •  Content inventory & task analysis

14

Page 15: How to Conduct Usability Studies: A Librarian Primer

Creating Tasks

Tasks should be: •  Reasonable and doable •  Described in terms of end goals •  Speci"c Run through tasks to see how long each will take Determine what quantitative data to collect

15

Page 16: How to Conduct Usability Studies: A Librarian Primer

Discussion

You need to do a usability study of your library’s website, because users "nd it difficult to "nd information they want on the website. What are some possible tasks you can use in the usability study?

16

Page 17: How to Conduct Usability Studies: A Librarian Primer

Writing A Study Script

•  Introduction (5 min) •  Warm-up interview, demographics survey (5 min) •  Scenarios and tasks (20-30 min) •  Interview (10-15 min) •  Wrap-up (5 min)

17

Page 18: How to Conduct Usability Studies: A Librarian Primer

Conducting the Study

The physical layout with observer

18

Page 19: How to Conduct Usability Studies: A Librarian Primer

Conducting the Study

The physical layout without observer

19

Page 20: How to Conduct Usability Studies: A Librarian Primer

Conducting the Study

TechSmith Morae www.techsmith.com/morae.html

20

Recorder  

Observer  #1  

Participant Test Moderator

Observer

Manager  

Observer  #2  …  

Page 21: How to Conduct Usability Studies: A Librarian Primer

Conducting the Study

21

http://silverbackapp.com/

Page 22: How to Conduct Usability Studies: A Librarian Primer

Conducting the Study

Testing on mobile devices

22

IPEVO Ziggi-HD High-Definition USB Document Camera

DIY Mobile Usability Testing - SXSW Interactive 2012 http://www.slideshare.net/beleniq/diy-mobile-usability-testing-sxsw-interactive-2012

Page 23: How to Conduct Usability Studies: A Librarian Primer

Moderation

•  Follow script but allow some !exibility •  Encourage participants to “think aloud” •  Ask about expectations •  Ask “why” a lot •  Suggest solutions when participants are stuck •  Discuss with observer right after test

23

Page 24: How to Conduct Usability Studies: A Librarian Primer

Two Basic Techniques

Observation Interview

24

Page 25: How to Conduct Usability Studies: A Librarian Primer

Observation

•  Watch, Understand •  Limit the time users have to perform a task •  Encourage users to think aloud with neutral

prompts •  Elicit detailed information

25

Page 26: How to Conduct Usability Studies: A Librarian Primer

Interview

•  Introduction •  Warm-up •  General issues •  Deep focus •  Wrap-up

26

Page 27: How to Conduct Usability Studies: A Librarian Primer

Nondirected Interview

Prepare nondirected questions •  Focused on the person answering it •  Focused on experience, not extrapolation •  Concentrate on immediate experience •  Avoid judgmental language •  Focus questions on a single topic •  Keep questions open-ended •  Avoid binary questions

27

Page 28: How to Conduct Usability Studies: A Librarian Primer

Nondirected Interview

Examples “Is this a useful feature?” “Is this feature valuable to the work you do right now?” “Do you think this feature would be better if it was also available on smart phones?”

28

Page 29: How to Conduct Usability Studies: A Librarian Primer

Nondirected Interview

Running the interview •  De"ne (technical) terms •  Don’t force opinions •  Restate answers •  Never say the participant is wrong •  Work with a partner

29

Page 30: How to Conduct Usability Studies: A Librarian Primer

Analyzing Results

•  Collecting observations –  Notes from moderator and observer –  Quantitative information: task successfulness,

navigational path, number of steps, errors, time to complete, subjective ratings (System Usability Scale)

–  Qualitative feedback from participants

•  Extracting trends –  Repeated concerns –  Multiple issues with possible common origin –  Identify effects on user experience

30

Page 31: How to Conduct Usability Studies: A Librarian Primer

Analyzing Results

•  A list of usability issues –  Quantitative and qualitative data as evidence –  Possible causes –  Effects on user experience

•  Assign importance/severity •  Suggest solutions

31

Page 32: How to Conduct Usability Studies: A Librarian Primer

Communicating Findings

Usability test report •  Project overview •  Participants •  Testing environment •  Tasks •  Procedure •  Key "ndings

–  Issues –  Recommendations

32

Page 33: How to Conduct Usability Studies: A Librarian Primer

Communicating Findings

Presentation •  Prepare the audience •  Use supporting evidence (e.g., highlight videos) •  Emphasize user perspectives •  Use numbers carefully (because of small sample) •  Leave enough time for questions

33

Page 34: How to Conduct Usability Studies: A Librarian Primer

Communicating Findings

Common problems: •  “This is not statistically signi"cant.” •  “This user is our typical user.” “This user does not

have relevant experience.” •  “User X did Y; therefore, everyone must do Y.” •  “More than half of users did not like this design;

therefore, we should change it.”

34

Page 35: How to Conduct Usability Studies: A Librarian Primer

What’s Essential?

•  Develop a study plan •  Find participants •  Conduct the sessions •  Debrief with observers and decide together •  Present results

35

Page 36: How to Conduct Usability Studies: A Librarian Primer

Discussion

Think of an existing design that you want to evaluate for effectiveness •  What are my objectives? •  What types of resources do I have? •  Who am I presenting this to?

36

Page 37: How to Conduct Usability Studies: A Librarian Primer

Other Types of Usability Studies

•  Remote usability testing http://www.userzoom.com/

•  Eye tracking

37

http://www.normalmodes.com/blog/2009/09/28/eye-tracking-heatmap-gallery-a-preview-discussion-of-ui-considerations/

Page 38: How to Conduct Usability Studies: A Librarian Primer

Where to Find Out More

•  6 Must-See Usability Testing Videos http://blog.usabilla.com/7-must-see-usability-testing-videos/

•  Moderating Usability Tests Joe Dumas & Beth Loring (2008)

•  A Field Guide to Usability Testing Smashing eBook https://shop.smashingmagazine.com/guide-to-usability-testing-1.html

•  Handbook of Usability Testing Rubin & Chisnell (2008)

•  Don’t Make me Think by Steve Krug Chapter: “Usability Testing on 10 cents a Day”’

38

Page 39: How to Conduct Usability Studies: A Librarian Primer

Learning Objectives

39

•  Participants will be able to describe the usability concept and it's applications to libraries.

•  Participants will be aware of the basic work!ows and techniques needed when doing usability tests.

•  Participants will be aware of resources for usability studies and where to "nd out more.

Page 40: How to Conduct Usability Studies: A Librarian Primer

Agenda

40

•  What is a usability study? •  Usability study process •  Conducting the study •  Analyzing Results •  Communicating •  Where to "nd out more

Page 41: How to Conduct Usability Studies: A Librarian Primer

Works Cited

•  6 Must-See Usability Testing Videos - The Usabilla Blog. (n.d.). Retrieved from http://blog.usabilla.com/7-must-see-usability-testing-videos/

•  Affairs, A. S. for P. (2013). Usability Testing. Retrieved from http://www.usability.gov/how-to-and-tools/methods/usability-testing.html

•  Clear, Consistent, Capable, NICE» Blog Archive » Eye Tracking Heatmap Gallery: A Preview & Discussion of UI Considerations. (n.d.). Retrieved from http://www.normalmodes.com/blog/2009/09/28/eye-tracking-heatmap-gallery-a-preview-discussion-of-ui-considerations/

•  Dumas. Joe, & Loring, B. (2008). Moderating Usability Tests. •  DIY Mobile Usability Testing - SXSW Interactive 2012. (n.d.). Retrieved from

http://www.slideshare.net/beleniq/diy-mobile-usability-testing-sxsw-interactive-2012 •  Morae usability testing software from TechSmith. (n.d.). Retrieved from

http://www.techsmith.com/morae.html •  Krug, S. (2005). Don’t Make Me Think. •  Rubin, J., & Chisnell, D. (2008). Handbook of Usability Testing. •  UserZoom | zooming in on the user experience. (n.d.). Retrieved from http://www.userzoom.com/ •  Silverback — guerrilla usability testing. (n.d.). Retrieved from http://silverbackapp.com/

41

Page 42: How to Conduct Usability Studies: A Librarian Primer

Final Notes

Evaluation of this webinar Questions? Tao Zhang [email protected] Ilana Barnes [email protected]

42