the benefits of "do it yourself" usability testing

The benefits of “Do It Yourself” Usability

Testing

By Jonathan RubinFirst Fridays Program ManagerGeneral Services Administration

2

Let me tell you a story

3

Hanging chads

4

inspections

5

butteflu

They didn’t listen to customer’s wants, needs and expectations.

TAKEWAY MESSAGE: Testing with customers gets you valuable feedback (duh)!

Why did problems occur?

7

In this hour…

1. What is usability2. Why you need usability3. Idea: Do your own testing!

1. What is usability?

Usability =

How people interact with a product, system or service, and how they expect it to work.

Usability ≠ Quality Control

10

Good products are usable

11

12

What is it? How do you use it?

13

What is it? How do you use it?

14

Don’t feel stupid!

1. Observe users (customers) using your product

2. (Re)design with users in mind3. Get a usable product4. Repeat

4 Steps to Working with Users

• Education• Demonstration Tests• Awareness

We’ve helped these folks

• DOT• Interior• IRS• U.S. Army• NASA• Labor• OMB• GSA

17

• State• FCC• USAJobs• Census• NSF• NIH• And 35

others

Free @ Howto.gov/firstfridays

• Usability Testing Scripts• Email Templates• Best Practices• Final Report Examples• Project Management• Before and After Screenshots• Videos• Etc.

2. Why you need usability

Top 5 Government Usability Problems

• Too many words


• Too many words• Gov speak / jargon


• Too many words• Gov speak / jargon• Top tasks hard to find


• Too many words• Gov speak / jargon• Top tasks hard to find• Ineffective navigation


• Too many words• Gov speak / jargon• Top tasks hard to find• Ineffective navigation• Ineffective search results

Example 2: GSA IntranetExample 2: GSA Intranet

04/10/2327

Before After

• Reduced developer time

Usability’s Return on Investment (ROI)

• Reduced developer time• Reduced training time


• Reduced developer time• Reduced training time• Reduced help desk calls / emails


• Reduced developer time• Reduced training time• Reduced help desk calls / emails• Higher task completion rate


• Reduced developer time• Reduced training time• Reduced help desk calls / emails• Higher task completion rate• PR boost


Know your users, for you are not them - Usability Yoda

3. Idea: Do your own testing!

38

Must read: ‘Rocket Surgery Made Easy’

• Commit to monthly product testing • Same time each month• More observers + more testing = more buy-in

Needed for “discount” usability testing

• Executive champion!

Needed for UT

• Executive champion!• One morning a month

Needed for UT

Needed for UT


• Executive champion!• One morning a month• 1-2 rooms


• Executive champion!• One morning a month• 1-2 rooms• Laptops


• Executive champion!• One morning a month• 1-2 rooms• Laptops• Screen sharing software ($1000/yr)


• Executive champion!• One morning a month • 1-2 rooms• Laptops• Screen sharing software ($1000/yr)• Volunteers


• Executive champion!• One morning a month • 1-2 rooms• Laptops• Screen sharing software ($1000/yr)• Volunteers• Food ($10/person)


• Executive champion! • One morning a month• 1-2 rooms• Laptops• Screen sharing software ($1000/yr)• Volunteers• Food ($10/person)• Incentives ($0- $80)


• Executive champion! • One morning a month• 1-2 rooms• Laptops• Screen sharing software ($1000/yr)• Volunteers• Food ($10/person)• Incentives ($0- $80)• Training (FF)


48

How many people do I need?

Spending on usability

49

When usability is present, 10% of development budget spent on usability

- Usability Return on Investment (2008, Nielsen Norman Group Report)

Staffing options (all are good)

1. Usability Contractor ($17,000 per test)

2. Dedicated Usability Staff ($60,000+)3. Get training and do it yourself (Free)

50

Matrixed Approach to Staffing

51

• Part-time volunteers!• Borrow from other departments• You are “providing skill building opportunities”• Flexible needs: 1 -20 hours a month

Types of First Fridays Tests

• 1. Formal, talk-aloud tests

Stakeholder Meeting

Develop Scenarios

Recruit Participants

+ Observers

Conduct Test

Write Report

Follow-up Meeting

Formal Testing Process

Testing RoomObservation Room

It takes two (rooms) to test

55

Formal Testing Schedule

• 8:20 – 8:30 Welcome• 8:30 - 9:30 Tester #1• 9:30 – 10:30 Tester #2• 10:30 – 11:30 Tester #3• 11:30 – Noon Lunch• Noon – 1:30 Debrief / Critique

Absolute MinimumUsability Staffing

Testing Room

T

F

T = Tester, F = Facilitator

Minimum First Fridays Staffing

Observation Room Testing Room

T

O

FM

OO

O O

O = Observer F = Facilitator M = Room Manager

Better Staffing

Observation Room Testing Room

T

FM

O

O

G = GreeterDG = Data Gatherer BU = Back up

G

FOOOOO

DG

BU

BU

BU

O = ObserverF = Facilitator M = Room Manager


1. Formal, talk-aloud tests2. Hallway Tests

2. Hallway Test

http://www.youtube.com/watch?v=pC0ODg2MpuE


1. Formal, talk-aloud tests2. Hallway Tests3. Navigation Improvement / Card sort

3. Navigation Improvement


1. Formal, talk-aloud tests2. Hallway Tests3. Navigation Improvement / Card sort4. Quick Win Evaluation (Heuristics

Analysis)

64

4. Quick Win Evaluation (Heuristics)

• Not for newbies• Group or single person feedback– Checklist– Task run-through

• PRO: Fast! <1 week• CON: No users

Teach em to fishTeach ‘em to fish

• Executive champion! • One morning a month• 1-2 rooms• Laptops• Screen sharing software• Volunteers• Food• Incentives for testers• Training (First Fridays!)

Review: What you need to fish

Howto.gov/firstfridays

Free usability resources!

• Observe a test

More info on First Fridays

• Observe a test• Recommend a website


• Observe a test• Recommend a website• Volunteer as a tester (remote or in person)


• Observe a test• Recommend a website• Volunteer as a tester (remote or in person)• Learn to fish (Facilitator in Training program)


Once a month

Thanks!

[email protected]

[email protected]

Howto.gov/firstfridays

the benefits of "do it yourself" usability testing

Documents