jonathan rubin - the benefits of do it yourself usability testing

The benefits of “Do It

Yourself” Usability Testing

By Jonathan Rubin

First Fridays Program Manager

General Services Administration

2

Let me tell you a story

3

Hanging chads

4

inspections

5

butteflu

They didn’t listen to customer’s

wants, needs and expectations.

TAKEWAY MESSAGE: Testing

with customers gets you

valuable feedback (duh)!

Why did problems occur?

7

In this hour…

1. What is usability

2. Why you need usability

3. Idea: Do your own testing!

1. What is usability?

Usability =

How people interact with a

product, system or service,

and how they expect it to

work.

Usability ≠ Quality Control

10

Good products are usable

11

12

What is it? How do you use it?

13

What is it? How do you use it?

14

Don’t feel stupid!

1. Observe users (customers) using your

product

2. (Re)design with users in mind

3. Get a usable product

4. Repeat

4 Steps to Working with Users

• Education • Demonstration Tests • Awareness

We’ve helped these folks

• DOT

• Interior

• IRS

• U.S. Army

• NASA

• Labor

• OMB

• GSA

17

• State

• FCC

• USAJobs

• Census

• NSF

• NIH

• And 35

others

Free @ Howto.gov/firstfridays

• Usability Testing Scripts • Email Templates • Best Practices • Final Report Examples • Project Management • Before and After Screenshots • Videos • Etc.

2. Why you need usability

Top 5 Government Usability

Problems

• Too many words

Top 5 Government Usability

Problems

• Too many words

• Gov speak / jargon

Top 5 Government Usability

Problems

• Too many words

• Gov speak / jargon

• Top tasks hard to find

Top 5 Government Usability

Problems

• Too many words

• Gov speak / jargon

• Top tasks hard to find

• Ineffective navigation

Top 5 Government Usability

Problems

• Too many words

• Gov speak / jargon

• Top tasks hard to find

• Ineffective navigation

• Ineffective search results

Example 2: GSA Intranet Example 2: GSA Intranet

?

9/11/2012 27

Before After

• Reduced developer time

Usability’s Return on Investment (ROI)

• Reduced developer time

• Reduced training time

Usability’s Return on Investment (ROI)

• Reduced developer time

• Reduced training time

• Reduced help desk calls / emails

Usability’s Return on Investment (ROI)

• Reduced developer time

• Reduced training time

• Reduced help desk calls / emails

• Higher task completion rate

Usability’s Return on Investment (ROI)

• Reduced developer time

• Reduced training time

• Reduced help desk calls / emails

• Higher task completion rate

• PR boost

Usability’s Return on Investment (ROI)

Know your users, for you are not them

- Usability Yoda

3. Idea: Do your own testing!

38

Must read: ‘Rocket Surgery Made Easy’

• Commit to monthly

product testing

• Same time each

month

• More observers +

more testing = more

buy-in

Needed for “discount” usability testing

• Executive champion!

Needed for UT

• Executive champion!

• One morning a month

Needed for UT

Needed for “discount” usability testing

• Executive champion!

• One morning a month

• 1-2 rooms

Needed for “discount” usability testing

• Executive champion!

• One morning a month

• 1-2 rooms

• Laptops

Needed for “discount” usability testing

• Executive champion!

• One morning a month

• 1-2 rooms

• Laptops

• Screen sharing

software ($1000/yr)

Needed for “discount” usability testing

• Executive champion!

• One morning a month

• 1-2 rooms

• Laptops

• Screen sharing

software ($1000/yr)

• Volunteers

Needed for “discount” usability testing

• Executive champion!

• One morning a month

• 1-2 rooms

• Laptops

• Screen sharing

software ($1000/yr)

• Volunteers

• Food ($10/person)

Needed for “discount” usability testing

• Executive champion!

• One morning a month

• 1-2 rooms

• Laptops

• Screen sharing

software ($1000/yr)

• Volunteers

• Food ($10/person)

• Incentives ($0- $80)

Needed for “discount” usability testing

• Executive champion!

• One morning a month

• 1-2 rooms

• Laptops

• Screen sharing

software ($1000/yr)

• Volunteers

• Food ($10/person)

• Incentives ($0- $80)

• Training (FF)

Needed for “discount” usability testing

48

How many people do I need?

Spending on usability

49

When usability is present, 10% of

development budget spent on

usability

- Usability Return on Investment (2008, Nielsen

Norman Group Report)

Staffing options (all are good)

1. Usability Contractor ($17,000 per test)

2. Dedicated Usability Staff ($60,000+)

3. Get training and do it yourself (Free)

50

Matrixed Approach to Staffing

51

• Part-time volunteers! • Borrow from other departments • You are “providing skill building opportunities” • Flexible needs: 1 -20 hours a month

Types of First Fridays Tests

• 1. Formal, talk-aloud tests

Stakeholder

Meeting

Develop

Scenarios

Recruit

Participants

+ Observers

Conduct

Test

Write

Report

Follow-up

Meeting

Formal Testing Process

Testing Room Observation Room

It takes two (rooms) to test

55

Formal Testing Schedule

• 8:20 – 8:30 Welcome

• 8:30 - 9:30 Tester #1

• 9:30 – 10:30 Tester #2

• 10:30 – 11:30 Tester #3

• 11:30 – Noon Lunch

• Noon – 1:30 Debrief / Critique

Absolute Minimum Usability Staffing

Testing Room

T

F

T = Tester,

F = Facilitator

Minimum First Fridays Staffing

Observation Room Testing Room

T

O

F M

O O

O O

O = Observer

F = Facilitator

M = Room Manager

Better Staffing

Observation Room Testing Room

T

F M

O

O

G = Greeter

DG = Data Gatherer

BU = Back up

G

F O O O O O

DG

BU

BU

BU

O = Observer

F = Facilitator

M = Room Manager

Types of First Fridays Tests

1. Formal, talk-aloud tests

2. Hallway Tests

2. Hallway Test

Types of First Fridays Tests

1. Formal, talk-aloud tests

2. Hallway Tests

3. Navigation Improvement / Card sort

3. Navigation Improvement

Types of First Fridays Tests

1. Formal, talk-aloud tests

2. Hallway Tests

3. Navigation Improvement / Card sort

4. Quick Win Evaluation (Heuristics Analysis)

64

4. Quick Win Evaluation (Heuristics)

• Not for newbies

• Group or single person feedback

– Checklist

– Task run-through

• PRO: Fast! <1 week

• CON: No users

Teach em to fish Teach ‘em to fish

• Executive champion!

• One morning a month

• 1-2 rooms

• Laptops

• Screen sharing

software

• Volunteers

• Food

• Incentives for testers

• Training (First Fridays!)

Review: What you need to fish

Howto.gov/firstfridays

Free usability resources!

• Observe a test

More info on First Fridays

• Observe a test

• Recommend a website

More info on First Fridays

• Observe a test

• Recommend a website

• Volunteer as a tester

(remote or in person)

More info on First Fridays

• Observe a test

• Recommend a website

• Volunteer as a tester

(remote or in person)

• Learn to fish (Facilitator

in Training program)

More info on First Fridays

Once a month

Thanks!

Jonathan.rubin@gsa.gov

firstfridays@gsa.gov

Howto.gov/firstfridays