Guerilla usability testing or

How I learned that perfectly imperfect tests are still worth it

Sara Snyder @sosarasays

Benefits of in-house, DIY testing (but you know this already!)• Uncover unexpected issues• Fix problems early, while it is still relatively easy• Test assumptions, early and often• Introduce objectivity into opinion wars• Continually cultivate empathy in the hearts of

stakeholders• Set baselines and measure improvement• Great ROI at such low cost, we’d be crazy not to!

“Listening to what people say is misleading: you have to watch what they actually do.”

-Jakob Nielsen, Usability 101, useit.com

How Usable is your Usability Testing?1. Get representative users.2. Ask users to perform

representative tasks with the design.

3. Observe where they succeed, and where they run into problems.

Easy, right? So how frequently are you testing?

The best usability test is the one you actually do. Repeatedly.The process & reporting MUST be fairly painless, or we’ll find some excuse to put it off:

“Too busy, can’t stop for testing right now.”“I’d like to get one more feature built before showing it to anybody.”“The testing software is too expensive.”(“I am scared or embarrassed to find out the truth: that my website isn’t usable!”)

Story 1: “It is a quick read, and there are lots of pictures.”

Story 1: Lessons Learned

• Shy? Get over it. Human-to-human interaction IS happening.

• Tailor your recruitment strategy for each audience you are seeking (the yell vs. whisper)

• Live-streaming tests can be technologically tricky, but really engages stakeholders.

• Pre-screen your candidates a bit before you sit them down at the computer.

• Make graduate students (like me) write the reports!

Story 2: “Pay no attention that row of women in glasses.”

Story 2: Lessons Learned

• The best way to convince management of the value of testing is by (duh) testing…even without software.

• Recorded tests are simpler to set up, especially if you don’t have much IT help.

• Be creative with non-monetary incentives. • Group screenings + coffee + donuts = maximum

stakeholder engagement!• Understand “Human Subjects Research” policies

Story 3: “You almost done with that sandwich?”

Story 3: Lessons Learned

• Still shy? Still too busy? Recruit a usability intern, treat them well, and keep them around as long a you can.

• When recording on the go, good sound is EVERYTHING (mic + facilitator earphone test).

• Test out tasks on staff who may not use website often. If one task seems too hard, IT IS! Drop it!

• Really nice people hang out in art museums, and boy, do they love tote bags.

Mistakes aren’t forever. When you’re testing often, it lowers the stakes.

It will always grow back.

D.I.Y. Usability “Lab”

•Laptop & USB microphone•Software options

•Morae ($1,500) •Camtasia ($300)•Silverback for Mac ($0-39)•Paper and pencil (free)

Analyzing and Reporting On Results•Holding screenings for staff helps develop their interest level & empathy

•A one-page list of improvements you can act on is better than a 10-page report that sits in a drawer!

Let the next round of imperfect-but-still-good-enough testing start!