Guerrilla Usability Testing

Usability"Good design is problem solving"

Jeffrey Veen(Analytics, Typekit, author...)

Book: Psychology of everyday things

by Donald Norman

What’s Usability

• Usable, "easy to use". It's a very intuitive and simple concept, yet many companies don't do it (officially)

• Like CT, but ABE one of the simplest Booking Engines.

• User Centered design. It has to be self-explained.

• Technology shouldn’t make us feel stupid, neither force us to learn complicated stuff. “Don’t Make me Think”, book by Steve Krug

• Jakob Nielsen : Focus on monetizing

Example 1 : Useit.com

Example 2 : Craiglists, another dull yet successful (usable) site

Example 3 : Amazon

Example 6 : Apple

and more; Google, Youtube...

Example 7 : ABE step 1. 3 simple steps (not just good for marketing)

ABE Step 2 grid layout. Not the most common layout, but it works.

ABE Step 3

Why things are unusable?

• Development focus on the machine

• It's hard to achieve

• Teams don't work integrated

• Design vs implementation

• Developer’s lack of perspective.

• like...

Unusable: Irishrail.ie : Complex, uses its own language...

Why things are usable?

• Early focus on users and task. Interactive Design

• Prototyping and wireframing

• Evaluation and measurement

• Research, Surveys, Observation

• like...

Usable : Dropbox

Analysis

Design

Implementation

Evaluation

Development process stages

Usability TestingUsability testing is a technique used to evaluate a product by testing it on users...

Wikipedia

Usability testing generally involves setting a series of tasks for people to complete and noting any problems they encounter

Andy Budd

Usability Testing Lab

Benefits on usability testing

• Help inform the design process

• Uncover unknown or unexpected issues

• Fix problems early (easier)

• Test assumptions

• Highlight unnecessary features

• Provide Objectivity solve opinion battles

• Set baselines and measure improvement

For designers For site owners

Andy Budd

• Improve customer satisfaction and retention

• Increase conversion rates

• Reduce maintenance, trainning and support costs

• Reduce project risk

• Highlight unnecessary features

• It’s so cheap you’d be stupid not to

Effective ways of testing

• Eye tracking. Heat maps

• A/B Testing

• Formal testing (made by a company, lab, focus groups...)

• Guerrilla style

Eye tracking: Heat map

A/B Testing : Used by Argus to implement Lightbox

A/B Testing : Used by Argus to implement Lightbox

“The results were encouraging: after 3 weeks of testing, the average landing page conversion rate had gone from 2.70% to 3.33% (an average rise of 20% in the conversion rate)

Following the positive impact of the test, the light box functionality was rolled out across all PPC landing pages, leading to marked increase in conversions (+19%)

This was then rolled out on to the rest of the site leading to an improvement of 5.99% in the conversion rate for all traffic mediums”

(Thanks Dave)

Guerrilla Usability TestingGuerrilla warfare is the unconventional warfare and combat in which a small group of combatants use mobile tactics in the form of ambushes and raids to combat a larger and less mobile formal army.

Guerrilla usability testing

• Get somebody from the street, and ask them to do simple task, observe and record (using Camtasia or Silverback)

• Affordable (small rewards), informal. Everybody can help or bring somebody

• Small group of people (5 get 85% issues, 12 get 99%)

• Moderator and “Think Aloud” protocol

• Precise task definition

• Using tools for metrics like analytics, surveys and feedbacks...

Pros and Cons of Guerrilla style

• Help inform the design process

• Uncover unknown or unexpected issues

• Fix problems early (easier)

• Test assumptions

• Highlight unnecessary features

• Provide Objectivity solve opinion battles

• Set baselines and measure improvement

PROS CONS

• Improve customer satisfaction and retention

• Increase conversion rates

• Reduce maintenance, trainning and support costs

• Reduce project risk

• Highlight unnecessary features

• It’s so cheap you’d be stupid not to

Silverback

How does it work

Gerrilla Testing

• We did it for the first time in September 2010 before push live the new Step 3

• We found just a few issues; actually step 1: Magic locations

• For a moment we though, Is this really working?

Results

Gerrilla Testing

• Then we did it again, in the process of developing the map, this time we did it earlier stage.

• Very different results, tough but positive feedback. They liked the idea.

• Things to work on and improve. But at least we knew what to focus on.

2nd Testing: The Map

• Pick up locations, most didn’t know they could put the address

• Zooms and controls

• Some didn’t see “I’m here”, confused the best price icon...

• Missing elements like scales

• Ignoring funcionalities, (Legend, Sort by, Refres)

• Didn’t fully understand which car was which flag in the map

Issues we found

Gerrilla Testing

Video

Thanks

questions?

Work in progress...