best practices from 10 years of remote research

Best Practices From 10 Years of

Remote UX Research

thx

UX Research vs.

Remote UX Research vs.

User Research

Watching people use interfaces in order to

design better interfaces

60 YEARS AGO

LET’S RECORD HUMANS USING INTERFACES

30 YEARS AGO

Vs.

10 YEARS AGO

1% Humans observing humans

99% Data

• UX Research (Facebook, Google) • Experience Research (AirBnB) • Insights (Netflix) • Design Research (Uber) • User Research (So Many People)

INTERCHANGEABLE TERMS

Analytics Data Science

Quantitative Research

User Research UX Research Usability User Research 1:1 Observation Lean Research

TEXT

Geography of remote UX Research

Btw I’m Nate Bolt

PLEASE BUY THIS BOOK

ethn.io

How To

Plan, Recruit, and Conduct

Remote UX Research

1. Ask the right questions

AKA Conversations with Stakeholders

Figuring out the right question(s) is tricky

Quantitative research lets us test our assumptions, and qualitative research lets us find out what we don't know.

“

”

What do our customers/users/humans want?

“”

• Remoteresear.ch site • Mental Models by Indi Young • Just Enough Research by Erika Hall • Cognition in The Wild by Ed Hutchins • Complete Beginner's Guide to Design Research • Observing the User Experience: A Practitioner's

Guide to User Research

A Deeper Read

2. Try to Incorporate Data

•Surveys •Experiments (A/B, etc) •System data •Purchase Behavior

3. Identify Testing Goals

What do our customers/users/humans want?

“”

(and write them down in a G&O and then in a Moderator Script)

remoteresear.ch/category/samples

3. Recruit real humans

ethn.io

The Recruiting Spectrum

Grab Someone in Your Office

Live Customer Intercept

Recruiting Agency

4. Combine Tools & Methods

Observe Behavior

5. Collaborate don’t report

And how does this fit into the design process?

Understand

Prototype

Build

Understand• review existing research • focus groups • interviews • ethnography • surveys • card sorts • data analysis • diary studies

Prototype• participatory design • collaborative design • brainstorming techniques • heuristic evaluation • paper prototype testing • static mock testing

Build• usability testing • heuristic evaluation • diary study • A/B test • field testing

1. Identify Testing Goals 2. Incorporate Data 3. Recruit Real Humans 4. Combine tools & methods 5. Collaborate Don’t Report

How ToPlan, Recruit, and Conduct

Remote UX Research

Pitfalls

And Now Some Common

Decision Theory & Adaptive Systems Group

• Hypothetical Questions • A False Premise • Opinions are worthless

Some Common

Pitfalls

of Remote UX Research

• Behavior • Context • Behavior

Best Practices From 10 Years of

Remote UX Research

• Bandwidth (games) • Security (prototypes) • Translation (time) • Rapport (lacking)

Even More

Pitfalls

How & When to Run Rapid

Remote UX Research

46 Million AmericansThat’s 22 Million Households

15% of the country

Now what percentage of those 46 million were

employed within a year of receiving food assistance?

> 80%

So these are working people. People with

families. Teachers, etc.

How do they apply?

90 Minutes / 53 Days

up to

Alan WilliamsDesign Lead, SNAP

Caroline Hays Health Focus Area Intern

Jacob SolomanPolicy analyst and project manager

SNAP(Supplemental Nutrition Assistance Program)

Let’s Try the Interface for a

Application

Just Stop

30 phone interviews10 minutes each58 unique experiences in California

And 15 Follow-Up Phone Interviews

Quant & Qual

11 Minutes / 53 Days (?)

• 30 Remote Interviews • Real context • 15 Follow-Ups • Little to No Report • Input straight to designer & engineer

Best Practices From 10 Years of

Remote UX Research

• Easy, cheap, & fast phone interviews • Quick “Guerilla” Human Observation • No statistical significance • Behavior repeats over small samples • Combine with other inputs

Best Practices From 10 Years of

Remote UX Research

Top Tools for

Remote UX Research

“At least a little behavioral” - Marv

Surveys. Web Analytics.

Five Categories

•Mobile (spans other categories) •Moderated (screen-sharing + audio) •Self-moderated (usertesting.com) •Automated (loop11, usabilla) •Fancy Analytics on Steroids (crazyegg)

remoteresear.ch/tools

Conceptual

For A Long Time

Self-Moderated

Moderated

Web Analytics

Automated Static

Automated Live

Tasks On Concepts

For A Long Time

People Talk To A Machine

Talk To People

Fancy-Ass Analytics

Task Bar On Images

Task Bar On A Live Site

I.A. & Navigation

For Ad Agencies

Ammunition

Insight

Just Better Analytics

Still More Ammunition

More Ammunition

Conceptual

For A Long Time

Self-Moderated

Moderated

Web Analytics

Automated Static

Automated Live

Deep Behavioral Ux Insight

Opinions

Conceptual

For A Long Time

Self-Moderated

Moderated

Web Analytics

Automated Static

Automated Live

Ad Agencies Love It

Conceptual

For A Long Time

Self-Moderated

Moderated

Web Analytics

Automated Static

Automated Live

USABILLA

Clicks & Questions Heatmaps You do your own recruiting $49 - $199

ethn.io

If NASA can do THIS from 153 million miles away, we can meet our users where they are, even if we’re a couple thousand miles away.

Thanks for attending

Go run some awesome remote UX research

fin.

• Remoteresear.ch site • Mental Models by Indi Young • Just Enough Research by Erika Hall • Cognition in The Wild by Ed Hutchins • Complete Beginner's Guide to Design Research • Observing the User Experience: A Practitioner's

Guide to User Research

A Deeper Read

nate@ethn.io@ethnio

Nate Bolt