best practices from 10 years of remote research
TRANSCRIPT
Best Practices From 10 Years of
Remote UX Research
thx
UX Research vs.
Remote UX Research vs.
User Research
Watching people use interfaces in order to
design better interfaces
60 YEARS AGO
LET’S RECORD HUMANS USING INTERFACES
30 YEARS AGO
Vs.
10 YEARS AGO
1% Humans observing humans
99% Data
• UX Research (Facebook, Google) • Experience Research (AirBnB) • Insights (Netflix) • Design Research (Uber) • User Research (So Many People)
INTERCHANGEABLE TERMS
Analytics Data Science
Quantitative Research
User Research UX Research Usability User Research 1:1 Observation Lean Research
TEXT
Geography of remote UX Research
Btw I’m Nate Bolt
PLEASE BUY THIS BOOK
ethn.io
How To
Plan, Recruit, and Conduct
Remote UX Research
1. Ask the right questions
AKA Conversations with Stakeholders
Figuring out the right question(s) is tricky
Quantitative research lets us test our assumptions, and qualitative research lets us find out what we don't know.
“
”
What do our customers/users/humans want?
“”
• Remoteresear.ch site • Mental Models by Indi Young • Just Enough Research by Erika Hall • Cognition in The Wild by Ed Hutchins • Complete Beginner's Guide to Design Research • Observing the User Experience: A Practitioner's
Guide to User Research
A Deeper Read
2. Try to Incorporate Data
•Surveys •Experiments (A/B, etc) •System data •Purchase Behavior
3. Identify Testing Goals
What do our customers/users/humans want?
“”
(and write them down in a G&O and then in a Moderator Script)
remoteresear.ch/category/samples
3. Recruit real humans
ethn.io
The Recruiting Spectrum
Grab Someone in Your Office
Live Customer Intercept
Recruiting Agency
4. Combine Tools & Methods
Observe Behavior
5. Collaborate don’t report
And how does this fit into the design process?
Understand
Prototype
Build
Understand• review existing research • focus groups • interviews • ethnography • surveys • card sorts • data analysis • diary studies
Prototype• participatory design • collaborative design • brainstorming techniques • heuristic evaluation • paper prototype testing • static mock testing
Build• usability testing • heuristic evaluation • diary study • A/B test • field testing
1. Identify Testing Goals 2. Incorporate Data 3. Recruit Real Humans 4. Combine tools & methods 5. Collaborate Don’t Report
How ToPlan, Recruit, and Conduct
Remote UX Research
Pitfalls
And Now Some Common
Decision Theory & Adaptive Systems Group
• Hypothetical Questions • A False Premise • Opinions are worthless
Some Common
Pitfalls
of Remote UX Research
• Behavior • Context • Behavior
Best Practices From 10 Years of
Remote UX Research
• Bandwidth (games) • Security (prototypes) • Translation (time) • Rapport (lacking)
Even More
Pitfalls
How & When to Run Rapid
Remote UX Research
46 Million AmericansThat’s 22 Million Households
15% of the country
Now what percentage of those 46 million were
employed within a year of receiving food assistance?
> 80%
So these are working people. People with
families. Teachers, etc.
How do they apply?
90 Minutes / 53 Days
up to
Alan WilliamsDesign Lead, SNAP
Caroline Hays Health Focus Area Intern
Jacob SolomanPolicy analyst and project manager
SNAP(Supplemental Nutrition Assistance Program)
Let’s Try the Interface for a
Application
Just Stop
30 phone interviews10 minutes each58 unique experiences in California
And 15 Follow-Up Phone Interviews
Quant & Qual
11 Minutes / 53 Days (?)
• 30 Remote Interviews • Real context • 15 Follow-Ups • Little to No Report • Input straight to designer & engineer
Best Practices From 10 Years of
Remote UX Research
• Easy, cheap, & fast phone interviews • Quick “Guerilla” Human Observation • No statistical significance • Behavior repeats over small samples • Combine with other inputs
Best Practices From 10 Years of
Remote UX Research
Top Tools for
Remote UX Research
“At least a little behavioral” - Marv
Surveys. Web Analytics.
Five Categories
•Mobile (spans other categories) •Moderated (screen-sharing + audio) •Self-moderated (usertesting.com) •Automated (loop11, usabilla) •Fancy Analytics on Steroids (crazyegg)
remoteresear.ch/tools
Conceptual
For A Long Time
Self-Moderated
Moderated
Web Analytics
Automated Static
Automated Live
Tasks On Concepts
For A Long Time
People Talk To A Machine
Talk To People
Fancy-Ass Analytics
Task Bar On Images
Task Bar On A Live Site
I.A. & Navigation
For Ad Agencies
Ammunition
Insight
Just Better Analytics
Still More Ammunition
More Ammunition
Conceptual
For A Long Time
Self-Moderated
Moderated
Web Analytics
Automated Static
Automated Live
Deep Behavioral Ux Insight
Opinions
Conceptual
For A Long Time
Self-Moderated
Moderated
Web Analytics
Automated Static
Automated Live
Ad Agencies Love It
Conceptual
For A Long Time
Self-Moderated
Moderated
Web Analytics
Automated Static
Automated Live
USABILLA
Clicks & Questions Heatmaps You do your own recruiting $49 - $199
ethn.io
If NASA can do THIS from 153 million miles away, we can meet our users where they are, even if we’re a couple thousand miles away.
Thanks for attending
Go run some awesome remote UX research
fin.
• Remoteresear.ch site • Mental Models by Indi Young • Just Enough Research by Erika Hall • Cognition in The Wild by Ed Hutchins • Complete Beginner's Guide to Design Research • Observing the User Experience: A Practitioner's
Guide to User Research
A Deeper Read
