Download - Rashmi Xerox Parc
1
MindCanvas: Reimagining online research
Rashmi Sinha
Uzantowww.uzanto.com
www.themindcanvas.com
2
Duct-tape research!
(in the trenches - Information Architecture Redesign of eBay.com)
3
Background to IA redesign
• eBay.com has grown in organic fashion – inconsistent, fractured user experience– Separate design / usability efforts for parts of eBay
• first effort to focus on user experience for entire site
• Basic eBay facts– World’s largest online marketplace (79 million in US)– Types of users
• Buyers & sellers• New & experienced• Individuals & corporations• Different countries & cultures
4
Challenges
• New IA would need to– Serve diverse user group
• varying goals / mental models
– Be future-oriented, allow for change & growth– Changes needed to be evolutionary not revolutionary
• accommodate users familiar with current site
• Senior management at eBay needed proof– Is there need for such drastic restructuring?
• Research to be completed in 6 weeks!
5
Goals of research
• Research has multiple goals– Need successful design, not just good design
• Serves user• Satisfies business goals • Is accepted by management
• 3-stage research method• Stages: Exploration Understanding Verification• Stages complement & build upon each other
• Mix of qualitative & quantitative methods • Qualitative: rich, open-ended understanding of users / domain• Quantitative: recommendations grounded in data (help make case
to management)
6
3-stage user research
Stage 1: ExploreInterviews, free-listing
Stage 3: Verify / Refine
Large-sample closed card-sorting
3 stages
Stakeholders
User thinking / categorization
Preliminary IA
Stage 2: UnderstandOpen card-sorting
ebay.com
Community
Buy SellSub cat1
SubCat2
SubCat3
xx
SubCat1
Subcat4
Subcat3
Subcat2
xx
xx
xxx
SubCat3
SubCat5
SubCat4
SubCat6
SubCat4
xx
xxx
xx
xxx
xxx
xxx
SubCat4
SubCat5
SubCat1
SubCat2
xxx
xx
xx
Verify categorization
Top-level
cate
gories
Scope of content & functionality
7
Stage 1: Explore domain with Freelisting
• Goals– Explore domain in open-ended fashion, map scope.– Generate list of current & “Horizon Tasks” (planned for
future)
• Method– Interviews & Freelisting– With eBay users, stakeholders & designers
• Freelisting tasks (eBay is about tasks!)– What are all the tasks users can do on eBay.com
• List an item, Pay for it– What else will users be able to do on eBay.com 2-5
years that they can’t do today?
• Results: List of 100 representative tasks (25 Horizon)
8
Stage 2: Explore user categorizations
• Goals: – Gain insight into user thinking about eBay– Identify top-level categories for ALL site
content and functionality
• Method: Interviews & Open card sorting – List of 100 tasks (including 20 Horizon
Tasks).– 35 participants (including sellers & buyers;
new and advanced users).
9
Identify top levels of hierarchy
• Hierarchical Cluster analysis to generate aggregate user categorizations.
• Separate analysis for 4 user groups – Identify inconsistencies in
categorizations– Reconcile to create one scheme.
• Result: Preliminary IA– 5 top-level categories accounting for all
site content and functionality.
0.000
0.200
0.400
0.600
0.800
1.000
See eBay time
Fn eBay help
Fn cusTrk SupportReq. tech. support
Read eBay guidance
See eBay overview
Get eBay appsRefer friend
Ask experts
Ask other members
Fn user groupsRead eBay msgs
View eBay news
Read member msgs
See aucTrk graphs
List item
Cancel auction
Add image
Add res./BIN priceRevise listings
Create listing
Answer question
Send invoiceManage inventory
List regionally
List internationally
List item to donateCross-sell items
Manage eBay store
Trk shipments
IntLrn customsEsTrk shipping
Reg. as buyerReg. as seller
Report bad itemsReport accTrk theft
Report NPB/fraudRsrch. promotions
Rsrch. items sold
Rsrch. for listing
Rsrch. categoriesLrn manage bus.
Lrn seller benefits
Get selling tools
Be Power SellerFn sales by member
Fn items by bidderFn info. on eBay-OK code
Rsrch. partneringRsrch. eBay services
Be Trad. AssTrk / consultSee aucTrk cancels.
See bid retractions
View id history
Fn eBay memberMember contact info.
Lrn manage rep.Read feedback
Post feedbackBIN
Bid
Buy digital
Bid promo. option
Reverse auction
Negotiate price
Offer for unsold item
EsTrk car payment
Get buying tools
Contact seller
Contact buyer
Fn accessoriesFn items by seller
Search & browse
Fn BIN items
Fn items nearby
Fn specific item
Fn types of items
Browse items
Fn proFn servicesView item desc.
Translate listing
Fn eBay internatLrn
Save itemsTask Y
Task U
Task OTask F
Task R
Task S
Task D
Task L
Task M
Task A
Task K
Task GeTask P
Task X
Task B
Task ZTask N
Dendrogram- User group !
0.000
0.200
0.400
0.600
0.800
1.000
See eBay time
Fn eBay help
Fn cusTrk SupportReq. tech. support
Read eBay guidance
See eBay overview
Get eBay appsRefer friend
Ask experts
Ask other members
Fn user groupsRead eBay msgs
View eBay news
Read member msgs
See aucTrk graphs
List item
Cancel auction
Add image
Add res./BIN priceRevise listings
Create listing
Answer question
Send invoiceManage inventory
List regionally
List internationally
List item to donateCross-sell items
Manage eBay store
Trk shipments
IntLrn customsEsTrk shipping
Reg. as buyerReg. as seller
Report bad itemsReport accTrk theft
Report NPB/fraudRsrch. promotions
Rsrch. items sold
Rsrch. for listing
Rsrch. categoriesLrn manage bus.
Lrn seller benefits
Get selling tools
Be Power SellerFn sales by member
Fn items by bidderFn info. on eBay-OK code
Rsrch. partneringRsrch. eBay services
Be Trad. AssTrk / consultSee aucTrk cancels.
See bid retractions
View id history
Fn eBay memberMember contact info.
Lrn manage rep.Read feedback
Post feedbackBIN
Bid
Buy digital
Bid promo. option
Reverse auction
Negotiate price
Offer for unsold item
EsTrk car payment
Get buying tools
Contact seller
Contact buyer
Fn accessoriesFn items by seller
Search & browse
Fn BIN items
Fn items nearby
Fn specific item
Fn types of items
Browse items
Fn proFn servicesView item desc.
Translate listing
Fn eBay internatLrn
Save itemsTask Y
Task U
Task OTask F
Task R
Task S
Task D
Task L
Task M
Task A
Task K
Task GeTask P
Task X
Task B
Task ZTask N
0.000
0.200
0.400
0.600
0.800
1.000
0.000
0.200
0.400
0.600
0.800
1.000
0.000
0.200
0.400
0.600
0.800
1.000
See eBay time
Fn eBay help
Fn cusTrk SupportReq. tech. support
Read eBay guidance
See eBay overview
Get eBay appsRefer friend
Ask experts
Ask other members
Fn user groupsRead eBay msgs
View eBay news
Read member msgs
See aucTrk graphs
List item
Cancel auction
Add image
Add res./BIN priceRevise listings
Create listing
Answer question
Send invoiceManage inventory
List regionally
List internationally
List item to donateCross-sell items
Manage eBay store
Trk shipments
IntLrn customsEsTrk shipping
Reg. as buyerReg. as seller
Report bad itemsReport accTrk theft
Report NPB/fraudRsrch. promotions
Rsrch. items sold
Rsrch. for listing
Rsrch. categoriesLrn manage bus.
Lrn seller benefits
Get selling tools
Be Power SellerFn sales by member
Fn items by bidderFn info. on eBay-OK code
Rsrch. partneringRsrch. eBay services
Be Trad. AssTrk / consultSee aucTrk cancels.
See bid retractions
View id history
Fn eBay memberMember contact info.
Lrn manage rep.Read feedback
Post feedbackBIN
Bid
Buy digital
Bid promo. option
Reverse auction
Negotiate price
Offer for unsold item
EsTrk car payment
Get buying tools
Contact seller
Contact buyer
Fn accessoriesFn items by seller
Search & browse
Fn BIN items
Fn items nearby
Fn specific item
Fn types of items
Browse items
Fn proFn servicesView item desc.
Translate listing
Fn eBay internatLrn
Save itemsTask Y
Task U
Task OTask F
Task R
Task S
Task D
Task L
Task M
Task A
Task K
Task GeTask P
Task X
Task B
Task ZTask N
See eBay time
Fn eBay help
Fn cusTrk SupportReq. tech. support
Read eBay guidance
See eBay overview
Get eBay appsRefer friend
Ask experts
Ask other members
Fn user groupsRead eBay msgs
View eBay news
Read member msgs
See aucTrk graphs
List item
Cancel auction
Add image
Add res./BIN priceRevise listings
Create listing
Answer question
Send invoiceManage inventory
List regionally
List internationally
List item to donateCross-sell items
Manage eBay store
Trk shipments
IntLrn customsEsTrk shipping
Reg. as buyerReg. as seller
Report bad itemsReport accTrk theft
Report NPB/fraudRsrch. promotions
Rsrch. items sold
Rsrch. for listing
Rsrch. categoriesLrn manage bus.
Lrn seller benefits
Get selling tools
Be Power SellerFn sales by member
Fn items by bidderFn info. on eBay-OK code
Rsrch. partneringRsrch. eBay services
Be Trad. AssTrk / consultSee aucTrk cancels.
See bid retractions
View id history
Fn eBay memberMember contact info.
Lrn manage rep.Read feedback
Post feedbackBIN
Bid
Buy digital
Bid promo. option
Reverse auction
Negotiate price
Offer for unsold item
EsTrk car payment
Get buying tools
Contact seller
Contact buyer
Fn accessoriesFn items by seller
Search & browse
Fn BIN items
Fn items nearby
Fn specific item
Fn types of items
Browse items
Fn proFn servicesView item desc.
Translate listing
Fn eBay internatLrn
Save itemsTask Y
Task U
Task OTask F
Task R
Task S
Task D
Task L
Task M
Task A
Task K
Task GeTask P
Task X
Task B
Task ZTask N
Dendrogram- User group !
Dendogram
10
Stage 3: Verify & refine scheme
• Goals:– Verify 5-category scheme covers all of eBay.
• What tasks/concepts don’t fit? Where do users expect this info?– Input from larger, more diverse set of users.– Establish method for future verification of IA.
• and to generalize scheme to international eBays.
• Method: closed card sorting – Large sample (~ 1,000) representing all eBay user types.– Study conducted using online survey.
• Participants categorized tasks into one of categories (or selected Other if it didn’t belong)
• Result– Site-map showing the structure for all main parts of the
eBay.com
11
Sell - Before
12
Sell Hub - After
13
eBay redesigned successfully!
• Three-stage methodology– Understand business context – Delineate user categorizations– Create IA blueprint
• Deliverable: site-map with structure for main parts of eBay.com. – map used as blueprint of IA redesign
(took 2 years).– Generalized to international eBays.– Positive business implications (ROI)!– Users did not protest!
site.com
TopCategory 3
TopCategory1
TopCategory 2
TopCategory 4
SubCat1Sub cat1
SubCat2
xx
SubCat3
xx
xx
SubCat2
SubCat3
SubCat5
xx
xx
xx
xx
SubCat1
Subcat4
Subcat3
Subcat2
xx
xx
xxx
SubCat3
SubCat5
SubCat4
SubCat6
SubCat4
xxx
xx
xx
xxx
xx
xxx
xxx
xxx
SubCat7
SubCat4
SubCat5
SubCat6
SubCat1
SubCat2
xx
xx
xxx
xx
xxx
xx
xx
SubCat4
xx
Preliminary IA
14
Duct-tape research!
There has to be a better way - fast, cheap (relatively) & online…
15
Why we built MindCanvas
• Qualitative research is great, takes a lot of time
• Online tools suck, get used anyway (users not engaged)
• Business stakeholders respond to quantitative analysis & large sample sizes
• Deliverables do not cater to designers
• Research findings remain locked up with analyst
16
Inspiration
• Luis Von Ahn’s ESP Game– Rely on games to label the web
• PopCap Games– Quick, short, engaging games
• Visualization research– bringing statistics to designers
17
18
back to the future!
MindCanvas interpretation
19
What is MindCanvas?
Remote research methods
Statistical Analysis & Data Mining
Interactive Visualizations
loosely coupled, use as qualitative or quantitative
20
Game-like elicitation methods
• Surveys do not engage people. Games are fun!– People engage in complex tasks willingly
• Do game-like methods work better?
– MindCanvas has game-like elements (its not a game)• Interactive, visual, screen build up, fun
• For both qualitative & quantitative– Remote moderated for qualitative– Remote unmoderated for quantitative
21
Rich interactive visualizations
• Large datasets need statistical analysis & data mining
• Visualizations-to-go• Self-contained files - embedded into PowerPoint, emailed, shared
• Interactive visualizations with slides & knobs!
• Easy to share with team
• Visualization posters– Designers want printouts to hang on walls, and scribble on
22
MindCanvas research focuses on the mind
• Inspiration from cognitive anthropology, psychology, market research
• Semi-structured methods
• Quantitative aggregation & visualization
Categorizations
PreferencesLanguage
23
Who we built for: designer persona
Needs convenient
methods to do research
Needs convenient
methods to do research
Does not want to spend time
learning statistics
Does not want to spend time
learning statistics
The busy designer / researcher who wants to look at patterns visually
Want to directly interact with users, does not have time
Want to directly interact with users, does not have time
24
Who we built for: participant persona
“This was the most fun I
have had in a while!”
“This was the most fun I
have had in a while!”
“It is very easy to complete the task. In fact, I had fun doing
this.”
“It is very easy to complete the task. In fact, I had fun doing
this.”
The not-tech savvy person who occasionally takes surveys
“I think there are too many cards. But it was kind of
fun.”
“I think there are too many cards. But it was kind of
fun.”
25
Where MindCanvas could help
• Product Development: Finding the right feature-set for the right users (or personas)
• Information Architecture: Understanding how people think for information design.
• Early visual design validation: Understanding what people think & feel about a design when its just an image.
26
MindCanvas in product development
• Initial understanding of domain– What’s important to people? What words do they
use?• Method: Freelisting• Deliverable: ListMap
• Prioritizing features / outcomes users care about– Method: Divide-the-dollar– Deliverable:
• WeightMap• ClusterBrowser
27
MindCanvas in Information Architecture
• How do people think about a domain?– Method: OpenSort– Deliverable
• Dendogram• SimilarityBrowser• VocabularyBrowser1, 2
• Is your information architecture effective? Is one scheme more effective than another?– Method: TreeSort– Deliverable:
• ClusterBrowser
28
29
Questions?
MindCanvas team is…Delhi
Kapil MohanKamalmeet SinghDebasish Biswas
Sunil GuptaAmit Ranjan
Mountain ViewJon Boutelle
Rashmi Sinha
30
Methods & Deliverables
• Current Methods– OpenSort: Dendogram, VocabBrowser, SimilarityBrowser
– TreeSort: ClusterBrowser, SimilarityBrowser
– Divide-the-Dollar: WeightMap, ClusterBrowser
– Free Listing: ListMap
– Utility Questions: MultipleChoice, EssayType
• Upcoming Methods: MenuTrace, MindMontage, Likert Sort
• Upcoming Designables: OnionMap
31
Collaborative Research Process
• High quality research needs expertise– research design + statistics
• Service should enable high quality research– Structured research process
• Templates for study design• Help with data collection
32
What is MindCanvas
• An innovative research platform for high quality research– Game-like elicitation methods– Interactive visualizations (designables-to-go)– Research expertise