UI/UX Foundations - Research

Download UI/UX Foundations - Research

Post on 13-Apr-2017

4.652 views

Category:

Design

2 download

TRANSCRIPT

  • UI/UX Foundations: Research & Analysis

    Meg Kurdziolek and Karen Tang

  • Your Goals

    What would you like to learn today?

  • Our Goals

    We want you to

    talk confidently to UX researchers

    critically understand research presented to you

    conduct basic UX research on your own

    have a basis to continue learning about UX research

  • Activity (setup)Which of these problems do you feel strongly about?

    Pittsburgh public transportation

    Food delivery in Pittsburgh

    Finding family-friendly activities in Pittsburgh

    Pittsburgh public schools

  • Agenda09:00 - 09:20 Breakfast and Introductions

    09:20 - 09:30 User-Centered Design

    09:30 - 10:10 Surveys, Diary Studies, Interviews

    10:10 - 10:55 Usability Studies, Field Studies

    10:55 - 11:25 A/B Testing, Log Analysis

    11:25 - 12:00 Adapting Your Methods

    12:00 - 12:30 LUNCH

    12:30 - 01:00 Interpreting Your Data

    01:00 - 01:40 Special Topics: Dark UX Patterns

    01:40 - 01:50 Case Studies

    01:50 - 02:00 Group reflections & wrap-up Q&A

  • User-centered design (n.) - a framework of processes in which the needs, wants, and limitations of end users of a product are given extensive attention at each stage of the design process.

    https://en.wikipedia.org/wiki/User-centered_design

  • Design Process

    Refine

    BuildLearn

  • UX Research & Design

    Refine Build Learn

  • UX Research & Design

    Refine Build Learn

    Data Collection, Validation, Evaluation

  • UX Research & Design

    Refine Build Learn

    Data Collection, Validation, Evaluation

  • UX Research & Design

    Refine Build Learn

    Data Collection, Validation, Evaluation

    A/B Testing Contextual Inquiry Diary Studies Ethnography Field Studies

    Focus Groups Hallway Testing Heuristic Evaluation Interviews Lab Testing

    Log Analysis Remote Testing Think-Aloud Surveys User Observation

  • Need Finding

    Refine Build Learn

    Data Collection, Validation, Evaluation

    A/B Testing Contextual Inquiry Diary Studies Ethnography Field Studies

    Focus Groups Hallway Testing Heuristic Evaluation Interviews Lab Testing

    Log Analysis Remote Testing Think-Aloud Surveys User Observation

  • Need Finding

    Refine Build Learn

    A/B Testing Contextual Inquiry Diary Studies Ethnography Field Studies

    Focus Groups Hallway Testing Heuristic Evaluation Interviews Lab Testing

    Log Analysis Remote Testing Think-Aloud Surveys User Observation

    Data Collection, Validation, Evaluation

  • Surveys

    http://www.baerpm.com/blog/what-a-customer-survey-can-do-for-your-business/

  • are good for learning:

    overall impressions

    who your users are (demographics)

    outstanding opinions

    whos might participate in further research

    Surveys

  • Common Survey Example

    How likely is it that youd recommend [brand] to a friend?

    Not at all Likely Neutral

    Very Likely

    1 2 3 4 5 6 7 8 9 10

  • Net Promoter ScoreNPS: research has shown that your NPS acts as a leading indicator of

    growth. If your organizations NPS is higher than those of your competitors, you will likely outperform the market

    https://www.netpromoter.com/know/

  • Surveys: Pros and ConsBenefits

    Cheap (in $ and time)

    Easy to recruit participants

    May receive high response rate

    Easy to analyze

    Limitations

    Limited in type & scope of data

    Question interpretation issues

    Response bias

    Easy to misinterpret or *over*-interpret results

  • Silly Survey Example

    Do you like to eat lunch alone?

    Yes

    No

  • Diary Studies

    http://hciresearch4.hcii.cs.cmu.edu/M-HCI/2011/BOA-PlanningTools/

  • are good for.

    day-to-day habits & patterns

    when, how, and why they use your product

    reflections on real problems encountered and how they were solved

    Diary Studies

  • Diary Studies: Pros and ConsBenefits

    A longitudinal scope of data

    Get a look at the mundane, every-day interactions and behaviors

    Limitations

    Costly (in $ and time)

    Difficult to recruit participants (& high attrition)

    Relies on self-report

  • Example Diary Study

    Radar: Intellicast vs. Weather Underground

  • Hot Radar

  • Interviews

  • are good for.

    users background

    their use of technology

    their goals and motivations

    their pain points

    what problems need to be addressed or solve

    Interviews

  • Interviews: Pros and ConsBenefits

    Cheap (in $)

    Can target specific users or be opportunistic

    Can engage with users personally

    Can get the answer to lots of why questions

    Limitations

    Takes a moderately amount of time

    Results indicate what people *say* they do (rather than actual behavior)

  • Interview TipsStart broad, then narrow-in. example: Overall, how do you think Pittsburgh public transit compares to other cities?

    Ask clarifying questions, and use their words. example: You said the bus system is hard to predict, could you explain that to me?

    Its okay to play-dumb. (But be honest.) example: Ive never used public transit here. Can you tell me how you would find out the schedules and figure it out how to get downtown from here?

  • More Interview TipsAvoid Yes/No questions.

    Avoid asking about feelings. Ask about behaviors instead.

    Dont number your questions. Organize by topics you want to cover. Be prepared to skip around.

    Always be prepared to go off-script.

    Ask the question, then pause. Dont rush to fill silence.

  • Activity - Part 1Partner with someone who is interested in a different topic than you. You will interview them on their chosen topic.

    Its your job to explore what the needs are and uncover the main issues, feelings, thoughts, and pain-points.

    Round 1: (5 minutes) Develop your Script Goal: Individually, develop a rough script that you will use to interview your partner. Start broad to gather overall impressions, then narrow in on specific topic areas. Remember, you are trying to understand overall impression and the biggest pain-points.

    Round 2: (20 minutes) Interview (10 minutes each) Goal: Take turns interviewing each other. Be sure to keep notes.

  • How do you know when you are done conducting interviews?

    Saturation (n.) - when the same topics (or themes) keep emerging in your interviews, and conducting

    more interviews results in no new themes.

    Rule of thumb - 12 interviews for saturation

  • Example of one thing you can do with interviews: build robust personas

  • Need Finding

    Refine Build Learn

    Data Collection, Validation, Evaluation

    A/B Testing Contextual Inquiry Diary Studies Ethnography Field Studies

    Focus Groups Hallway Testing Heuristic Evaluation Interviews Lab Testing

    Log Analysis Remote Testing Think-Aloud Surveys User Observation

  • Validation & Evaluation

    Build Learn

    A/B Testing Contextual Inquiry Diary Studies Ethnography Field Studies

    Focus Groups Hallway Testing Heuristic Evaluation Interviews Lab Studies

    Log Analysis Remote Testing Think-Aloud Surveys User Observation

    Data Collection, Validation, Evaluation

  • Types of Usability TestingFormative Summative

    A/B Testing Field Studies Hallway Testing Heuristic Evaluation Interviews Log Analysis Remote Testing Surveys Think-Aloud Wizard of Oz

  • Lab Usability StudiesFormative Summative

    A/B Testing Field Studies Hallway Testing Heuristic Evaluation Hypothesis Testing Interviews Log Analysis Remote Testing Surveys Think-Aloud Wizard of Oz

  • Lab Usability Testing

    http://trydevkit.com/blog-post/a-beginner-s-guide-to-usability-testing/81da0af5-fb17-fd8e-016b-536948e32ced

  • http://trydevkit.com/blog-post/a-beginner-s-guide-to-usability-testing/81da0af5-fb17-fd8e-016b-536948e32ced

  • http://usabilitygeek.com/an-introduction-to-website-usability-testing/

  • are good for.

    learning how easy or difficult it is for users to learn and use your interface

    if language and iconography are intuitive

    how users encounter and recover from errors

    Usability Studies

  • Lab Usability StudiesBenefits

    Cheap (in $)

    Observe user behavior as they encounter a design for the first time

    See the consequences of design decisions first-hand

    Limitations

    Usually takes a moderate amount of time and set-up

    Can sometimes feel staged, or unauthentic

  • Running a Usability StudyPlanning: create test plan, recruit participants

    Pilot: practice with internal users, resolve any technical or logistical issues

    Test session: run test plan, be present (formative) or simply observe (summative)

    Debrief: short Q&A with participants, discuss observations with other study observers

    Analysis

  • Example Usability Study

  • Contextual Usability StudiesFormative Summative

    A/B Testing Field Studies Hallway Testing Heuristic Evaluation Hypothesis Testing Interviews Log Analysis Remote Testing Surveys Think-Aloud Wizard of Oz

  • Field Studies

    http://www.oracle.com/webfolder/ux/applications/getInvolved/customerFeedback.html

  • are good for.

    learning how customers actually use your product in day-to-day life

    Field Studies

  • Field StudiesBenefits

    Allows you to observe authentic, contextual, user behavior

    Can observe the day-to-day experience users have with your product, across a longer period of time

    Limitations

    Significant cost ($)

    Takes more time to run

  • Field Study Example

  • Field Study Example

    One laptop + projector

    Computer Lab

    Laptop Carts

  • Field Study Example[00:24:19.08] Boy says to the girl on his right: "you cheating

    [00:24:21.19] Girl to the left: "what? Its fun. ::mumble:: the simulation. Look.

    [00:24:25.21] The boy looks to the girl on his left, then back to the girl on the right, then down to his workbook in front of him. He puts his head on the table.

  • Contextual Usability StudiesFormative Summative

    A/B Testing Field Studies Hallway Testing Heuristic Evaluation Hypothesis Testing Interviews Log Analysis Remote Testing Surveys Think-Aloud Wizard of Oz

  • A/B Testing

  • A/B Testing

    http://www.smashingmagazine.com/2010/06/the-ultimate-guide-to-a-b-testing/

  • is good for.

    asking how much, how many, which one is better

    sampling from actual users

    testing live apps/services

    A/B Testing

  • A/B TestingBenefits

    Low maintenance: release and wait for data

    Can measure very specific questions

    Live testing, measures actual vs. self-reported user behavior

    Limitations

    Missing context of why users take an action

    May have no results, not guaranteed to be conclusive

    Only measures certain user interactions

  • Running an A/B TestMake sure to:

    test conditions simultaneously (fewer confounding factors)

    be consistent, keep track of which users see which version

    deploy tests cautiously; user research can help inform

    Things to watch out for: dont jump to conclusions, wait for statistical significance

    waiting too long could cost you potential conversions

    you might be interfering with user habit

  • Running an A/B TestMake sure to:

    test conditions simultaneously (fewer confounding factors)

    be consistent, keep track of which users see which version

    deploy tests cautiously; user research can help inform

    Things to watch out for: dont jump to conclusions, wait for statistical significance

    waiting too long could cost you potential conversions

    you might be interfering with user habit

  • https://medium.com/@adlon/threats-of-a-b-tests-and-ux-research-adoption-time-and-incrementalism-991c0c3c61b6

  • A/B Testing Tools

    Optimizely

    Visual Website Optimizer

    Unbounce*

  • Log Analysis

  • is good for.

    seeing page views, entry/exit, platforms, engagement

    sampling from actual users

    testing live apps/services

    Log Analysis

  • Log AnalysisBenefits

    Low maintenance: release and wait for data

    Flexibility, can measure a wide range of data

    Live testing, measures actual vs. self-reported user behavior

    Limitations

    Missing context of why users take an action

    Often requires initial development overhead

  • Example: Google AnalyticsQuestion: how many mobile users does my app have?

    http://www.smallbox.com/blog/quick-and-dirty-ux-3-things-google-analytics-can-tell-you-about-your-users

  • Example: Google AnalyticsQuestion: how many mobile users does my app have?

    http://www.smallbox.com/blog/quick-and-dirty-ux-3-things-google-analytics-can-tell-you-about-your-users

  • Example: Google AnalyticsQuestion: what paths do users take on my site/app?

  • Example: Google AnalyticsQuestion: what paths do users take on my site/app?

  • Example: Google AnalyticsQuestion: what paths do users take on my site/app?

    http://www.smallbox.com/blog/quick-and-dirty-ux-3-things-google-analytics-can-tell-you-about-your-users

  • Example: Google AnalyticsQuestion: how long are users spending on my site?

    http://www.smallbox.com/blog/quick-and-dirty-ux-3-things-google-analytics-can-tell-you-about-your-users

  • Example: Google AnalyticsQuestion: how long are users spending on my site?

    http://www.smallbox.com/blog/quick-and-dirty-ux-3-things-google-analytics-can-tell-you-about-your-users

  • Example: Google AnalyticsCustom logging: track any event you want (links, performance, etc.)

    http://www.sitepoint.com/5-ways-use-google-analytics-ux-research/

  • Usability StudiesFormative Summative

    A/B Testing Field Studies Hallway Testing Heuristic Evaluation Hypothesis Testing Interviews Log Analysis Remote Testing Surveys Think-Aloud Wizard of Oz

  • How to Choose a Usability Study?

    Triangulation (n.) - using two or more methods to discover and validate a finding.

  • Blind Men and the Elephant Parable

  • Consider Tradeoffs and Select Methods that Meet Your Needs

  • Usability StudiesFormative Summative

    A/B Testing Field Studies Hallway Testing Heuristic Evaluation Hypothesis Testing Interviews Log Analysis Remote Testing Surveys Think-Aloud Wizard of Oz

  • Usability StudiesFormative Summative Quantitative Qualitative

    A/B Testing Field Studies Hallway Testing Heuristic Evaluation Hypothesis Testing Interviews Log Analysis Remote Testing Surveys Think-Aloud Wizard of Oz

    what are you testing? what kind of results do you want?

  • Usability StudiesFormative Summative Quantitative Qualitative

    A/B Testing Field Studies Hallway Testing Heuristic Evaluation Hypothesis Testing Interviews Log Analysis Remote Testing Surveys Think-Aloud Wizard of Oz

    what are you testing? what kind of results do you want?

  • Potential Pitfalls of Quantitative Research

    Easy to make mistakes: phantom correlations

    finding may not generalize (participant selection)

    requires sound experimental design

    But its a great supplement to qualitative research

  • Butterfly Ballot

  • Activity - Part 2Reflect on the one interview you conducted.

    How has your knowledge grown? What do you still need to learn about?

    What do you need to get there? (What do you need to do complete a full persona?)

    How would you build a research plan for your topic?

  • Usability StudiesFormative Summative Quantitative Qualitative

    A/B Testing Field Studies Hallway Testing Heuristic Evaluation Hypothesis Testing Interviews Log Analysis Remote Testing Surveys Think-Aloud Wizard of Oz

    what are you testing? what kind of results do you want?

  • Validation & EvaluationA/B Testing Contextual Inquiry Diary Studies Ethnography Field Studies

    Focus Groups Hallway Testing Heuristic Evaluation Interviews Lab Studies

    Log Analysis Remote Testing Think-Aloud Surveys User Observation

    Data Collection, Validation, Evaluation

    Learn

  • http://anotheruxguy.com/2015/07/06/analysis-is-cool/

    Understanding Your Data

  • So Why Doesnt My UI Work?

    Seven Stages of Action

    Mental Models

    Psychological Biases

    Dark UX Patterns

  • Seven Stages of Action

    USER

    SYSTEM...