© idc, iit bombay 1. 2 baddesigns.com © idc, iit bombay 3

84
© IDC, IIT Bombay 1

Upload: joseph-booker

Post on 24-Dec-2015

251 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay1

Page 2: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay2

BadDesigns.com

Page 3: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay3

Page 4: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay4

Page 5: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay5

Page 6: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay6

Page 7: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay7

Page 8: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay8

Page 9: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay9

Page 10: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay10

Page 11: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay11

Page 12: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay12

Page 13: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay13

Page 14: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay14

Page 15: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay15

Page 16: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay16

Page 17: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay17

Page 18: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay18

Page 19: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

19

World Is Full of Unusable Objects

Home assignment– Find 3 unique usability problems in the day-to-day objects

that you have observed or encountered– Describe them as 3 slides in a presentation

Page 20: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

Usability Evaluation

Page 21: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay21

Objective

To identify usability problems in an interface– To improve the design – formative evaluation– To compare competitive products or as acceptance test

– summative evaluation

Page 22: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

22

Many Methods

Heuristic evaluation User Testing

Thinking Aloud Protocol Card sort Feedback from real life usage

– Observational field studies– Logging actual use– User Feedback– Questionnaires and Interviews– Focus groups

Page 23: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

23

Usability Evaluation Methods

Heuristic evaluation– Jakob Nielsen, Usability Engineering, Morgan Kaufman,

1990

Page 24: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay24

Task

For use by a shopkeeper in a kirana shop to make a bill– Find as many usability problems

as you can in 5 minutes in the Standard Windows calculator

For each problem – State it succinctly (if you

thought of a design idea, tell us what problem do you solve)

– Identify a (potential) user experience goal that was not met

– Rate it for severity– Identify layer of user

experience

Page 25: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay25

Task

Analyse the alarm app shown in this video for use by a patient educated till class VII as reminder tool for his medication – Find as many usability problems as you can in 5

minutes

Page 26: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

Analyse the app shown in this videohttp://www.youtube.com/watch?v=s94JxnVwJtI

26

Page 27: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay27

Task

Analyse the alarm app shown in this video for use by a patient educated till class VII as reminder tool for his medication – Find as many usability problems as you can in 5

minutes For each problem

– State it succinctly (if you thought of a design idea, tell us what problem do you solve)

– Identify a (potential) user experience goal that was not met

– Rate it for severity– Identify layer of user experience where the problem

occurs

Page 28: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay28

Heuristic Evaluation Technique

5-10 evaluators examine the product and identify usability problems

It helps to have a mix of usability experts, domain experts and users– Single / double experts– Same evaluators provide consistency across versions

and save time– New evaluators provide fresh eyes

Page 29: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay29

Heuristic Evaluation Technique

Give assistance regarding the domain knowledge– Brief about users, personas, scenarios, primary goals,

and usability goals– Evaluators may be asked to focus on parts of the

product / specific personas Give a checklist of

– HCI heuristics and principles– Platform standards, common practices– Domain standards, common practices

Page 30: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay30

Heuristic Evaluation Technique …

Each evaluator has the freedom to choose his/her method of evaluation– Each evaluator evaluates the product independently– Evaluators should go through product 2-3 times

First iteration– Look for problems novice might face– Analyze Surface and Skeleton issues

Second iteration– Use primary goals, usability goals, heuristics, domain

standards, platform standards to identify problems– Look for conceptual problems, analyze Structure

Third iteration– Look for what is being missed out – Question Scope and Strategy issues

Page 31: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay31

Heuristic Evaluation Technique …

Sometimes, an observer notes the findings– Each of us will note for ourselves here

Observer may give hints about product use – These are evaluators, not users– Save time, capture the problem, and move on– Help only when evaluators ask, after noting problems

Page 32: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay32

Heuristic Evaluation Technique …

After the evaluations, have a debriefing meeting to compile findings

The debriefing meeting also produces design suggestions– Make them sound like suggestions rather than decisions– Do not express only a design idea, point to the problem

first Typical debriefing sessions last 1-4 hours

– Produce preliminary report during the debriefing meeting

– Fine-tune later

Page 33: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay33

Heuristic Evaluation Output

Output of heuristic evaluation is a list of usability problems

Report formats– Introduction, product evaluated, method used– Scenario based problems

~ Enter the cubicle, swipe the card, enter PIN, check balance, withdraw cash, collect cash, card and receipt, exit the cubicle

– Screen based problems in callouts~ Idle screen, test keyboard screen, login screen,

password screen, main menu Common Industry Format (CIF)

~ ISO/IEC 25062:2006

Page 34: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay34

Heuristic Evaluation Some Numbers

5 evaluators find about 75% of the problems 10 evaluators find about 90% of the problems

– HE is prone to reporting false alarms– Don’t negotiate for too long unless it is a very important

issue– If someone thinks it is a problem, capture it

Page 35: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

35

Usability Evaluation Methods

Heuristic evaluation User Testing

Page 36: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay36

User Testing

A representative set of tasks Representative users try to perform these tasks An experimenter conducts the experiment

– Observes users– Captures, interprets and prioritizes problems

Page 37: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay37

Planning a User Test

Determine the key question that the UT should answer~ Should we design an IVR? Can low-tech savvy users

use IVRs? ~ Is the design OK? Should we start the development?~ Can all voters use the new EVM? Can we use it in the

next election?– Subset of usability goals and persona goals

Page 38: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay38

Planning a User Test

Determine the key question that the UT should answer Determine test tasks

– Determine training to be given prior to tasks– Provide a scenario description– Give users a goal to achieve, not instructions to execute– Avoid leading instructions / hints / questions– Set ‘error traps’ and planned disturbances– Try to provide early success experiences

Page 39: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay39

Planning a User Test

Determine the key question that the UT should answer Determine test tasks Determine evaluation criteria

– Determine the output of the UT– Identify performance measures

~ Quantitative~ Success, time, errors, efficiency, satisfaction~ Qualitative~ Ability to handle new situations~ Conceptual understandability of the product~ Questions asked by users~ Training procedure

>> Output of a user test

Page 40: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay40

User Testing Output

Qualitative output~ What is the first reaction?~ What is confusing? What do users ask? Why?~ What do the users like / hate most?~ What features are users likely to use?~ What are user priorities?

– Good for formative evaluation

Page 41: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay41

User Testing Output

Qualitative output Task success

~ Find the share price of Reliance~ Find a good insurance policy

– Levels of success ~ Yes / no~ Successful in first attempt / second attempt / with

help

Page 42: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay42

User Testing Output

Qualitative output Task success Task time

– Tell the participant to be as fast as possible – But avoid declaring

~ “You took 6.3 seconds for that task. Great show.”– Decide when to stop the timer– To probe or not to probe?– Include unsuccessful tasks in timing?

Page 43: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay43

User Testing Output

Qualitative output Task success Task time Errors

– Error “an action of the user that does not accomplish the desired goal”

– Severity of errors “the more difficult it is to recover from the error, more sever it is”

Few users face the problem / problem occurs infrequently

Many users face the problem / problem occurs frequently

Easy to recover Low Medium

Difficult to recover Medium High

Page 44: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay44

User Testing Output

Qualitative output Task success Task time Errors Efficiency

– Efficiency = success + time– Number of mouse clicks– “Lostness” – Smith

Page 45: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay45

User Testing Output

Qualitative output Task success Task time Errors Efficiency Satisfaction scores

– What to measure?~ Expectation, ease, satisfaction, time (no watch),

smiles / frowns – How to measure?

~ Likert scales, semantic differential scales– When to measure?

~ Pre-test, post-test, in the field– Try to make it look anonymous – online reduces bias

Page 46: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay46

Planning a User Test

Determine the key question that the UT should answer Determine test tasks Determine evaluation criteria

– This will be the output of the UT~ Qualitative~ Quantitative: success, time errors, efficiency,

satisfaction– Correctness criteria for each task– Things to measure– Things to watch out for– When to help users

Page 47: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay47

Planning a User Test

Determine the key question that the UT should answer Determine test tasks Determine evaluation criteria Invite / recruit appropriate users

– Matching personas, target markets– Covering independent variables– Professional recruiters– Screener– Telephonic screening

Page 48: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay48

User Testing Screener

Make it short – Less than 20 questions that take about 5 minutes to run

through. Be clear and specific

– Avoid questions that can lead to ambiguous answers Avoid unnecessary questions

– A screener is not an interview– Each question should eliminate some users

Order questions from general to specific – Try to eliminate more people sooner

Communicate the format of the UT and the expected time commitments – Clearly communicate if the user needs to be prepared

in some way Plan on drop outs and no-shows

– Recruit 20% more users than you need

Page 49: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay49

Planning a User Test

Determine the key question that the UT should answer Determine test tasks Determine evaluation criteria Invite / recruit appropriate users Prepare the infrastructure

– Carry out all test tasks to ensure readiness– Carry out pilot tests for practice

Page 50: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay50

User Testing Process

Introduce the user to the topic and put him/her at ease– Explicitly say that it is the product that is being tested– Explain that the product is new and untested– Promise confidentiality, inform about any recordings– Avoid large crowds, conspicuous cameras, interruptions other

than the planned ones Give test tasks one at time Give hints to only if the user is stuck for a long time

– Or if they run into a known usability problem After the tests, debrief the user clarify any doubts Review test plan before repeating with another user

Page 51: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay51

Interview Structure

Start– Break ice – factual questions– Ask 1-2 open ended questions

Middle– Give tasks, one at a time– Contextualize findings – concrete, detailed data– Keep track of test goals

End– Debrief, answer questions, discuss insights– Ask for suggestions

Let the user talk – Avoid interruptions– Try to build a partnership

Page 52: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay52

Questions for Non-directed Interviewing

Don’t imply the answer~ “We put in a lot of effort to design this product. Do

you like it?”

Page 53: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay53

Questions for Non-directed Interviewing

Don’t imply the answer Questions should be non-judgmental

~ “Don’t you think that calorie advice will be more useful as a graph?”

~ “What do you learn from this graph?” or “With this tool select an appropriate dessert for the family.” then observe and probe as appropriate

Page 54: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay54

Questions for Non-directed Interviewing

Don’t imply the answer Questions should be non-judgmental Keep questions “open-ended”

~ “Which of these features is most important to you?” implies “At least some features should be important to you.”

~ “Rate the importance of these features on a scale 1-5”

Page 55: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay55

Questions for Non-directed Interviewing

Don’t imply the answer Questions should be non-judgmental Keep questions open-ended Focus on experience, not extrapolation

~ “Do you think car pooling is a useful feature?” Implies “In the universe of all things, do you think that someone somewhere will find carpooling useful?”

~ “Given your lifestyle and the amount of commuting you do, do you think carpooling valuable to you right now?”

Page 56: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay56

Questions for Non-directed Interviewing

Don’t imply the answer Questions should be non-judgmental Keep questions open-ended Focus on experience, not extrapolation Further, focus on immediate experience

~ “Is a in-car GPS interesting to you?” Implies “Will a in-car GPS facility be of interest to you at some time in your life?”

~ “If your car is equipped with this GPS facility, would you use it?” Then, if yes “How will you use it?”

Page 57: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay57

Questions for Non-directed Interviewing

Don’t imply the answer Questions should be non-judgmental Keep questions open-ended Focus on experience, not extrapolation Further, focus on immediate experience Focus a question on a single topic

~ “How will you use the MP3 option at a party and while driving?”

Page 58: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay58

Running a Non-directed Interview

Define ambiguous terms Restate answers to verify (= share interpretation)

– Listen to user’s questions carefully Use artifacts to ground discussions Don’t force opinions

– Never say that the user is wrong – you are here to find out what the user thinks, not to convince him / her about your opinions

Follow up with examples to probe– But wait for an undirected answer first

Review interview tapes– You will find your own ways of improvement

Page 59: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay59

What can go wrong?

Reliability– “Can we get the result if we repeat the test”– 10x difference in individual computer usage speeds– Background of users

~ Access, age, familiarity with UTs

Page 60: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay60

What can go wrong?

Reliability Validity

– “Does the test reflect the usability issues of interest”– Choice of users

~ Are they a representative sample of real users?– Choice of tasks

~ Are these tasks important? To whom?– Conclusions and their relation with the experiment

~ User took time in first attempt, but this is a frequent task

~ We tested a prototype, then built the whole system, but the prototype was too small, and the system is far too complex

~ We tested the new widget, but the real problem was finding it

~ Probing confusions and evaluating task time at the same time

Page 61: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay61

What can go wrong?

Reliability Validity Experimental design (mostly more validity issues)

– Testing too many things~ The UT took 4 hours, but the user was exhausted~ You cannot text everything, you have to rely on the

abilities and the experience of the designers– Confusing independent & dependent variables

~ Measuring tech-savvyness from task success

Page 62: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay62

What can go wrong?

Reliability Validity Experimental design Experimenter bias

– Experimenter likes one design– The way the task is given– What the experimenter chooses to see, probe

Page 63: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay63

What can go wrong?

Reliability Validity Experimental design Experimenter bias Intervening variables

– Unforeseen circumstances that might interfere with the testing or performance~ Interruptions in task time~ User took three attempts to enter amount (Rs. 500) –

this gave him extra practice with the PIN

Page 64: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay64

User Testing

Gives ‘better’ results than heuristic evaluation– More confidence about findings

Needs more time and money

Page 65: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay65

Task: Find 2-3 UTs of Each Type

Qualitative output Task success Task time Errors Efficiency Satisfaction scores

Page 66: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

66

Usability Evaluation Methods

Heuristic evaluation User Testing

Think Aloud Protocol

Page 67: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay67

Think Aloud Protocol

A set of users use the product while continuously thinking out aloud– An evaluator notes all comments

Users need practice before real tasks– Use a short, familiar but unrelated task

Result is a summary of qualitative data– Excellent for formative evaluation– Can’t get performance data, particularly tasks times

Videotaping sessions helps capture nuances

Page 68: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay68

Thinking Aloud Protocol Variants

Constructive Interaction– Two users use the system together

Retrospective Testing– Let users think aloud in a video-tape playback

Coaching method– Let an expert teach a novice

Page 69: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

69

Usability Evaluation Methods

Heuristic evaluation User Testing

Think Aloud Protocol Card sort

– Louis Rosenfield, Peter Morville, Information Architecture for the World Wide Web

Page 70: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay70

Categorise...

Brinjal

Brush

Camera

Cauliflower

Chicken

DVDs

Electronics

Easel

Eraser

Fruits

Food

Mango

Milk

Mobile phone

Non-veg

Notebook

Pen

Pencil

Potatoes

Raw Banana

Rice

Scale

Shampoo

Stationery

Sweet potatoes

Tomatoes

Toiletries

Toothpaste

Vegetables

Water colours

Page 71: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay71

What did you use as categories? Did you make new categories?

Brinjal

Brush

Camera

Cauliflower

Chicken

DVDs

Electronics

Easel

Eraser

Fruits

Food

Mango

Milk

Mobile phone

Non-veg

Notebook

Pen

Pencil

Potatoes

Raw Banana

Rice

Scale

Shampoo

Stationery

Sweet potatoes

Tomatoes

Toiletries

Toothpaste

Vegetables

Water colours

Page 72: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay72

Where did you put these?

Brinjal

Brush

Camera

Cauliflower

Chicken

DVDs

Electronics

Easel

Eraser

Fruits

Food

Mango

Milk

Mobile phone

Non-veg

Notebook

Pen

Pencil

Potatoes

Raw Banana

Rice

Scale

Shampoo

Stationery

Sweet potatoes

Tomatoes

Toiletries

Toothpaste

Vegetables

Water colours

Page 73: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay73

Card Sort Technique

Make index cards– Categories– Sub-categories– Content– Number the cards (for later analysis)

Page 74: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay74

Card Sort Technique

Make index cards Give a practice task

– People we know– Celebrities– Fruits and vegetables

Page 75: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay75

Card Sort Technique

Make index cards Give a practice task Ask users to

– Make piles of cards– Label piles– One pile could be ‘I don’t care’ pile– Think aloud

Take notes while users work

Page 76: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay76

Card Sort Variables

Open / Closed– Totally open – users write their own cards and

categories– In between – cards are given, users write category and

sub-category labels– Totally closed – pre-labelled cards and categories

– Early on – use open (discovery)– Later on – closed (validation)

Page 77: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay77

Card Sort Variables

Open / Closed Phrasing

– Word, phrase, sentence~ Equipment, Having fun, Honesty is the best policy

– Category with sample sub-categories~ Equipment (Computers, Lathe machines, ACs)

– Picture, question, answer, topic, task

Page 78: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay78

Card Sort Variables

Open / Closed Phrasing Granularity

– High-level or detailed– Paragraphs, pages, main pages, sub-sites– Focussed or generic– Focussed or random

Page 79: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay79

Card Sort Variables

Open / Closed Phrasing Granularity Heterogeneity

– Early on – mix it up– Later on – high consistency

Page 80: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay80

Card Sort Variables

Open / Closed Phrasing Granularity Heterogeneity Cross-listing

– Early on – avoid copies (structure)– Later on – allow copies (navigation)

Page 81: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay81

Card Sort Variables

Open / Closed Phrasing Granularity Heterogeneity Cross-listing Qualitative / Quantitative

– Qualitative suitable for CI / mind-mapping / evaluating IA

– Quantitative suitable for summative evaluation, to do cluster analysis

Page 82: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

© IDC, IIT Bombay82

Card Sort Analysis

Users’ terminologies Mind-maps

– Users’ conceptual models % times two cards go together % times a card goes in a category

Page 83: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

83

Questions?

Heuristic evaluation User Testing

Thinking Aloud Protocol Card sort Feedback from real life usage

– Observational field studies– Logging actual use– User Feedback– Questionnaires and Interviews– Focus groups

Page 84: © IDC, IIT Bombay 1. 2 BadDesigns.com © IDC, IIT Bombay 3

84

Assignment 4

Pick a topic of your interest– Accessibility, HCI and robotics, e-learning,

Read 5 recent papers (2 years) on this topic Prepare a 10-minute presentation on this topic

– Submit a 2-page abstract with references Inform your topics to the Aakash and Abhishek by next class

– Presentations start from the class of 19-3– The presentation slots will be drawn randomly and

announced by 12-3– This assignment will be evaluated by your peers