© simeon keates 2008 usability with project lecture 3 – 16/9/09 dr. simeon keates

© Simeon Keates 2008

Usability with ProjectLecture 3 – 16/9/09Dr. Simeon Keates


Friday’s exercise – part 1

Look at your product selection page

Confirm that your products are in the correct families• Done for most groups – I will check the last 2 groups today…

Develop three (brief) personas/descriptions for users of your site. Explain why each person would want to visit your site and complete this task.


A note on your personas/user descriptions

The final stage of the project will be to compare the usability of your original and final designs

In the final report you will have to explain any design changes that you made

These should be rationalised in terms of the personas that you developed

Note: you can modify your personas/descriptions as the project progresses


Interesting persona attributes

Try to identify persona features that affect how the site is used

For example:• “Taste for the high life” or “Get as much as you can”?• “Time to browse” or “Time is money”?• “Children” or “No children”?• “Experienced computer shopper” or “Novice”?


Notes on different search techniques

“Expert”• Knows exactly what product they are looking for

“Novice”• Probably wants to see everything first before deciding

“Forager”• Keeps looking until they find something worth having• [Picture a wild animal hunting for food – keeps going until something worth

eating is found]

Need to reflect these approaches in the personas…



Estimate the minimum, maximum and expected number of button/key presses for the user to select their desired products

Minimum• Select the 5*, a 4* and a 1* product or 2 x 4* and a 2 *, etc.

• 3 presses for a simple list (2 presses if repeat selection allowed)• 6 presses for product clusters (assuming none in the same category)• 5 presses more likely for product clusters (low * product in same category as one

of the high * ones)• e.g. 5* category, 5* product, 4* category, 4* product, 1* product

• Could be 4 if the 5*, 4* and 1* (or 4*, 3*, 3* or 4*, 4*, 2* …) are in the same category … or if there is a repeat selection (i.e. same 3* product or 4* products chosen)

• … and 3 if that is the default category (or 2 if that includes the 5* product)




Maximum• Assume no exploring or changes of mind• Select 10 x 1* products

• 10 presses for a simple list • 20 presses for 10 product clusters (assuming none in the same category)• If <10 product clusters:• 2 x no_of_product_clusters + (10 – no_of_product_clusters)




Expected• Assume no exploring or changes of mind• Very complex calculation because of 10 * limit!!!• Need to map all of the possible combinations of 10* and then calculate their

probabilities…• Need a computer program for this




Average• 107 stars / 62 products = 1.7258 stars / product• 10 stars => 5.794 products on average • 5.794 presses (on average) for a simple list



Sketch at least one other layout for the 62 products• Suggestions: Simple list, menus, product families, etc.

Compare and discuss the merits of your original design and the alternative one sketched here


Layout options

All groups have tried to cluster products

Many groups have included pictures

Is this necessarily good?

Tesco.com vs. tesco.com/access


Tesco.com


Tesco.com/access


Layout options

All groups have tried to cluster products

Many groups have included pictures

Is this necessarily good?

Tesco.com vs. tesco.com/access

It depends on your selected personas!!!



Q: How would your answer differ for 5 products and for 500 products?

Button press calculations suggest big long list works best for 62 products

Also works best for 5 products

Would not work well for 500 products• Too long to read


The need for critical thinking


The need for critical thinking…

Before we look at methods of usability assessment… Need to consider what we are doing …and how we are doing it See “Stable boy” example from last week

Bad experimental design leads to bad data and wrong conclusions

“There are three kinds of lies: lies, damned lies, and statistics.” – Benjamin Disreali


How can we get “bad” data?

Examples: Not using a balanced experimental design Not using the right participants Asking the wrong questions


Critical thinking exercise – 1

Read the factsheet on DHMO

Q – Do you support a ban on DHMO?

Now read the opposing factsheet

Q – Do you support a ban on DHMO?


Critical thinking exercise – 1

What is DHMO?• Dihydrogen monoxide

Hydrogen H

Dihydrogen H2

Monoxide O

Dihydrogen monoxide H2O


How can we get “bad” data?

Examples: Not using a balanced experimental design Not using the right participants Asking the wrong questions Asking the right questions in the wrong way

http://www.youtube.com/watch?v=2yhN1IDLQjo


Critical thinking - summary

Never take results at face value

Always ask how the data was collected and from whom

Then ask if you believe a sound methodology was used

Have the results been interpreted appropriately?

Have any obvious counter-theories been addressed?


Critical thinking in practice – Speed cameras

Wigantoday.net reports:

Speed cameras cut road fatalities• Published Date: 09 February 2009

“Speed cameras have helped reduce road deaths in Wigan by half, police have revealed.

Six people died on the borough's roads over the past 12 months, compared with a dozen between February 2006 and January 2007.

Countywide, say police, the death toll has also been slashed over the past five years.”

Q – Do you believe these claims?


Speed cameras – the proof!

UK has highest concentration of speed cameras per mile of highway in the world

Reductions in road deaths are often attributed to speed cameras European Transport Safety Council's first road safety report states that

UK deaths decreased by 7% in past years Speed cameras have increased significantly in the same time:

Camera types 2000 2009

Fixed cameras 1,935 5,562

Mobile cameras 173 2,373

=> Reduction is due to increased use of cameras


Speed cameras – the counter arguments

In the same time period 25% drop in Sweden and the Netherlands and 35% in France• … and no increase in speed camera use

5 other countries are safer to drive in Netherlands, Sweden, Switzerland, Norway and Malta• …and minimal use of cameras


Speed cameras – contributing factors

Cars are getting safer Cameras are often used as a cost-cutting tool• Reduced police presence

Cannot deal with unregistered drivers Differences in driver behaviour


Speed cameras – local data

“But the council put that speed camera in where there were 3 accidents last year and there have been none this year. Surely that shows that it is working!”

No – not necessarily If you assume that accidents are randomly distributed, any reduction

could be pure chance• “Regression to the mean”• Any extreme score - high or low - at one point in time will probably be less

extreme the next time it's tested for purely statistical reasons• If you are at the top of a list of accident hotspots, there's only way to go and

that's down


Introduction to “discount” usability…


Simple list is best?

Button press calculation is very simplistic

No allowance for:• Effects of download time• User decision time• User uncertainty, etc.

We will look at this further later in the course


Better methods of evaluating interfaces

“Button press”-type calculations are too simplistic

Need more comprehensive methods of evaluation

Most common: user observation / user trials / user testing*

However, we also need to be fast and fit in with iterative development cycle• c.f. Inclusive design knowledge loop

* Note: try not to use this name… The “t” word is generally best avoided


Q: Why not user observation sessions?

A: Users are not always available

A: User observation sessions are resource-intensive• Time to find users• Time to find location• Time to set up location• Time to brief users• Time for sessions themselves• Time for debrief• Time for data analysis


So they invented “discount usability”…

One interpretation: • How to do usability without users…

Can you think of methods of testing a product that do not involve users?

User models/avatars

Self-assessment

Simulation

Expert assessment


User models/avatars

Method: Use a simulated model of the user to test the product design

Advantages: Great for ergonomic issues

Disadvantages: Hard to correct for user cognitive effects, deviations in behavioural

patterns, etc.


Example


Self-assessment

Method: Design team tests its own design

Advantages: Very fast Very cheap Can be done at any time

Disadvantages: Highly unreliable Results extremely variable

If you can find the problem with a simple test,

why did you design it that way in the first place?


Simulation

Method: Assessor physically recreates the context of use and user

characteristics

Advantages: More reliable than self-assessment More repeatable than self-assessment Can be done at any time, quickly and cheaply

Disadvantages: Only as good as the simulation Recreates “symptoms” not “causes”


Example of simulation in use


Ford focus and the third age suit


Ford Focus and the third age suit


[An aside – The marketing of the Focus]


Discount (DIY) simulation

i.e. discount discount usability


Expert assessment

Method: Find and employ an appropriate “expert” to examine the usability of

your product

Advantages: Perceived to be cheap (no need to acquire expensive skills in-house) Can be very cost-effective

Disadvantages: How do you know you have the “right” expert? Depends on the availability of the expert


Adding rigour to assessments

Need frameworks for ensuring that testing is as reliable, repeatable and robust as possible

Examples: Performance estimates Checklists


Performance estimates

Method: Estimate user performance based on critical path (typically) + known

user performance parameters

Advantages: Great for verifying underlying information architecture

Disadvantages: As discussed with PIP exercise


Checklists

Method: Use a pre-prepared list of questions and ensure that each item is met

Advantages: Fast Can be completed by a non-expert (maybe!)

Disadvantages Culture of “minimum compliance” Only as good as the list


Introducing heuristic evaluation


Other methods of discount usability

Another interpretation: • Discount usability methods are about simplifying the methods of data

collection

Examples: Low-fidelity prototypes Scenarios Thinking aloud usability tests Heuristic evaluation


Low fidelity prototypes

Example: paper prototypes from last week

Scroll up

Scroll down

<Back

Select

Help

Quit

Category 1

Category 2

Category 3

…


Low fidelity prototypes – Why use them?

Discount usability is good for checking “obvious” problems

Best to identify them early

…and fix them early

…then test again…

…and fix again…

…and test again…

Case studies

Design iterations


Scenarios

Scenarios are “extreme” prototyping They reduce level of functionality and number of features to a bare

minimum

As a result, they are cheap to design and implement … but can only simulate the UI as long as the user follows the chosen

path

Source: http://www.useit.com/papers/guerilla_hci.html


Scenarios – a conceptual model

Fu

nc

tio

nal

ity

Horizontal prototypeDifferent features

Vertical prototype FULL SYSTEM

Scenario


“Thinking aloud” usability

A small number of users• Or user stand-ins

Asked to perform typical tasks

Asked to talk through their actions and thoughts while they try to navigate the task

Typically identifies many usability “issues”, not just “problems”


Heuristics

Method: Variant on checklists – a list of attributes (heuristics) to be evaluated Can be done by user or designer

Advantages Fast Well structured

Disadvantages: Can be difficult to interpret Still often a third-party’s best guess of user behaviour


Nielsen on formal usability

Q: How many programmers does it take to change a light bulb? A: None – it’s a hardware problem!

Q: How many usability specialists does it take to change a light bulb? A: Four• 2 to conduct a field study and task analysis to determine whether people really

need light• 1 to observe the user who actually screws in the light bulb• 1 to control the video camera filming the event

Alternative answer:• It depends!!!

Source: http://www.useit.com/papers/guerilla_hci.html


Nielsen’s usability heuristics – in his own words

“I originally developed the heuristics for heuristic evaluation in collaboration with Rolf Molich in 1990 [Molich and Nielsen 1990; Nielsen and Molich 1990]. I since refined the heuristics based on a factor analysis of 249 usability problems [Nielsen 1994a] to derive a set of heuristics with maximum explanatory power, resulting in this revised set of heuristics [Nielsen 1994b].”References:• Molich, R., and Nielsen, J. (1990). Improving a human-computer dialogue,

Communications of the ACM 33, 3 (March), 338-348.• Nielsen, J., and Molich, R. (1990). Heuristic evaluation of user interfaces, Proc.

ACM CHI'90 Conf. (Seattle, WA, 1-5 April), 249-256.• Nielsen, J. (1994a). Enhancing the explanatory power of usability heuristics.

Proc. ACM CHI'94 Conf. (Boston, MA, April 24-28), 152-158.• Nielsen, J. (1994b). Heuristic evaluation. In Nielsen, J., and Mack, R.L. (Eds.),

Usability Inspection Methods, John Wiley & Sons, New York, NY.


Nielsen’s 10 usability heuristics – The list…

Visibility of system status Match between system and the real world User control and freedom Consistency and standards Error prevention Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help users recognize, diagnose, and recover from errors Help and documentation


Recap – Jordan’s 10 principles of usable design

Consistency Compatibility Consideration of user resources Feedback Error prevention and recovery User control Visual clarity Prioritisation of functionality and information Appropriate transfer of technology Explicitness


Visibility of system status

The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.


Match between system and the real world

The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

ConcurDemur

Please depress the third rod linkage with your

dextral lower appendage by rotating it about your talus

OKCancel

Please press the accelerator pedal with your right foot


User control and freedom

Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

OKCancel

Format your hard disk?


Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.


Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

OK Cancel


Windows

OKCancel


Apple OSX


Submit

Error prevention

Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

Error! You must complete all required fields!


Error prevention

Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

Submit

Field 1*

Field 2*

Field 3*

Field 4*

Field 5*

5 required fields still be completed


Recognition rather than recall

Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

Submit

Select product > Confirm payment > Delivery address > Confirmation

Field 1*

Field 2*

Field 3*

Field 4*


Flexibility and efficiency of use

Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.


Aesthetic and minimalist design

Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

Payment information:

MastercardCredit card type*

Number*

mm/yyExpiry date*

CVV2*

Submit

?


Help users recognize, diagnose, and recover from errors

Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

ConcurDemur

XML failure.

Guru meditation code: X001532442XML34542 !

OKCancel

Our coders messed up. Sorry!!


Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.


The science of the heuristics…


Scientific basis for the heuristics (source: Nielsen CHI ’94 paper in reading material)

As mentioned earlier, Nielsen analysed 7 sets of usability heuristics He had 249 usability problems from 11 earlier projects• 7 evaluated by heuristics• 4 evaluated with usability sessions

Stage of project:• 4 were early in their development lifecycle• 7 were later in their development lifecycle

Types of interfaces:• 2 were character-based• 6 were GUIs• 3 were telephone-operated


Comparing the heuristics

The 7 sets of usability heuristics:• Nielsen’s 10 heuristics• “Usability principles for the Star user interface”• “Usability principles” from Holcomb and Tharp• “Design principles for successful guessing” from Polson and Lewis• “Artifact claims analysis questions” from Carroll and Rosson• “Human interface principles” from Macintosh Human Interface Guidelines• “SunSoft usability guidelines”


Comparing the heuristics

Nielsen took each set of heuristics and rated each of the 101 heuristics Rated for how well they explained each of the 249 usability problems• 0 = does not explain at all• 1 = may superficially address some aspect of the problem• 2 = explains a small part of the problem, but major areas unexplained• 3 = explains a major part of the problem, but some aspects unexplained• 4 = fairly, but not totally, complete explanation of why this is a problem• 5 = complete explanation of why this is a problem


Analysing the results

Nielsen performed a “Principal Components Analysis” on the data PCA is a statistical technique for identifying the major factors in a set of

data It reduces the “dimensions” of the data Its aim is to transform the data (through eigenvectors) such that:• The component that explains most of the variance is on the first co-ordinate• The compenent that explains the next most variance is on the second• etc.

“Explains most of the variance”• The proportion to which a mathematical model accounts for the variation (i.e.

apparent randomness) of a given data set


How many factors?

The 2 most important factors (heuristics) only account for 6% of the variation each

The top 7 factors (>3% of variance) only account for 30% of the total The top 25 factors (>1%) account for 62% of total No clear drop-off point• Gradual decline in significance of factors

No “core factor” set

i.e. no “definitive” list of heuristics


Identify meta-heuristics

The seven most “important” factors were:• Factor 1: “Visibility of system status”• Factor 2: “Match between system and real world”• Factor 3: “User control and freedom”• Factor 4: “Consistency and standards”• Factor 5: “Error prevention”• Factor 6: “Recognition rather than recall”• Factor 7: “Flexibility and efficiency of use”

Variance explained:6.1%

5.9%

4.6%

4.2%

3.7%

3.1%

2.8%

Note: These are Nielsen’s names for the factors – hence the relationto the names of his own heuristics.

Other names could have been chosen.


Explanatory coverage

53 factors needed for 90% of variance Too many for a heuristic evaluation!

Need to find a reduced set …but one that offers “pretty good” coverage…


Top heuristics to explain all the usability problems

% explained

Cumulative % explained

A4: Consistency: same thing, same way 23% 23%

A2: Speak user’s language 16% 39%

F7: Feedback: show receipt of user’s input 13% 52%

B2: Seeing/pointing vs. remembering/typing 7% 59%

F10: Aesthetic integrity, keep design simple 7% 65%

A7: Shortcuts and accelerators 6% 71%

G18: Real-world conventions 4% 76%

E18: Help error recognition/recovery 4% 80%

F8: Forgiveness: reversible computer actions 3% 83%

D1: Salient repertoire of available actions 2% 85%


Importance of “serious’ usability problems

Heuristics are known to identify many “minor” problems Often cited as a criticism of this approach

Of the 249 problems, 82 are “serious” Top four heuristics give 70% of coverage c.f. 65% for “all” problems

Table follows…


Top heuristics to explain the serious usability problems

% explained

Cumulative % explained

B2: Seeing/pointing vs. remembering/typing 22% 22%

F4: Consistency: same thing looks the same 18% 40%

G5: Feedback timely and accurate 17% 57%

D1: Salient repertoire of available actions 12% 70%

F8: Forgiveness: reversible computer actions 7% 77%

B1: Familiar user’s conceptual model 5% 82%

F7: Feedback: show receipt of user’s input 5% 87%

A9: Prevent errors from occurring 4% 90%

D5: East to discriminate action alternatives 2% 93%

B7: Modeless interaction 2% 95%


Nielsen’s heuristics applied to real life…

(From: http://www.zenhaiku.com/archives/usability_applied_to_life.html) User control and freedom: • If anything takes up too much of my time, I take a step back and make sure

that it's not taking over me.

Consistency and standards: • I'm very concerned with consistency and standards when it comes to

technology, but my personal life has very little of either.

Error prevention: • I listen to my mom.


Heuristic evaluation of web-sites

Need 3 volunteers…

Perform “think aloud” navigation of 3 web-sites


Perform a “think aloud” walkthrough on this site

http://www.leoburnett.ca/FLASH/index.htm

Task 1: Find out what they do…

Task 2: Try to use the menus at the bottom of the page

Everyone else: What you liked about the site What you did not like about the site Which of the heuristics do these sites break? Suggestions for how to make the site better



http://www.bow-wowbooks.com/ Task: Find the book “Bow-Wow Attracts Opposites”




http://www.havenworks.com/ Task: Find all news stories related to transportation



Summary of discount usability

Designed for rapid testing of products

Great as an integral part of an iterative design approach

Great for identifying problems early in the design lifecycle

Not so great at telling you: “How usable is this system?”

Also not so great at telling you: “How does this compare with my competitors?”


Exercise


Exercise – part 1

Each group will be given a web-site on which to perform a heuristic evaluation

Analyse the site for the particular task that you have been given

Work as a group to analyse the site and identify as many usability shortcomings as possible


Task

A group of Danish students working on a project about British produce decide to find out if British food really is as unhealthy as it sounds…

[…and to bring some back for their lecturer!] The 4 (or 5) students book a cheap flight with Easyjet / SAS to

Stansted / Heathrow and want to hire a car for 3 days to explore the culinary delights of England

Car needs to be large enough for all 4 (or 5) plus luggage. Also needs a child safety seat (for testing the baby products) And 0 GBP insurance excess collision damage cover


Exercise – part 2

Prepare a 5 minute presentation for Friday morning

E-mail it to me, Susanne and Stina by Thursday lunchtime at the latest!

Presentation must address:• Name of site• Type/purpose of site• Task analysed• What you liked about the site• What you did not like about the site• Problems found (number, type, severity)• Suggestions for fixing the problems

© simeon keates 2008 usability with project lecture 3 – 16/9/09 dr. simeon keates

Documents