iwmw 2007: usability testing for the www

A centre of expertise in digital information managementwww.ukoln.ac.uk

UKOLN is supported by:

Usability testing for the WWW

Emma Tonkin UKOLN

www.bath.ac.uk


Introduction• UKOLN, the University of Bath• Why this session?


Why do projects fail?Project Impaired Factors % of the Responses1. Incomplete Requirements 13.1%2. Lack of User Involvement 12.4%3. Lack of Resources 10.6%4. Unrealistic Expectations 9.9%5. Lack of Executive Support 9.3%6. Changing Requirements & Specifications 8.7%7. Lack of Planning 8.1%8. Didn't Need It Any Longer 7.5%9. Lack of IT Management 6.2%10. Technology Illiteracy 4.3%11. Other 9.9%


Introducing usability• Definition: the measure of a product’s

potential to accomplish the goals of a user• How easy a user interface is to understand

and use• Ability of a system to be used [easily?

Efficiently? Quickly?]• The people who use the project can

accomplish their tasks quickly and easily


Assumptions• There are several dimensions to

usability– Focus on users– ‘People use products to be productive’– Users are busy people trying to

accomplish tasks quickly– Users decide when a product is easy to

use• (Adapted from Redish & Dumas, A Practical Guide to User Testing)


However…• Are users always busy? Does this imply that

usability is only present in the workplace?!• Effectiveness vs. efficiency vs. satisfaction• Do users know when a product is ready?• Do all users agree about usability?• Is usability actually measurable?• Is there one statistic that == ‘% usability’?


Elements of usability

• Nielsen refers to five elements or components of usability:– Learnability– Efficiency– Memorability– Errors– Satisfaction – Usability Engineering, 1993, p.26

• These may not be of equal importance in all cases.


In other words…• Usability depends on context

– What does the user want to do?– Who is the user?

• Related to:– Internationalisation; cultural, social issues– Task analysis; working out what the user

wants to do (what the goal is) and how he/she would expect to accomplish it


Science vs craft• Formal approaches:

– Research-driven– ‘hard science’ – Laboratory-based

• Informal approaches:– Naturalistic, qualitative observations– Informal setting


User model vs user testing• Either we apply our understanding of

the way users act, and test the interface that way

• Or we simply observe users...


A note about rule-based testing/validation

• ‘Should be’ vs ‘is’ – model vs reality

• Great handwriting does not guarantee a compellingly readable result


Scenario-based user testing • Based around tasks• Simple scenarios (‘hypothetical

stories’/’abstract-level test cases’):– For a company web page, locating and

using contact details– Registration and login to a wiki

• Process: provide a task and ask the user to complete it– It is important to test the right tasks!


Cognitive walkthrough• Works something like this:

Task: Climb mountain and find the highest peak


Required for CW• A description of the interface• A task scenario• Assumptions: What knowledge does

the user already have?• Functionality: What actions will

accomplish the task with this interface?


Method:• Look at each step that is required to

accomplish the task:– Will the user try this step?– Will the user notice that this action (control,

button, switch) is available?– Will the user associate this action with the effect

that they are hoping for?– If this action is performed, does it appear that

progress is being made?• Can you 'tell a success story' for each step? If not,

there is a usability problem.


Recording your test:• Create a diary format:

– Trying to achieve whatever:• Looking for something that does whatever• Found a button marked foo• But clicking on foo took me to unrelated-

looking screen blah• Like the mountain-climbing line, you

can go back and try another trajectory – document this in a similar way.


Developing appropriate task scenarios• Probably the hardest thing about any

usability testing• On the one hand, you are not required

to support very improbable scenarios. • On the other hand, developing and

supporting probable scenarios is key to a user-centred development process.


Trying out a CW• Who's got a mobile phone?• In groups:

– Work out a couple of tasks.– Working from the perspective of a user

with an appropriate level of knowledge (you will have to define what that means!), try the tasks. Document the result.


User testing (with real users!)• The popular example is heuristic

evaluation.• Heuristics are rules of thumb. • Heuristic evaluation requires about six

people and a large amount of coffee. • Provide them with a list of the ten (or

twelve, or...) heuristics, and ask them to examine each page ('screen') for problems, according to the heuristics.


Ten heuristics• Visibility of system status: Does the system give timely & appropriate

feedback?• Match between system and the real world: Is it speaking the users’ language?• User control and freedom: How hard is it to undo unwanted actions?• Consistency and standards: Does it follow conventions and expectations?• Error prevention: Are potential errors recognised before becoming a problem?• Recognition rather than recall: Does the system rely on the users’ memory? • Aesthetic & minimalist design: Are dialogs cluttered with information?• Help users recognise, diagnose & recover from errors: Are error messages

useful?• Help and documentation: Is there online help? Is it useful?


Evaluating the results• Again, a diary form can be helpful:

'Screen 1 violates heuristic 10 because...'

• Merge these notes. • List by frequency order to see most

obvious bugs• List by heuristic to see severity for your

purposes


Applying the resultsBug fixes

Feature requests

Major objections

Misnamed elementConfusing colours

It would be much easier if…

…‘this textbox autocompleted’…’the system remembered my email preferences’

I don’t like [type of application]I prefer [totally different type of application]

…(Oh)

Strange interaction flow

‘Low-hanging fruit’?


Testing layouts via greeked text• Wasn't going to talk about this, but it's

turned out to be useful• Early stage of web site design often involves

developing layouts/templates• Because no real content exists yet, these

may be hard to test using the above methods

• However, a layout should communicate something about page function. Does it?


Preparing a template• Get greeked text from the Lorem

Ipsum generator:– http://lorem-ipsum.perbang.dk/

• Place it into template. Do not leave a single readable word!

• Make yourself a list of elements that should be visible on the page

• Find/bribe about six test subjects

http://lorem-ipsum.perbang.dk/


Example list:• Main page content• Page title• Person responsible for page• Navigation elements• Last updated date• Logo• How to get back to the front page?• News items


Testing process• One user at a time:• On each layout 'greeked', ask the user to

identify each element or group of elements. If they can't find it, invite them to mark where they think it ought to be.

• Asking the user to 'think aloud' can be helpful• Also, ask the user to give a mark (out of ten, or

from -3 to +3, or whatever...) on 'subjective appeal'

• Note: Randomising order reduces systematic error


Coming up with a preamble• This is a strange thing to ask someone to do.

Do not be surprised if you get some funny looks.• Come up with a short, reassuring introduction to

the test. Useful items to include:– Introducing the software (purpose, not detail)– Your participation will help us...– Remember, we are testing the software, not

your performance...– Please think out loud...– This style of test helps us to...


Examining the results• Build a table:

• As the layout is improved, the number of misidentified elements should reduce

iwmw 2007: usability testing for the www

Education