Usability Engineering Presentationfor

October 14, 2010

Presented by Judy Ketterer, Usability Analyst

What is this field of Human-Computer Interaction?

“People are quite different from computers. This is hardly a novel observation, but whenever

people use computers, there is necessarily a zone of mutual accommodation and this

defines our area of interest.

People are so adaptable that they are capable of shouldering the entire burden of

accommodation to an artifact, but skillful designers make large parts of this burden vanish

by adapting the artifact to its users.

To understand successful design requires an understanding of

the technology,

the person,

and their mutual interaction.”

Stephen Draper and Donald Norman

User Centered System Design

(1986)

So…So…

Why are there so many high-tech products, such as

computer-based systems, electronic equipment, and

even every day appliances, so hard to use??

(The Handbook of Usability Testing – Jeff Rubin)

Usability Engineering: The Whole PackageUsability Engineering: The Whole Package

When addressing “Usability” think of all of these:

User and task analysis

User interface design

Usability testing

Accessibility

UE Service: User and Task AnalysisUE Service: User and Task Analysis

User and Task Analysis

o Process of identifying who will use the software and what tasks they will complete with it

o Done by observing people in their workplace, and interviewing them about their jobs, frustrations, tasks, and the information

they have when performing those tasks

o Outcome is a set of user profiles, task scenarios, and a user-task matrix, which is used to design user interfaces that support

the tasks that people must accomplish with the system

Benefits

o Foundation on which you build your application

o Understanding users and their tasks early will help avoid costly missed requirements later

o Able to provide user interfaces that more closely meet the needs of your users

o Able to lower training and support costs

When It Should Be Done

o Occurs at the beginning of the project, at the same time as requirements-gathering and before 

user interface design or coding begin

UE Service: User Interface DesignUE Service: User Interface Design

User Interface Design

o Working out the layout, content, controls, and structure of the user interface – the part of a system or product with which the

user interacts to complete tasks

o Identifying the number and type of screens the user will see, the sequence and manner in which they will be accessed, and the

content and controls that will appear on them

o Understanding the users and their tasks - you can provide an application with a nice layout and standard controls, but if you do

not understand the users and what they're trying to accomplish, your application can still be highly unusable

Benefits

o Screen mockups help the project team and users understand how they will perform their tasks in the new application

o Testing screen mockups means you can make changes on paper, before beginning the more costly coding effort

o Attention to user interface design means screen flows that support user tasks, controls that match user expectations,

and screens that are professional and polished

o User interface specifications capture screen shots and descriptions of how the interface looks and functions

(based on meetings with the project teams and users)

When It Should Be Done

o Before user interface coding begins

UE Service: Usability TestingUE Service: Usability Testing

Usability Testing

o Includes a range of methods for having users try out a software product in a real-world environment

o In a typical usability test lab, users perform a variety of tasks with a prototype (or the actual system) while observers record

notes on what each user does and says

o Testing gathers information about the paths users take to complete tasks, the difficulties they encounter, whether they succeed

in completing tasks, and user satisfaction

Benefits

o Accurately describes what is both right and wrong with a system and enables you to keep the good elements, improve on what

is weak, explore new areas users identify, and architect new areas of development

o Provide the opportunity to fix problems before they are released to production

o Decreases customer support needs

o Increases customer buy-in

When It Should Be Done

o Early and often; in the design phases and in iterations using paper and/or electronic prototypes

Impact Without TestingImpact Without Testing

“The rule of thumb in many usability-aware organizations

is that the cost-benefit ratio for usability is

$1:$10:$100

Once a system is in development,

correcting a problem costs 10 times

as much as fixing the same problem in design.

If the system has been released,

it costs 100 times as much relative to fixing the design.”

– Tom Gilb

The “Ideal” Testing SetupThe “Ideal” Testing Setup

Testing Room Observation RoomTesting PC Remote View PCs

2-way window

You Can Get By With…You Can Get By With…

Conference Room / User Workstation

What’s most important is going to the user for their input.

More About The Usability Testing ProcessMore About The Usability Testing Process

Planning Meeting

Testing Scripts / Tasks / Scenarios

Post-Session Survey

Conducting the Test

Findings Report

Step 1: Planning Meeting with Your CustomerStep 1: Planning Meeting with Your Customer

Meet to discuss:

o Your customer’s needs - what is being tested

o Customer’s goals and objectives of testing

o Location

o Length of time for one lab session

o Number of participants (6-8)

- Identifying / Personas

o Number of testing days

o Lab Script / Tasks / Scenarios

o Post-Session Survey

o Pilot Test

- Verify product functioning as expected

- Review script and survey

o Findings Report

- Format, severity codes, date distributed

Testing Script / Tasks / Scenarios ExamplesTesting Script / Tasks / Scenarios Examples

Script mimics real-life scenarios and user tasks.

Examples:

Company Web SiteLocate information related to the company’s products.How can you contact the company?How would you research careers with the company?

Retail Web SiteHow would you shop for something specific?How would you compare prices for various items?Can you determine different shipping methods for your product?

Business Process Application(Provide specific data for entry) Enter required data for processing.Determine next steps based on directives on screen.Submit and confirm data.

Post-Session Survey QuestionsPost-Session Survey Questions

o Make customer-specific

o Address at least these 5 issues:

Learnability: How easy was it for you to accomplish the tasks?

Efficiency: Once you were more familiar with the design, how quickly did you feel you could perform the tasks?

Memorability: Do you feel it was easy to recall the steps you needed to follow to complete the tasks?

Errors: Do you feel it was easy to avoid errors, or when an error occurred, it was easy to correct?

Satisfaction: How pleased are you with the design?

o You can use a rating scale such as:

1 2 3 4 5

difficult very easy

complicated uncomplicated

unsatisfied very satisfied

confusing clear

o You can also ask about :

Training needs

What do you like best

What do you like least

Recommendations/Suggestions

Step 2: Conducting the Test - Guidelines / QuestionsStep 2: Conducting the Test - Guidelines / Questions

General Guidelineso Brief participants ahead of time for what you’re looking for and what they’re being asked to do

o Give participants time to think; what they see is new to them so they need time to acclimate themselves to the format

o Don’t guide the participant; you want to see how they naturally use the system and how intuitive it is

o Let the participant make mistakes, but don’t let them get too frustrated (this identifies the problem areas)

o Keep the participant talking

o Try not to answer questions; you want to observe the unaided user (answer questions with a question)

o Never get defensive, the participant is only identifying the roadblocks and problems areas, not being critical

Suggested Questions and Promptso What are you thinking or trying to do now? (generic prompt to get the participant talking)

o What do you think you can do in this context?

o Please describe the parts of the screen (identifies how the user is interpreting what they see)

o What do you think will happen?

o Is that what you expected?

o How can this be improved?

Questions and Prompts to AVOIDo Why did you do that? or You’re not supposed to do that.

o Don’t do that again!

o Hurry up, you’re running out of time.

As The Tester You Will…As The Tester You Will…

Get Participants “Unstuck”o Find the balance between sufficient and too much assistance

o Remember, once you tell participants how to do something, you forever lose the opportunity to learn why it was

confusing and not intuitive for them

o Provide help when participants show signs of distress or observers are getting restless and testing needs to move on

o Use questions rather than answers whenever possible – this encourages the user to explain what they are thinking,

trying to do, or what they would expect would happen

o Move from general to specific

What do you think the next step is?

Do you see anything on the screen that might help you?

What do you think the “xx” button does?

Guide Observerso Participant should have their full attention

o Follow the General Rules of Testing

o Avoid interrupting the participant unnecessarily; keep questions until the participant has finished their thoughts or

comments

REMEMBER:

Usability is about people NOT products

Focus groups are NOT usability tests

Step 3: The Findings ReportStep 3: The Findings Report

Title Pageo Important to have your contact information listed

Overviewo What was testedo Why it was testedo Participant personaso Any “unique” aspects of the report

Performance Resultso Your observationso User comments and/or recommendations (with Usability Defect Severity Classification)o Comparisons/Statistics/Percentageso Video clips

Post-Session Surveyo Survey results

Usability Defect Severity ClassificationUsability Defect Severity Classification

Classifying “defects” allows your customer to identify what is most important to address.

Critical - showstopper; user unable to proceed; system failure; data lost

o Critical data is lost because users are expected to “know” to save their data or do something that is complex

to preserve the data. For example, if the application allows users to exit without prompting them to save

their changes or asking if they want to save a session.

o The usability problem is likely to result in an error that will cost the customer a large amount of wasted time

or money (for example, a problem that would delay the recovery of a transaction database.)

High - user requires assistance to proceed; continued operation impaired

o There is no keyboard access method for a feature.

o The way to work around the problem is difficult to explain and/or remember.

o The problem is likely to result in many support calls.

Medium - functionality missed; minor problem causing user confusion or irritation

o A common menu item or toolbar button does not do what is normally expected.

o Missing or incorrect titles or units.

Low - aesthetic change or user preference

o Typos, colors, animation, etc.

o Design is different from current state or is unfavorable to user.

NR (not rated) - usability analyst / customer observation or comment

Inclusive Design: Where Usability and Accessibility IntersectInclusive Design: Where Usability and Accessibility Intersect

"Accessibility" generally describes the degree to which something is usable by as many people as

possible. The term is usually focused on people with disabilities.

REMEMBER

Just because something is “Usable” it doesn’t mean it is “Accessible”

and

Just because something is “Accessible” it doesn’t mean it is “Usable”

Four Key Points for Accessibility

To be accessible and usable, the user needs to know:

If something exists

What it is

Where it is

What you can do with it

Accessibility ToolsAccessibility Tools

Check Web accessibility

Worldwide Web Consortium web site

Screen Reader

Freedom Scientific's JAWS®

System Access to Go

satogo.com

Browser Toolbar for MS Internet Explorer

Vision Australia Web Accessibility Toolbar

Questions?

Thank you

Judy Ketterer, Usability Analyst