some usability engineering...

47
R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected] i Some Usability Engineering Methods Randolph Bias 2/12/2014

Upload: others

Post on 12-Jun-2020

6 views

Category:

Documents


0 download

TRANSCRIPT

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i

Some Usability Engineering Methods

Randolph Bias 2/12/2014

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Today

– Discount usability evaluation methods •  Pluralistic usability walkthrough demo •  HE exercise

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i General, kinda, process flow •  Someone has an idea for a product/site.

–  Maybe there was a problem that needs to be fixed, or an identified efficiency

•  Gotta figure out WHAT to build – Gather user requirements

•  Build something – Scientific underpinnings, Design support

•  Don’t be satisfied with the first design – employ iterative design approach -- Evaluation

•  Don’t just throw your findings “over the transom” -- Advocacy

3

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i The Methods of Usability Engineering . . .

•  Are employed in order to enable you to bring user data (empiricism) to bear on the emerging product design.

•  You (the usability engineer) become an advocate for the user, in the product development process.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Last week we talked about the Gold Standard: End-user Testing

•  Also called “lab testing” •  Can be done on paper-and-pencil

design, prototype, early code, existing product

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i EUT - Benefits

•  Gather performance and satisfaction data – Performance data: time on task, error rates,

# calls to the help desk, # references to the documentation, . . . .

– Satisfaction data: End-of-test questionnaire •  Can be find-and-fix or benchmarking •  Ensure coverage of certain parts of the

UI – we have good control over the tasks

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i EUT - Limitations

•  Artificial situation •  Successful test doesn’t “prove” the

product works – Aside – It’s ALL about confidence.

•  Need representative users! •  Ease of learning vs. ease of use •  Hard to test longitudinally

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i EUT -- What to test?

•  Can rarely cover all the UI. •  I like to test:

– critical tasks –  frequent tasks – nettlesome tasks

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Rubin’s 4 Types •  Exploratory (working on the “skeleton” –

maybe the Information Architecture) •  Assessment test (working on the “meat and

flesh” of the UI) •  Validation test (does it meet the objectives?) •  Comparison test (compare two competing

designs)

–  Rubin, J. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Wiley: New York, NY, 1994. (Superseded by Rubin and Chisnell.)

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Set up •  Create environment (ambient setting, HW,

SW) •  Identify participants •  Establish test roles (test monitor/

administrator, data logger, timer, video operator, product experts (SMEs), other observers)

•  Create test plan •  Prepare observers •  Prepare test materials

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i What materials? •  Instructions •  Informed consent form •  NDA •  Permission to videotape •  Test scenarios •  Questionnaire(s)

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Conduct the Test •  Welcome the test participant •  Communicate that this is not a test of THEM,

and they can leave any time •  The scenarios should match the real world

setting •  Ask the test participants to “think aloud” to

better understand intent •  Offer post-test questionnaire, and debrief

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i After the Test

•  Quick data to product team •  Assign severities and build

recommendations •  Build archival report •  Serve as change agents!

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i One big problem

•  Cost

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Three Methods to address cost

•  Remote End-user testing (lab testing) (more in another class, in March)

•  Heuristic Evaluations •  Usability Walkthroughs

Today let’s talk about WHY and WHEN we employ one method or another, and HOW to carry them out.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Remote EUT

•  Saves tons of travel money •  Allows you to get otherwise hard-to-get test

participants. •  Allows them to be in their own environments. •  Might allow product designers/developers to

watch from their own office. •  But . . . lose some fidelity of the test

environment (video?) •  Some added set-up cost (time)

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i What is a Heuristic Evaluation?

Evaluators systematically inspect the application interface to check for compliance with recognized usability guidelines (heuristics). (Thus, an INSPECTION method.)

-  Identifies major and minor usability problems - Conducted by three to five experienced usability engineers (or one!) - Problems are reported along with the heuristic it violates

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Problems Identified

The probability of one evaluator finding . . . –  A major usability problem - 42% * –  A minor usability problem - 32% More evaluators = more problems identified

*from www.useit.com

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Strengths of an HE

- Done by people experienced in usability not just “dumb” users - Can identify both major and minor usability problems - Can be done relatively quickly and inexpensively - UNlike EUT, can sometimes cover every corner of a UI or web site

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Weaknesses of an HE

-  If done at end of design, designers may be resistant to changes -  Some designers/developers may be unmoved by “just opinions” -  Experienced usability evaluators may miss content problems that actual users would find

- Can HELP address this issue by using SMEs

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Typical Methodology

- Interface is exercised -1st pass to develop the big picture -2nd pass to accomplish typical tasks

- Each problem is reported along with the heuristic it violates - Comments are consolidated - Severity levels – Critical, Major, Moderate, Minor

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Nielsen’s Usability Heuristics

•  Visibility of system status –  The system should always keep users informed about what is going on, through appropriate feedback

within reasonable time.

•  Match between system and the real world –  The system should speak the users' language, with words, phrases and concepts familiar to the user,

rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

•  User control and freedom –  Users often choose system functions by mistake and will need a clearly marked "emergency exit" to

leave the unwanted state without having to go through an extended dialogue. Support undo and redo. •  Consistency and standards

–  Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

•  Error prevention –  Even better than good error messages is a careful design which prevents a problem from occurring in

the first place.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Nielsen’s Heuristics (cont’d.) •  Recognition rather than recall

–  Make objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

•  Flexibility and efficiency of use –  Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such

that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

•  Aesthetic and minimalist design –  Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of

information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

•  Help users recognize, diagnose, and recover from errors –  Error messages should be expressed in plain language (no codes), precisely indicate the problem, and

constructively suggest a solution.

•  Help and documentation

–  Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's

task, list concrete steps to be carried out, and not be too large.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Severity Levels Used

Major Usability Issue –

-  Completed task but considerable frustration or extra steps

Critical Usability Issue:

-  Loss of user data

-  System shutdown

-  Abandoned task

Moderate Usability Issue –

- Moderate work around or multiple attempts

* Usability Suggestion

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Sample Summary of Results

38 4 0 Totals

2 0 0 Help

2 0 0 Icons

7 0 0 Main IDE

1 0 0 Debug

1 0 0 Run Project

1 1 0 Code Editor

16 3 0 Visual Layout

5 0 0 Project Creation

3 0 0 Installation

MINOR MAJOR CRITICAL TYPE

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i How to . . .

•  http://www.useit.com/papers/heuristic/heuristic_evaluation.html

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Other Heuristics

•  http://www.stcsig.org/usability/topics/articles/he-checklist.html

•  http://www.asktog.com/basics/firstPrinciples.html

•  http://designingwebinterfaces.com/6-tips-for-a-great-flex-ux-part-5

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Let’s do one.

•  Pair up as before. •  First, take 20 minutes to go through and

conduct your own, individual HE of weather.com. –  Identify some number of problems – Associate each with a heuristic it violates – Assign severities

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i HE (cont’d.) •  First pass, general look. •  Second pass, carry out some specific tasks

– Add a city to your list of cities presented at the top of the page

– Remove that city – Find current radar map for Austin – Find the weather for Austin tomorrow at

4:00 p.m. •  After about 20 minutes, pair up and compare,

aggregate. Reconvene at ______.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Welcome . . .

•  . . . to today’s Usability Walkthrough. •  You are here to evaluate the Zara Tours

web site. •  This is a test of the site, not of you.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Usability Walkthroughs

Purpose

•  To collect user data -- from multiple users at one time -- to help drive the design of a user interface

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Context

•  This is just one step in a series of “User-Centered Design” methods we’re employing: – Heuristic evaluation (professional judgment) – Usability walkthrough (YOU ARE HERE) – End-user testing in the lab

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Key Characteristics •  Three types of people in the room •  We’re going to go through some task flows, as a

group. •  You’ll have hard-copy packets, where you will write the

actions you would take, if you were online, carrying out a certain, prescribed task.

•  No discussion until all have written a response. •  We’ll announce the “correct” action (according to the

current design). •  We’ll discuss the page -- representative users first.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Please note . . .

  Design   is

 HARD   SW development teams budget time to debug

the code; we’re debugging the design!

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i This is a test of the DESIGN!

•  Not a test of YOU. •  If you have trouble finding the “right”

answers, then WE have a problem with the UI.

•  The site design team is being very bold, to expose their design to users early like this -- but they’re doing it because they realize the benefit, and they want to get it right!

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i And so . . .

•  Given the input we hear today, the site design team (assisted by a usability professional) will redesign the interface.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i The Flow of the Day

•  We’ll hand you a piece of paper with a scenario description on it.

•  We’ll hand you a packet of screen shots, in order. DO NOT LOOK AHEAD, please.

•  We’ll ask you to write down on the page what action you’d take, to accomplish the task in the scenario.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i The Flow (cont’d.)

•  We’ll announce the “right” answer. •  I’ll ask you to indicate if you got it right.

When you did not, I’ll welcome discussion.

•  I’ll welcome the designers and developers to jump in, as discussion winds down.

•  I’ll let SOME redesign go on, real-time. •  Write more comments under the screen.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Yet more flow . . .

•  Then you’ll be asked to turn the page. •  Now, you’ll have to “reset,” and assume

you got to this new page (somehow). •  Then, what action would you take on

THIS page, while still trying to accomplish the task?

•  After a scenario, we’ll give you a questionnaire to complete.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Benefits

•  Lot of data early in the design cycle. •  Usability of individual screens,

terminology, SOME task flow. •  Collaborative redesign on the fly.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Limitations

•  Can’t get some data (e.g., time on task). •  You can’t browse, as you might online --

tendency to “lose your place.” •  Feel free to turn BACK in your packet,

but not ahead.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Any questions?

•  We’ll try to finish up by 2:30 or so. •  Then have some more informal time to

discuss the interface. •  We may get more informal at points. •  THANK YOU very much for being here. •  Absolutely NO ONE should be

embarrassed -- not the users, not the developers.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i OK, first task:

•  Task 1: Find how many days it takes to make the trek up Mt. Kilimanjaro via the Machame Route.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Which, Why, When?

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Check it out

•  This is a pretty good representation re ROI:

http://www.youtube.com/watch?v=O94kYyzqvTc&fee=aturrelated

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i Next Week

•  Book reviews •  Remember – 4-minute presentations

–  If you wish to have a ppt accompany your presentation, please send it to Dan by noon, Friday.

•  It’s gonna be like we read 20 books!! •  White papers due in 3 weeks. •  Test plans due in 6 weeks.

R. G. Bias | School of Information | SZB 562B | Phone: 512 471 7046 | [email protected]

i OK, let’s talk about those test plans!