usability methods

Usability Methods to ensure success of your web site

Mike WardeinerKelly Heidman

Institute for International Research

User Experience Design Forum

January 24-25, 2002

San Francisco, California

User Experience Design Forum January 24-25, 2002

2

Case Study

Redesign of the internal home pages for NASA Glenn Research Center

A focus on usability throughout the process ensured a successful release


3

Two Design Efforts

• Information Design• Interface Design

Separate but related efforts following same user-centered design process


4

User-Centered Design Process

Multidisciplinary Project Team Software Engineers Usability Engineer Technical Writers Trainers KM Specialist and Users

User Profiles User Needs and Requirements

Conceptual DesignDetailed Design and Prototyping

Program and TestPrepare Documentationand Help Systems

Finished Product Easy to Learn Easy to Use

Analyze

Test

Improve

Requirements

Analyze

Test

ImproveDesign

Development

Analyze

Test

Improve

ImplementationMarketing strategy

Analyze

Test

Improve


5

Background – Why we needed a better design?

• Image makeover• No one could “find

anything”• Content was scattered• Only two navigational

tools– “Transporter”

– taxonomy

• First design– Developed in early

1999– Taxonomy grew

haphazardly– Linked about 12 web

pages– Index and news


6

What this was like…

• People/Places– Which people/what

places?

• Services Directory– Services to whom?

• Publications – Which publications?


7

1. User Task Analysis

• Understand how the users will use the web site• What tasks are critical• What tasks are most frequent• Step by step how users complete tasks• User interviews, observation to gather input

Usability means designing interface to allow user to perform their tasks efficiently and effectively

See Task Analysis Form in appendix


8

Task Analysis

• Primary user task is to find information • Studied usage statistics to identify trends• Critical tasks are linking a site to homepage and

Transporter • Most frequent task is to find cafeteria menu• Different methods used to locate information (by subject,

organization, search, etc.)

Case Study


9

2. User Profiling

• Identify user groups– Job duties– User tasks to be performed– Level of job experience– Level of computer experience

Identify traits that will make a difference in how a group of users will use the system

See User Profile Form in appendix


10

User Profiles

• Engineers, scientists, technicians, managers, secretaries will all look for different information related to their job

• Different levels of computer/web experience from novice to expert

• Web developers perform some different tasks from other web users

Case Study


11

3. Heuristic Review

• Evaluate interface based on set list of standards/guidelines• Guidelines applicable to the following areas:

– Appearance– Navigation– Functionality– Consistency– Language– User Feedback– Error Prevention and Correction

See Gerry Gaffney Web Site Evaluation Checklist in appendix.


12

Transporter not visible without scrolling

Icons not intuitive buttons, especially NASA logo

Propeller image not intuitive link to home page

List of links does not provide any clear organization of content.

No clear indication of internal vs. external links.

Black background and colored text not easy to read.

Overuse of color for no apparent reason.

Non-standard text hyperlink colors caused confusion in identifying visited links.


17

4. Usability Test Existing Site

• Evaluate existing site to establish baseline user performance

• Baseline performance can be used to set usability goals

• Usability test to discover additional performance problems to improve


18

Usability Test Steps

• Observe users performing typical, real-life tasks• Measure usability by gathering data on the

following areas:– Time to learn task– Time to complete tasks– Quantity of errors while performing task– Retention of learning over time– User subjective satisfaction

• Analyze results, factoring in user profiles


19

Usability Test Documents

• Pretest questionnaire to gather user profile information on evaluators

• Evaluation tasks that target questionable areas of the design

• Data collection forms• Post test questionnaire to gather user opinions• Data analysis summaries• Evaluation and recommendations report

See Pretest Questionnaire, Test Intro, Tracking form, Post test Questionnaire,User Ratings, Test Results, Evaluation and Recommendation reports in Appendix


20

Usability Test Findings

• Established baseline success rate (50% success for locating items within 3 attempts)

• Transporter not used much due to location (not visible) and user lack of understanding

• Users didn’t know what LeWeb is• Users didn’t know how to link web pages to intranet home

page• Users didn’t know to link web pages to Transporter• Users didn’t understand internal vs. external web pages• Users employ different search methods for locating items

Case Study


21

5. Establish Usability Goals

• Usability goals provide target to indicate when design is done

• Goals based on measurable performance– Time to complete tasks– Errors made while performing task– Improved performance over time– How much training / help was needed– Subjective satisfaction of users


22

Usability Goals

• Users should be able to locate 50% of the items on first try

• Users should be able to locate 75% of the items within 3 tries

• Individual web sites no more than 3 clicks away• 75% of users should judge new design to be

better than old design

Case Study

Project team hesitant about setting goals because:

•Change is not well accepted by GRC community

•Users are accustomed to current interface

Information Design

Background, Development, Outcomes


24

What is Information Design?

• Organization/Navigation– Taxonomies, site “blueprints”, metadata, controlled

vocabularies, thesauri, search systems, site maps, indices

• Content Management/Indexing– Content classification, mapping, see and see also

references– What to include and what not (also driven by 508)

Electronic card catalog – finding the information

See Data Organizing samples in appendix


25

Why is this important?

• Managers spend over 6 weeks each year searching for information

• Productivity loss – 35% of productive time spent searching

• Liability for wrong/dated information

Solutions supported via user profiling


26

Information Design Best Practices

• Provide sense of location through taxonomy, hierarchy, navigation, layout, branding

• Addresses goals of user and designer• Breadth versus depth• Consistent via authority rules or hierarchical

structure

Clear to user and always involves USER TESTING


27

Taxonomy Focus

• General principles of scientific classification

• Show groups and relationships• Identify the terms and then decide the

possible term definitions


28

First Homepage Topics

• Authority rules/term focus example:– Connections Link:

• Submissions should be limited to resources of value to the Lewis community


29

Developing the Taxonomy

• Consider– Existing documents, manuals, online content– Description of business groups/processes– Planned applications/processes/future

content

See Information Design Checklist in appendix


30

Developing the Taxonomy

• Depth not > 3 layers• Incorporate key links

– (help, index)

• Focus on business processes

• Apply “custom” authority rules

• Memory through meaningful relationships– Associative learning

• Human indexers• Utilized subcategories

with main topics

Case Study

Center Finance

Budget & Accounting

Buy it, Ship it, Track it


31

Before the Design

• Know:– Your site goals, organization mission– Your audience makeup– Your content inventory (type and

classification)• Ex. Static – privacy statements

• Review similar sites in your industry


32

6. Card Sorting

• Write down each topic on an index card• Give the cards to users and have them

group into similar piles• Avoid pre-labeling groups • Collate the results


33

Card Sort

• Captured existing sites on cards• Team sorted the cards• Tried to fit into JPL’s taxonomy ( built

around their processes) *poor idea• Resulted in our first six categories

Case Study


34

7. Interface/Information Design Prototyping

• Initial “throw-away” design• Low fidelity prototype allows for rapid

change with minimal effort• True interface functionality not required• Hand-drawn on paper is lowest fidelity

Rapid prototyping enables iterative design approach


35

Interface/Information Prototype

• Built html text-only site following taxonomy from card sort

• One hour total development time

Case Study

Prototype of information design provides taxonomy for interface


36

Glenn Research Center

About You and Glenn

Core Research

Finance

Information Depot

Leader's View

Support Services


37


Home

About You and Glenn

Health and Family

Leisure

Money

Safety

Work


38


Home

About You and Glenn - Health / Family

EAP Employee Assistance

Fitness Center

LLF

Medical


39

8. Usability Test Taxonomy

• Purpose is to verify results of card sort with additional users

• “Real” users attempt to locate 30 topics• Use low-fidelity prototype• Gather performance measures for validation• Analyze performance results• Modify information design based on results

Evaluating the information design.


40

Usability Test Results

• Successful find rate only at 73% (within 3 attempts – just under goal of 75%)

• First time success rate at 50% (met goal)• 3 main categories are too unclear• Need to expand from 6 to 8 categories• Needed to relocate topics into other categories• Renamed some main headings• Gathered additional information on user attitudes toward

current intranet

Case Study

User feedback made decisions clear


41

Taxonomy Views

1st Draft 2nd / 3rd Draft Final Version

Core Research Core Research Core Research

Finance Center Finance Center Finance

About Me & Glenn Glenn Workplace Glenn Workplace

Errands & Leisure Personal Activities

Leaders View Corporate Focus Corporate Focus

Support Services Infrastructure & Maintenance

Infrastructure & Maintenance

Information Depot News & Events News & Events

Public Affairs Public Relations


42

Interface Design


43

Interface Design Issues

• Provide new image; cleaner look • Search vs. Transporter• Simplify use of Transporter options if used• Use of “hot links” buttons (need rules to define which

ones get chosen as a “hot link”)• Provide multiple search mechanisms• More intuitive process needed to link web pages to

homepage• Address other identified problems with current interface


44


45

9. Usability Test Interface

• Tasks targeted specific areas:– Search vs. Transporter– Link to NASA main page– Link to GRC external page– Use of “hot link” buttons– How to link a web page– Find an organization’s homepage– Repeated links to topics within the same main category

to test user recall


46

Usability Test Results Case Study

• Successful find rate of over 90% within 3 attempts (surpassed goal of 75%)

• First time success rate of 77% (surpassed goal of 50%)

• Minor name changes will increase success rate• 7 of 9 users preferred new design vs. existing

User performance and user satisfaction surpassed usability goals.


47

Not recognized as links

Search feature was not a Transporter

Hot link buttons not noticed

Taxonomy was intuitive

Easy to link a new page


48

Recommended Changes

• Keep Transporter name and functionality• Make Transporter a main feature• Eliminate NASA logo and GRC hanger as

image links• Remove black background from behind “hot

links” buttons to separate them from header• Modify some link names

Case Study


49


50


51

Highly visible Transporter with easy options

Buttons more obvious when separated from header

All NASA links listed. No image links.

Modified category name


52

10. Implementing Changes

• Allow time in project schedule to make program changes

• Repeat usability testing to verify improvement

• Number of iterations depends on severity of usability issues

Usability evaluations and recommendations are only as effective as the willingness of developers to implement improvements


53

Usability Methods

• Task analysis• User profiles• Heuristic evaluation• Comparison usability testing • Establishing usability goals

Requirements Phase


54

Usability Methods

• Card sorting• Low-fidelity prototyping• Usability testing • Implementing changes in the design

Design Phase


55

Usability Methods

• Usability testing• Implementing changes into design

Development Phase


56

Usability Methods

• User profiles - identified stakeholders• Usability testing - involved stakeholders

and gained ownership• Usability testing - results gained support

and confidence from stakeholders

Implementation Phase


57

Usage Statistics

Before After Monthly hits 149,000 196,000

Transporter use6,000/m 8,000/m

Requests forTransporter keyword 2-3/m 50 in first month

+ 31%

+ 33%


58

Post Implementation Issues

• Less than 20 user complaints received– Personal preferences for category headings– Dissatisfaction of where a page was linked (even

though user found it)– Could not find a page (did not exist or creator never

linked it)

User performance during usability testing provided concrete evidence to support all decisions related to design.

usability methods

Documents

better design

user profilesengineers

user task analysis

task analysisprimary

systemsee user profile

usersuser profiles user

web sitewhat tasks

different tasks