Specifying Usability Requirements: The Need for Standards

Mary Theofanos

Brian StantonVisualization and Usability Group

How do you specify usability requirements?

Despite usability horror stories

Usability requirements are rarely identified by organizations purchasing or developing software

What is the state of usability in Government procurements?

Federal agencies have virtually no visibility of software product usability before we make procurement decisions

We do not know how to include usability requirements in procurements.

We do not know how to compare product usability or to plan for or measure usability costs

But Usability can be objectively defined and measured

This implies that we can:

1. Identify usability requirements2. Measure usability before we deploy or

purchase a product

ISO Standards define 3 measures of Usability

Effectiveness -- a measure of user productivity, how well a user can perform his job accurately and completely. (i.e.: completion rate, number of errors)

Efficiency -- a measure of how quickly a user can perform work, the resources expended to accomplish the task. (i.e.: time on task)

Satisfaction--The degree to which users like the product – a subjective response in terms of ease of use, frustration, and usefulness.

Usability: The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use

Example of an Operational Definition for Usability for a travel system:

On their first try, within 15 minutes, 75% of Government travelers shall be able to correctly-

Create a travel request form Select one departure flight and one return flight Designate one hotel Reserve one rental car Forward the travel request form for approval . .

By their second try, within 15 minutes, 90% shall be able to complete the tasks correctly

In 1998, Industry approached NIST to develop a method for factoring usability into procurement decisions

Goal: Increase the visibility of software usabilityReduce uncontrolled overhead costs of software

usability problems, while improving user productivity and morale.

Encourage software suppliers and consumer organizations to work together to understand user needs and tasks.

Define and validate an industry-wide process for providing visibility of software usability to support product decision-making.

Participants span industry, government and academia

Government

Brookhaven National LabsCensus BureauDISADHHSGSAIRSLibrary of CongressOCLCOPMSSAUSDAUS Army Corps of EngineersUS Air ForceState of Georgia, DOT

Universities

Dalian Maritime Univ. China

Rensselaer Polytechnic Inst.

University of IndianaSan Jose StateSEI/CMUShizuoka UniversitySyracuse UniversityUC BerkeleyUCLAUniversity of MarylandUniversity of MichiganUniversity of CopenhagenUniversity College LondonUniversity of Bologna

Industry

Boeing MicrosoftOracleDellFidelity InvestmentsMotorolaAppleGeneral ElectricFordHoneywellPhillipsWhirlpoolSAPState FarmXerox

The role of NIST has been to:

Act as a facilitator in bringing together industry usability professionals, academics, and government representatives.

Maintain the documents and support the ISO standardization process

Collect and analyze data to determine the value of incorporating usability into product decision-making and the impact of the adoption of IUSR “products”.

How do we incorporate usability requirements into the procurement process?

The Common Industry Specification for Usability Requirements

Defining usability requirements in sufficient detail to make an effective contribution to design and development

Defining usability criteria that can be empirically validated subsequently if needed.

Developed by NIST and IUSR to provide a structure for:

CISUR supports communication between usability professionals

Procurers can specify usability in a Request for Proposals or a contract for software

Supplier organizations can determine if usability requirements specified by a customer are realistic for their product, and plan on how to ensure that a product meets these requirements.

Suppliers can assess if the usability requirements specified for product development meet the needs of the customer organizations.

CISUR supports communication within and between organizations

Among members of the development team to specify requirements for use by the development team

Between the customer and supplier of a custom product to define specific customer requirements

Between a range of potential customers and a supplier of an off the shelf product, to define diverse requirements.

The CISUR identifies 3 components to specifying requirements

Context of Use: description of intended users, their goals, equipment, and environment in which product will be used

Performance and satisfaction criteria: ways in which the usability of the product can be measured

Test Methods: how the product will be tested to determine whether the usability requirements have been met

The CISUR identifies 3 levels of specification for the components

The CISUR was developed with the following characteristics: Independent of specific design process Facilitates iterative development of

requirements Complements other user centered design

standards Only applies to usability requirements Does not specify a format for specification As a complement to the CIF

Next Steps

1. ISO Standardization

2. Case Studies

Getting a copy of the CISUR

http://www.nist.gov/iusr

Questions?

Mary Theofanos

NIST

(301) 975-5889

Maryt@nist.gov

Brian Stanton

NIST

(301) 975-2103

brian.stanton@nist.gov

www.nist.gov/iusr