The ISO Standard and the LAS incident

Michael McDougallCIS 573November 18th, 1999

Last Class

Recall: LAS CAD system Background: Pressure to fix problems

rapidly Planning: Unrealistic budget and deadline Development: No project management, QA

or audit Result: The system failed

Would the failure have occurred if a standard was being followed?

This Class

ISO 12207 - Software life cycle processesPrimary life cycle processesSecondary life cycle processesOrganizational life cycle processes

Would ISO 12207 have made a difference? Yes!Critical requirementsBeneficial requirementsProblems ISO would not address

LAS II - the sequel

ISO 12207 Goals

Establish a common set of terms and techniques that are used when building software

Designed for 2 party situations - not for off-the-shelf software

Processes

ISO 12207 is broken up into 17 life cycle processes

Some processes run as sub-processes of other processes

You can view these as concurrent processes that are running during the software project.

ISO life cycle processes

Primary Acquisition Supply Development Operations Maintenance

Supporting Documentation Configuration

management Quality assurance

Supporting cont. Verification Validation Joint review Audit Problem resolution

Organizational Management Infrastructure Improvement Training

Primary life cycle processes

There are 5 primary life cycle processesAcquisition process

Defines what problem needs to be solvedSupply process

Responsible for supplying the softwareDevelopment process

Responsible for designing, writing and testing the software

Primary processes continued

Operations process Checks the software, operates it, and

offers user support.Maintenance process

“Modifies existing software while preserving its integrity”

Who does what?

In a given software project different parties are responsible for the various primary life cycle processes. Different projects may divide things differently.

Example 1: NASA acts as acquirer IBM acts as supplier, developer,

operator and maintainer.

Who does what? II

Example 2: AT&T acts as acquirer Microsoft acts as supplier, but they

subcontract to … Startup Inc. who acts as developer Once the software is finished Microsoft

gives the code to AT&T who acts as operator and maintainer (and possibly developer).

Supporting life cycle processes

The supporting life cycle processes are processes that the primary life cycle processes can use to make the project progress smoothly.

There are 8 of these processesDocumentation process

Records the information created and needed by primary life cycle processes

Supporting life cycle processes II

Configuration management process Maintains different versions of the

software Evaluates potential changes

Quality assurance process Ensures that everyone is conforming to

the standard and their plans

QA guy

Supporting life cycle processes III

Verification process Ensures that each step yields the required

resultsValidation process

Ensures that the final product actually solves the problem it was intended to solve

Joint review process Evaluates the status of the project One party reviews a separate party

Supporting life cycle processes IV

Audit process Checks that the standard and plans are

being followed One party audits a separate party

Problem resolution process Analyzes and resolves any problems

that arise during the project

Organizational life cycle processes

These processes are responsible for organizing a group of people so that they can accomplish the goal of the project

Not unique to software projectsThere are 4 organizational life cycle

processesManagement process

Plans, manages and evaluates the project

Organizational life cycle processes II

Infrastructure process Manages and maintains the technology

needed by the other processes Example: computers, networks, espresso

machineImprovement process

Evaluates other processes and improves themTraining process

Responsible for training

ISO life cycle processes

Primary Acquisition Supply Development Operations Maintenance

Supporting Documentation Configuration

management Quality assurance

Supporting cont. Verification Validation Joint review Audit Problem resolution

Organizational Management Infrastructure Improvement Training

Applying ISO to LAS

Would ISO have made a difference in the CAD project?

Would ISO have prevented the failure?

YES

ISO requirements

The ISO requirements fall into 3 categories Critical Requirements - If LAS had

followed any of these requirements the failure probably woulf not have happened

Beneficial Requirements - These would not have prevented the failure, but would have helped the project

Null Requirements - No effect on project.

Critical requirements

Acceptance criteria ISO requires the acquirer to define

acceptance criteria. The system will not be accepted until it satisfies such criteria.

Acceptance tests would have shown that the CAD system was not acceptable on Oct 26th.

Critical requirements II

System testing ISO requires that the entire system be

tested. Such testing would have revealed the flaws in the system before it was used.

Critical requirements III

Verification ISO states that the need for verification is

based on:“a) The potential of an undetected

error ...causing death or personal injury”“b) The maturity of and risks associated with

the software technology”“c) Availability of funds and resource”

LAS would warrant rigorous and independent verification

Critical requirements IV

Validation ISO requires that the system be

validated The CAD system would have failed such

a validation test

Beneficial requirements

Bid evaluation ISO requires the supplier to establish a

procedure for evaluating bids LAS would have had to explain their choice of

contractor

Supply process ISO requires that there be a well defined

supply process SO was initially the supplier, but LAS became

supplier

Beneficial requirements II

Human-computer interface ISO requires that the HCI to be considered

by the developer CAD system was frustrating to use

Operations process ISO requires a well defined supply process SO managed system, but they did not

give full-time user support

Beneficial requirements III

Configuration management ISO requires a configuration

management process CAD project did not analyze or track

changes to the software

Beneficial requirements IV

Quality assurance (QA) Required by ISO for all processes LAS had no QA at the system level. Management had no way to gauge

readiness of the systemProblem resolution

CAD developers violated the problem tracking system

Problems outside the scope of ISO

Management - staff relations The LAS inquiry concluded that

“industrial relations” were partly responsible for the failure of the CAD system

but ISO may have helped indirectly

Problems outside the scope of ISO II

Life cycle choice ISO does not forbid using a single-phase

life cycle but ISO would have made single-phase

less attractive

Regard for safety

My analysis assumes that LAS management was well intentioned

ISO does not force parties to value safety

ISO allows tailoring for specific projectsThe standard can be abusedbut ISO makes it harder to ignore

safety - you must explicitly ignore it

LAS II - The Sequel

On January 23rd, 1996, LAS moved to a new £2,000,000 CAD system

The new system is considered an interim solution

New system was prototyped by usersDevelopment was under constant reviewSystem passed user acceptance tests

before being used

Performance

LAS now posts performance data on the web http://www.lond-amb.sthames.nhs.uk/

3 minute activation 1996: 41 %1999: 85 %

8 minute response1996: 16 %1999: 31 %

14 minute response

1996: 76 %

1999: 86 %

New LAS system

Server written in C, runs on UnixWorkstations run terminal emulation

programsWon the British Computer Society

Excellence Award in 1997