software quality iso-cmm-psp

Quality

0Basic definition of quality:

0Fitness of purpose:0A quality product does exactly what the users want it to do.

2

What is Fitness for Purpose?

0Fitness of purpose for software products means:0Adherence/Satisfaction of the requirements specified in SRS document.

3

Quality for Software Products

Consider a software product:0Functionally correct: 0Performs all functions as specified in the SRS

document.

0But, has an almost unusable user interface. 0Cannot be considered as a quality product.

4

Quality for Software Products

Consider another example:0A product which does everything that users

want.

0But has an almost incomprehensible and unmaintainable code.

0Will you call it a quality product?

5

Modern View of QualityThere are several quality factors that are associated with a software product :

1. Correctness

2. Reliability

3. Efficiency (includes efficiency of resource utilization)

4. Portability

5. Usability

6. Reusability

7. Maintainability

6

Correctness

A software product is correct: 0 If different requirements as specified in the

SRS document have been correctly implemented.

0 Results are accurate.

7

Portability

A software product is said to be portable:

0 If it can be easily made to work0 On different operating systems.

0 On different machines,

0 With other software products, etc.

8

Reusability

A software product has good reusability:0 If different modules of the product can

easily be reused to develop new products.

9

Usability

A software product has good usability:0 If different categories of users (i.e. both expert

and novice users) can easily invoke the functions of the product.

10

Maintainability

A software product is maintainable: 0 If errors can be easily corrected as and when they show

up,

0 New functions can be easily added to the product,

0 Functionalities of the product can be easily modified.

11

Software Quality Management System

Quality management system (or quality system):

0 Principal methodology used by organizations to ensure that the products have desired quality.

12

Quality System

A quality system consists of the following:

1. Managerial Structure

2. Individual Responsibilities.

Responsibility of the organization as a whole.

13

Quality System Activities

1. Auditing of projects

2. Development of:0 standards, procedures, and guidelines.

3. Production of reports for the top management: 0 Summarizing the effectiveness of the quality

system in the organization.

4. Review of the quality system itself.

14

Quality System

A good quality system must be well documented.0Without a properly documented quality

system, 0 Application of quality procedures become ad hoc,

0 Results in large variations in the quality of the products delivered.

15

Quality System

0 An undocumented quality system:0 Sends clear messages to the staff about the attitude of the

organization towards quality assurance.

0 International standards such as ISO 9000 provide:0Guidance on how to organize a quality system.

16

Evolution of Quality Systems

17

Evolution of Quality Systems

0Initial product inspection method:

0 Gave way to quality control (QC).

0Quality control:

0 Not only detect the defective products and eliminate them

0 But also determine the causes behind the defects.

18

Quality Control (QC)

0Quality control aims at correcting the causes of errors:

0Not just rejecting defective products.0 Statistical quality control (SQC):

0Quality of the output of the process is inferred using statistical methods.

0In stead of inspection or testing of all products.

19

Quality Control (QC)

0The next breakthrough:

0Development of quality assurance principles.

20

Quality Assurance

0Basic premise of modern quality assurance:

0If an organization's processes are good and are followed rigorously:

0The products are bound to be of good quality.

21

Quality Assurance

0All modern quality paradigms include:

0Guidance for recognizing, defining, analyzing, and improving the production process.

22

Total Quality Management (TQM)

0TQM advocates:

0Continuous process improvements through process measurements.

23

Process Improvement0 Implies introducing process

changes to improve:1. Product quality

2. Reduce costs

3. Accelerate schedules.

0 Most process improvement work so far has focused on defect reduction. 24

Process Attributes

25

Processcharacteristic

Description

Understandability To whatextent is the process explicitly defined and how easy is it tounderstand the process definition?

Visibility Do the process activities culminate in clear results so that the progressof theprocess is externally visible?

Supportability To what extent can CASE tools be used to support the processactivities?

Acceptability Is the defined process acceptable to and usable by the engineersresponsible for producing the software product?

Reliability Is the processdesigned in such a way thatprocess errorsare avoided ortrapped before they result in product errors?

Robustness Can theprocess continue in spite ofunexpected problems?

Maintainability Can the processevolve to reflectchanging organisational requirementsor identified process improvements?

Rapidity How fast can the process of delivering a system from a givenspecification becompleted?

The Process Improvement Cycle

26

Analyse

Measure

Change

Process Improvement Stages0 Process measurement

0 Attributes of the process are measured. 0 Form a baseline for assessing improvements.

0 Process analysis0 The process is assessed and bottlenecks and weaknesses

are identified.0 Process change

0 Changes to the process that have been identified during the analysis are introduced.

27

Process and Product Quality0 A good process is usually required to produce a good

product.

0 For manufactured goods, process is the principal quality determinant.

0 For design-based activity, other factors are also involved:

0 For example, the capabilities of the designers.

28

ISO 90000ISO (international Standards

Organization):

0a consortium of 63 countries established to formulate and foster standardization.

0ISO published its 9000 series of standards in 1987.

29

What is ISO 9000 Certification?

0ISO 9000 certification:

0Serves as a reference for contract between independent parties.

0The ISO 9000 standard:

0Specifies guidelines for maintaining a quality system.

30

What is ISO 9000 Certification?

0ISO 9000 specifies:

0Guidelines for repeatable and high quality product development.

0Also addresses organizational aspects

0Responsibilities, reporting, procedures, processes, and resources for implementing quality management.

31

ISO 9000

0A set of guidelines for the production process.

0Not directly concerned about the product it self.

0A series of three standards:

0ISO 9001, ISO 9002, and ISO 9003.

32

ISO 9000

0Based on the premise:

0If a proper process is followed for production:

0Good quality products are bound to follow.

33

ISO 9001

0Applies to:

0Organizations engaged in design, development, production, and servicing of goods.

0Applicable to most software development organizations.

34

ISO 90020 ISO 9002 applies to:

0 Organizations who do not design products:

0 but are only involved in production.

0 Examples of this category of industries:

0 Steel or car manufacturing industries

0 Buy the product and plant designs from external sources:

0 only manufacture products.

0 Not applicable to software development organizations.

35

ISO 9003

0ISO 9003 applies to:

0Organizations involved only in installation and testing of the products.

36

ISO 9000 for Software Industry

0 ISO 9000 is a generic standard:

0 Applicable to many industries,

0 Starting from a steel manufacturing industry to a service rendering company.

0 Many clauses of ISO 9000 documents:

0 Use generic terminologies

0 Very difficult to interpret them in the context of software organizations.

37

Software vs. Other Industries

0Very difficult to interpret many clauses for software industry:

0Software development is radically different from development of other products.

38


0 Software is intangible:0 Therefore difficult to control.

0 It is difficult to control anything that we cannot see and feel.

0 In contrast, in a car manufacturing unit:

0 We can see a product being developed through stages such as fitting engine, fitting doors, etc.

0 One can accurately tell about the status of the product at any time.

0 Software project management is an altogether different ball game.

39


0 During software development:

0 The only raw material consumed is data.

0 For any other product development:

0 Lot of raw materials consumed

0 e.g. Steel industry consumes large volumes of iron ore, coal, limestone, etc.

0 ISO 9000 standards have many clauses corresponding to raw material control .

0Not relevant to software organizations.

40


0Radical differences exist between software and other product development:

0Difficult to interpret various clauses of the original ISO standard in the context of software industry.

41

ISO 9000 Part-3

0ISO released a separate document called ISO 9000 part-3 in 1991:

0To help interpret the ISO standard for software industry.

0At present:

0Official guidance is inadequate.

42

ISO 9000: 20000 ISO 9001:2000:

0 Combines the three standards 9001, 9002, and 9003 into one. 0 Design and development procedures are required:

0 Only if a company does in fact engage in the creation of new products.

0 The 2000 version sought to make a radical change in thinking:0 By actually highlighting the concept of process management.

43

ISO 9000: 20000Another goal is to improve effectiveness

via process performance metrics:

0Numerical measurement of the effectiveness of tasks and activities.

0Continual process improvement and tracking customer satisfaction were made explicit.

44

Why Get ISO 9000 Certification?

0Several benefits:

0Confidence of customers in an organization increases.

0If organization qualified for ISO 9001 certification.

0This is especially true in the international market.

45


0Many international software development contracts insist:

0Development organization to have ISO 9000 certification.

46


0Requires:

0A well-documented software production process to be in place.

0Contributes to repeatable and higher quality software.

0Makes development process:

0Focussed, efficient, and cost-effective

47


0Points out the weakness of an organizations:

0Recommends remedial action.

0Sets the basic framework:

0For development of an optimal process and TQM.

48

How to Get ISO 9000 Certification?

0An organization intending to obtain ISO 9000 certification:

0Applies to a ISO 9000 registrar for registration.

0ISO 9000 registration process consists of several stages.

49


0Application stage:

0Applies to a registrar for registration.

0Pre-assessment:

0The registrar makes a rough assessment of the organization.

50


0Document review and adequacy audit:0Process and quality-related

documents.

0The registrar reviews the documents.

0Makes suggestions for improvements. 51


•Compliance audit: The registrar checks:•Whether the suggestions made by

it during review have been complied.

52


0Registration:

0The registrar awards ISO 9000 certificate after successful completions of all previous phases.

0Continued surveillance:

0 The registrar continues monitoring the organization periodically.

53

ISO 9000 Certification

0An ISO certified organization :0Can use the certificate for corporate

advertizements.0Cannot use the certificate to advertize its

products.0ISO 9000 certifies organization's process 0Not any product of the organization.

0An organization using ISO certificate for product advertizements: 0Risks withdrawal of the certificate.

54

Summary of ISO 9001 Requirements

0 Management responsibility(4.1):

0Management must have an effective quality policy.

0The responsibility and authority of all those whose work affects quality:

0 Must be defined and documented.

55

Salient Features of ISO 9001 Requirements

0Important documents independently checked and reviewed:

0 For effectiveness and correctness.

0The product should be tested :

0Against specification.

0Several organizational aspects:

0e.g., management reporting of the quality team.

72

Shortcomings of ISO 9001 Certification

0ISO 9000 requires a production process to be adhered to:

0But does not guarantee the process to be of high quality.

0Does not give any guideline for defining an appropriate process.

73

Shortcomings of ISO 9001 Certification cont…

0ISO 9000 certification process:0Not fool-proof0No international accredition agency

exists. 0Likely variations in the norms of

awarding certificates: 0Among different accredition agencies and

among the registrars.74

Shortcomings of ISO 9001 Certification (3)

0Organizations qualifying for ISO 9001 certification:

0Tend to downplay domain expertise.

0Tend to believe that since a good process is in place,

0Any engineer is as effective as any other engineer in doing any particular activity relating to software development.

75


0In manufacturing industry:0Clear link between process quality and

product quality.0Once a process is calibrated:0Can be run again and again producing quality

goods.

0Software development is a creative process:0Individual skills and experience is

significant. 76


0Many areas of software development are very specialized: 0Special expertize and experience (domain

expertize) required.

0ISO 9001:0Does not automatically lead to continuous

process improvement, 0Does not automatically lead to TQM.

77


0 ISO 9001 addresses mostly management aspects.

0 Techniques specific to software development have been ignored:

0 Configuration management

0 Reviews

0 Release builds

0 Problem Notification system

0 Intranets

78

SEI Capability Maturity Model (CMM)

0Developed by Software Engineering Institute (SEI) of the Carnegie Mellon University, USA:0To assist the U.S. Department of Defense

(DoD) in software acquisition. 0The rationale was to include:0Likely contractor performance as a factor in

contract awards.

79

SEI Capability Maturity Model

0 Major DoD contractors began CMM-based process improvement initiatives:

0 As they vied for DoD contracts.

0 SEI CMM helped organizations:

0 Helped Improve quality of software they developed

0 Realized adoption of SEI CMM model had significant business benefits.

0 Other organizations adopted CMM.

80


0In simple words:

0CMM is a model for apprising the software process maturity of a contractor into different levels.

0Can be used to predict the most likely outcome to be expected:

0 from the next project that the organization undertakes.

81


0Can be used in two ways:

0Capability evaluation

0Software process assessment.

82

Capability Evaluation

0Provides a way to assess the software process capability of an organization:0Helps in selecting a contractor

0Indicates the likely contractor performance.

83

Software Process Assessment

0Used by an organization to assess its current process:

0Suggests ways to improve the process capability.

0This type of assessment is for purely internal use.

84


•The SEI CMM classifies software development industries into:

•Five maturity levels.

•Stages are ordered so that improvements at one stage provide foundations for the next.

•Based on the pioneering work of Philip Crosby.

85


86

Initial (1)

Repeatable (2)

Defined (3)

Managed (4)

Optimizing (5)

Level 1: (Initial)

0Organization operates

0Without any formalized process or project plans

0An organization at this level is characterized by

0Ad hoc and often chaotic activities.

87

Level 1: (Initial)

0Software production processes are not defined,

0Different engineers follow their own process

0Development efforts become chaotic.

0The success of projects depend on individual efforts and heroics.

88

Level 2: (Repeatable)

0Basic project management practices 0Tracking cost, schedule, and functionality

are followed.

0Size and cost estimation techniques: 0Function point analysis, COCOMO, etc. used.

0 Production process is ad hoc:

0Not formally defined0Also not documented.

89

Level 2: (Repeatable)

0Process used for different projects might vary between projects:

0Earlier success on projects with similar applications can be repeated.

0Opportunity to repeat process exist when a company produces a family of products.

90

Level 3: (Defined)

0Management and development activities: 0Defined and documented.

0Common organization-wide understanding of activities, roles, and responsibilities.

91

Level 3: (Defined)

0The process though defined:0Process and product qualities are

not measured.

0ISO 9001 aims at achieving this level.

92

Level 4: (Managed)

0Quantitative quality goals for products are set.

0Software process and product quality are measured:0The measured values are used to control the

product quality.0Results of measurement used to evaluate

project performance: 0Rather than improve process.

93

Level 4: (Managed)

0Organization sets quantitative quality goals.

0World-wide about 100 organizations assessed at this level.

94

Level 5: (Optimizing)

0Statistics collected from process and product measurements are analyzed:0Continuous process improvement based

on the measurements.0Known types of defects are prevented from

recurring by tuning the process

0Lessons learned from specific projects incorporated into the process

95

Level 5: (Optimizing)

0Identify best software engineering practices and innovations:

0Tools, methods, or process are identified.

0Transferred throughout the organization.0 World-wide about 500 organizations have been assessed at

this level.

96

Key Process Areas

0Each level is associated with a key process area (KPA) identifies:

0Where an organization at the previous level must focus to reach this level.

97

Level 2 KPAs

0Software project planning:

0Size, cost, schedule.

0Project monitoring

0Configuration management

0Subcontract management

98

Level 3 KPAs

0Process definition and documentation.

0Reviews

0Training program

99

Level 4 KPAs

0Quantitative measurements.

0Process management.

100

Level 5 KPAs

0Defect prevention.

0Technology change management.

0Process change management.

101

Comparison Between ISO 9001 and SEI CMM

0ISO 9001 awarded by an international standards body:0Can be quoted in official documents

and communications.

0SEI CMM assessment is purely for internal use.

102


0SEI CMM was developed specifically for software industry:

0Addresses many issues specific to software industry.

0SEI goes beyond quality assurance0Aims for TQM.

0ISO 9001 correspond to SEI level 3.

103


0 SEI CMM provides a list of key areas:

0 On which to focus to take an organization from one level to the other

0 Provides a way for gradual quality improvements over several stages.

0 e.g trying to implement a defined process before a repeatable process:

0 Counterproductive as managers are overwhelmed by schedule and budget pressure.

104

CMMI (CMM Integration)

0 CMMI is the successor of the CMM.

0 The CMM was developed from 1987 until 1997.

0 In 2002, CMMI Version 1.1 was released. 0 Version 1.2 followed in August 2006.

0 The goal of the CMMI to integrate many different models into one framework. 0 It was created by members of industry, government and the SEI.

105

Remarks on Quality Model Usage

0 Highly systematic and measured approach to software development process suits certain circumstances0 Negotiated software, safety-critical software, etc.

0 What about small organizations?0 Typically handle applications such as internet, e-comm. 0 Without an established product range,0 Without revenue base, experience on past projects, etc.0 CMM may be incompatible

106

Sl.No ISO CMM

1 ISO is a generic standard. It is applicable for all

industries.CMM is S/W specific

2 ISO is a standard. It tells what needs to be

done, in an organization.

CMM is a model. It doesn’t mandate the

practices

3 ISO focuses on the entire organization’s

processes

CMM is specific to S/W processes

4 ISO Certification is followed by Surveillance

audit once in 6 months

After CMM assessment, there are no such

checks. It is upto the organization to use it for

internal process improvements

5 ISO is continuous CMM is staged (It has different levels of

process maturity)

6 Internal Audits are mandatory Not mandatory

Comparing SW-CMM to CMMI

SW-CMM key process areas CMMI Process Areas

Level 5

Optimizing

Level 4

Managed

Level 3

Defined

Level 2

Repeatable

Defect Prevention

Technology Change Management

Process Change Management

Quantitative Process Management

Software Quality Management

Organization Process Focus

Organization Process Definition

Training Program

Integrated Software Management

Software Product Engineering

Intergroup Coordination

Peer Reviews

Requirements Mgmt

Software Project Planning

Software Project Tracking & Oversight

Software Subcontractor Management

Software Quality Assurance

Software Configuration Management

Causal Analysis and Resolution

Organizational Innovation and Deployment

Organizational Process Performance

Quantitative Project Management

Organizational Process Focus

Organizational Process Definition

Organizational Training

Integrated Project Management

Risk Management

Requirements Development

Technical Solution

Product Integration

Verification

Validation

Decision Analysis and Resolution

Requirements Management

Project Planning

Project Monitoring and Control

Supplier Agreement Management

Product & Process Quality Assurance

Configuration Management

Measurement and Analysis

Personal Software Process (PSP)

0Based on the work of Humphrey.

0PSP is a scaled down version of industrial software process:

0Suitable for individual use.

0Even CMM assumes that engineers use effective personal practices.

109


0A process is the set of steps for doing a job.

0The quality and productivity of an engineer 0Largely determined by his process

0PSP framework:0Helps software engineers to measure and

improve the way they work.110


0Helps developing personal skills and methods.

0Estimating and planning method.

0Shows how to track performance against plans.

0Provides a defined process;0Can be fine tuned by individuals.

0Recognizes that a process for individual use is different from that necessary for a team project.

111

Time Management0Track the way you spend time:

0Boring activities seem longer then actual.

0Interesting activities seem short.

0Record time for:

0Designing

0Writing code

0Compiling

0Testing112


113

Planning

Design

Code

Compile

Test

Postmortem

Logs

Project plan

summary

PSP-Planning0Problem definition0Estimate max, min, and total LOC0Determine minutes/LOC0Calculate max,min, and total

development times0Enter the plan data in project plan

summary form0Record the planned time in Log

114

PSP-Design

0Design the program.0Record the design in specified

format.0Record the Design time in time

recording log.

115

PSP-Code0Implement the design.0Use a standard format for code text.0Record the coding time in time

recording log.

116

PSP-Compile

0Compile the program.0Fix all the defects.0Record compile time in time

recording log.

117

PSP-Test/Postmortem0 Test:

0 Test the program.

0 Fix all the defects found.

0 Record testing time in time recording log.

0 Postmortem:0 Complete project plan summary form with actual time and size

data.

0 Record postmortem time in time record.

118


119

PSP 0

PSP 1

PSP 2

PSP 3

• Personal measurement

• Basic size measures

•Personal planning

• Time and schedule

• Personal quality management

• Design and code reviews

• Personal process

evolution

Six Sigma

0 Six sigma is a quantitative approach to eliminate defects:0 Applicable to all types of industry - from manufacturing,

product development, to service.

0 The statistical representation of Six Sigma quantitatively describes :0 How a process is performing.

120

Six Sigma0 To achieve six sigma:

0 A process must not produce more than 3.4 defects per million opportunities.

0 5 Sigma -> 230 defects per million.

0 4 Sigma -> 6210 defects per million.

0 Six sigma methodologies:0 DMAIC (Define, Measure, Analyze, Improve, Control).

0 DMADV: (Define, Measure, Analyze, Design, Verify).

121

Six Sigma Methodologies

0 The methodologies are implemented by Green belt and Black belt workers:0 Supervised by Master black belt worker.

0 Pareto Chart:0 Simple bar chart to represent defect data

0 Identify the problems that occurs with greatest frequency

0 or incur the highest cost

122