quality management systems: methods and techniques of software quality management

Software MethodsMö/3.6.2002

slide 1Methods and Techniques of Software Quality Management

ICEL

Quality Management Systems:Methods and Techniques of

Software Quality Management

Karl Heinrich MöllerGaertnerstr. 29

D-82194 GroebenzellTel: +49(8142)570144Fax: +49(8142)570145

Email: [email protected]



ICEL

• Definitions

• User friendliness

• Software Development as a Process

• Faults as a cost factor

• Fault causing factors

• Strategies to fault reduction

Topics



ICEL

Quality:Degree to which a set of inherent characteristics fulfils requirements

Note 1:The term quality can be used with adjectives such as poor or excellent

Definition of Quality

ISO 9000:2000; Par. 3.1.1



ICEL

Characteristic:Distinguishing feature

Note 1:A characteristic can be inherent or assigned

Note 2:A characteristic can be qualitative or quantitative

Note 2:There are various classes of characteristics(physical, sensory, behavioural, temporal, etc.)

Definition of Characteristic

ISO 9000:2000; Par. 3.5.1



ICEL

Nonconformity:Non - fulfilment of a requirement

Definition of Nonconformity

ISO 9000:2000; Par. 3.6.2



ICEL

Defect:Non - fulfilment of a requirement related to an intended or specified use

Note:The distinction between the concepts defect and nonconformity is important as it has legal connotations, particularly those associated with product liability.

Definition of Defect

ISO 9000:2000; Par. 3.6.3



ICEL

Quality-related Costs :Those costs incurred in ensuring and assuring satisfactory quality, as well as the losses incurred when satisfactory quality is not achieved

Quality losses: Losses caused by not realizing the potential of resources in processes and activities

Definition of Quality Cost and Losses

ISO 8402:1995; Par. 4.2 and 4.3



ICEL

Defect Prevention Costs:Group of quality cost elements to collect costs which will be caused by taking measures of prevention and correction in order of QM

Test CostsGroup of quality cost elements to collect costs which will be caused by all planned quality tests

Defect Costs Group of quality cost elements to collect costs which will be caused by the losses incurred when satisfactory quality is not achieved

Definition of Quality Cost Elements

Internationally not defined



ICEL

Development of software should incorporate two characteristics:

Development process of high quality (will result in high productivity, efficiency)

Products with high customer value (will result in high sales)

--> Assuring the economic survival

Quality Objectives of Software Development



ICEL

1. FunctionalitySuitability, Accuracy, Interoperability, Security, Functionality Compliance

2. ReliabilityMaturity, Fault Tolerance, Recoverability, Reliability Compliance

3. UsabilityUnderstand ability, Learn ability, Operability, Attractiveness, Usability Compliance

4. EfficiencyTime Behaviour, Resource Utilisation, Efficiency Compliance

5. MaintainabilityAnalysability, Changeability, Stability, Testability, Maintainability Compliance

6. Portability Adaptability, Install ability, Co-Existence, Replace ability, Portability Compliance

ISO 9126:2001 Software Engineering; Quality Model



ICEL

User Friendliness

Requirement Product Characteristic

Usability

Portability

Reliability

Efficiency

Design Characteristic

Hardware Independency

Insularity

Completeness

Robustness

Consistency

Hardware Efficiency

Accessibility

Connect ability

Self-explanatoribility

Diagnose ability

Upgradeability

Accuracy

Readability

User Adaptability

Testability

Comprehensibility

Changeability



ICEL

Development Process as Waterfall Model

Market Require-ments

Product Concept

Product Defini-tion

DesignImplem.

& Test

Integr. &

funct. Test

System Test

Real Product

Product Maintenance

Customer Satisfaction

Development Process

Product

Planning

Field

Defects

Realisation



ICEL

Schedule =fct(Product Structure + Process)

Development Process

Product

PlanningFieldRealisation

Product Structure

ProductPart A

Part C

Part B

Part D

Schedule

Realisation Part AProduct Planning

Realisation Part B

Realisation Part C

Realisation Part D

System Test



ICEL

Quality Assurance for Software Development

By using metrics and setting objectives:

Without Clear Objectives

Quality Cannot be Improved



ICEL

Two Approaches to Quality Measurement

Quality improvement through measurement

– Presence of qualityHow good is this product? - Subjective metrics

– Absence of qualityWhat is wrong with this product? - Objective metrics



ICEL

Open and Hidden Resistance

Resistance

– Open• Metrics are too expensive

• Metrics are not obvious

• Metrics are not connected to the business

– Hidden• Management

• Developers



ICEL

Open and Hidden Resistance

Overcoming Resistance

– Involve Everyone

– Build Trust

– Metrics Program Champion

– Work Circles



ICEL

Faults as a Cost Factor (to Boehm)

Req

uire

men

ts

Arc

hite

ctur

e D

esi

gn

Det

aile

d D

esi

gn

Cod

ing

Fun

ctio

nal

Tes

t

Integration Test &System Test

Management

Quality Assurance, Config.-Management

18

9

10

6

12

4

10

1

5

10

9

6

Corrections in integration test

and system test amount to 18%

of the costs

Correct. in ...

Correctionsin ...

amount to 6%of the costs

Correctionsin ...

amount to 10%of the costs

Total amount of costs for finding and correcting the defects: 39%



ICEL

Fault StreamDevelopment Process

Requirement Review

Design Review

Code Review

Functional Test

System Test

Deploy-ment

Require-ments Design Implemen-

tation

Time

10% 40% 50%

Stream of Faults

3% 5% 7% 25% 50% 10%

Goal: <5%

Goal: <25%

Fault Finding Process



ICEL

Improvement Potential

Fault OriginFault Origin

Faultdetection

Faultdetection

Require-ments

Design Coding FunctionalTest

SystemTest

Deploy-ment

Costs per

Fault

10%40% 50%

3% 5% 7%25%

50% 10%

1 kDM 1 kDM 1 kDM6 kDM

12 kDM

20 kDM



ICEL

Fault Distribution in Moduls

Number of faults

Number of modules

100%

100%10%

50%

50%

90%

fault concentration in a small number of modules



ICEL

Fault rates as a function of the change rate

0

1

2

3

4

0,125 0,375 0,625 0,875 1

V6

V7.1

V7.5

faults/k DNLCC

DNLCCNLCC

fault driving factors

fault rate



ICEL

Fault rates as a function of the module length

0

5

10

15

20

1,5 2,2 2,8 3,4

V7.1

V7.5

V8

faults/kNLCC

log (NLCC)

fault driving factorsfault rate for new modules



ICEL

Fault rates as a function of the module length

0

5

10

15

1,5 2,2 2,8 3,4

V7.1

V7.5

V8

faults/k DNLCC

log (DNLCC)

fault driving factorsfaults rate for changed modules



ICEL

Particular Factors for High Fault Rates

•Code changes

•Short modules

•„Old“ Programming Languages

•Data flow instead of control flow

•More missing than defect

•Late changes, corrections

•Incomprehensible Messages

•Logic expressions

•Missing initialization



ICEL

Early Fault Removal for a Operating System



ICEL

Fault Rates and Productivity

Size

[kNLCC]

Productivity

[DNLCC/MD]

Fault Rate

[Faults/kDNLCC]

100 35 0.3

200 25 0.4

500 17 0.9

1000 10 1.2

1500 7 1.5

3000 5 1.6



ICEL

Strategies for Reducing Faults

1. Introducing less faults in developing(will be about 5% per year for experienced development teams)

2. Finding faults earlier(Reducing field defects at about 50% in particular years possible)

3. Fault Prevention by re-use



ICEL


• Fault Introduction AnalysisWhere Faults are Introduced within the Process

• Target GoalsFor Faults Found in Development and the Fieldwithin all FunctionsFor all Products

• Monitoring the Target GoalsThrough Reporting and ProjectionComparison with PlanCorrective Action

• Supporting Measures- Found Faults Counts for all Phases- Management Support of Quality Improvement- Improved Review Methods- Methods Training- Tools Use- Many Small Steps



ICEL


Forecasting Methods1. Forecasting rest faults by faults found

2. Forecasting faults based on the defect input in older versions

3. Forecasting faults with a database based on experience

4. Forecasting faults by introducing faults synthetically



ICEL


0

1000

2000

3000

4000

0,25 1,5 2,75 4 5,25

V7.1

V7.5

V8

V8.5

Years0 1 2 3 4 5 6

Detected faults in total



ICEL


faults found in % of all faults

Time

Fault foundF(t)

100%

90- 97%

t(x)

85- 95%

faults to findfaults found

System Test

Pilot

Deployment

Development

60- 75%

F(t) = N(1-e-t)



ICEL


Fault forecasting for product lines

Version n

Version n - 1

Analysis of the fault rates between defined milestones

Programming language

Change rate

Group

Module group

Version n + 1

Forecasting of the faults to expect

New agreement with the development group for number of faults to find

Difference of the fault rates of the version n-1 and n

quality management systems: methods and techniques of software quality management

Documents