Lecture 9: Project Quality Management
Original Slides by Lindsey Brodie
Adapted for CMT3342 by Elke Duncker
Learning outcomes• Understand the importance of quality
management
• Describe the main processes of project quality management
• Describe several traditional quality control techniques
• Understand the contribution of the major quality experts to quality management
• Discuss quality standards and models
What is Quality?
• Day-to-day meanings: – Inherent or distinguishing characteristic,
feature or property of a person or a thing– Level of excellence of something
• Business meanings:– Measure of excellence of a product or a
process– State of being free from defects, deficiencies
and unwanted variations
Aspects and Types of Quality
• Quality of design: – Decide the level of quality required - characteristics of the product or
service such as grade of materials and their performance, Designed aspects of products such as clothes.
• Quality of conformance:– The degree to which the design specifications are met, ‘conformance to
requirements’– Engineering and production– How well does the IT system meet the requirements and standards for this
system?
• Fitness for purpose:– Means that the product can be used for the purpose it was intended.
Considered more rigorous than ‘fitness for use’– How well does the IT system support the task for which it was made?
ISO 8402-1986 Definition
• Quality is "the totality of features and characteristics of a product or service that bears its ability to satisfy stated or implied needs.” http://www.businessdictionary.com/definition/quality.html#ixzz1lcGRIlnb
• The degree to which the customer’s needs and expectations have been satisfied by the product.
Different views of quality
• Quality as:– A product-based property
• Quantifiable and measurable attribute, does not cater for user taste
– A user-based or customer-based view• Satisfaction of user needs, but they vary hugely
– A value-based approach• Consumer: value as quality relative to price
– A transcendent property• Cannot define it, but I will know when I see it (marketing)
– A continuous property• Continuous quality improvement, total quality management –
processes such as health care
WHY BOTHER WITH QUALITY MANAGEMENT?
The cost of quality
• Cost of quality (COQ) includes all the costs associated with quality-related processes
• Cost of quality factors:– Prevention costs (includes quality planning,
technical reviews, test equipment and training)– Appraisal costs (includes inspections and testing)– Failure costs
• Internal failure costs (rework)• External failure costs (helpline support and fixing customer
fault reports)
The cost of quality
COQ = POC + PONC
Cost of quality (COQ) consists of the
Price of conformance (POC): price of ensuring “things are done right the first time”. The sum of prevention and appraisal costs.
AND the
Price of non-conformance (PONC): the failure costs
Crosby (1980)
Cost to fix errors escalates over time
13 - 6x
10x
15 - 40x
30 - 70x
40 - 1000x
82xIBMaverage
Requirements Design Coding DevelopmentTesting
AcceptanceTesting
Operation
Rel
ativ
e co
st t
o fix
an
erro
r
Justification for Quality Management
• Costs of fixing errors and defects can spiral out of control and can ruin an entire organisation
• The later the fix, the higher the costs• Cost for quality management is much less• Quality management is absolutely
essential to any project and to any organisation
PROJECT QUALITY MANAGEMENT
Project quality management
• Quality planning: – Identifying the relevant requirements, quality procedures and
standards, and determining how a project will meet them. Must have both product/system metrics and project process metrics
• Quality assurance:– Carrying out the planned quality activities to ensure the project
delivers a quality product/service
• Quality control:– Monitoring the project results to ensure they meet the relevant
quality standards. Outputs include quality control status reports, rework and process improvements
Traditional Quality planning
• The process of identifying and scheduling Quality Assurance and Quality Control activities to improve the level of quality within a project.
• Make a quality plan including– the relevant customer requirements– the planned project deliverables and outcomes– Quality criteria for these deliverables– Quality standards for the deliverables– Can become part of the customer-vendor agreement
Quality Assurance
• Makes sure standards, policies and other quality related processes are in place and carried out correctly
• Monitors the processes often by way of checkpoints along the way
• This is different to quality control, which checks that the results are what was to be expected
QUALITY CONTROL
Traditional Quality Control Methods
• Management reviews• Testing• Pareto Analysis• Control charts• Walkthroughs• Inspections
Project management reviews
• Progress reviewsRegular meetings reporting progress against plan and
discussing/resolving any issues
• Business reviews– Less frequent meetings to consider if the project is
doing the ‘right thing’ for the organisation– Linked to contractual points: ‘Go / No Go’ decisions
Testing
• Unit testing (component testing)
• Integration testing
• Function testing
• Systems testing
• User acceptance testing
• Field testing (field trialing)
Pareto Principle
• 80/20 Principle• 80% of a result is caused by 20% of the
contributing factors• Examples
– 80% of the inventory value is tied up in 20% of the stock
– Implementing the last 20% of the requirements costs 80% of the time.
Pareto Analysis
• Uses the Pareto Principle to separate the ‘vital few’ from the ‘useful many’
• Statistical technique in decision making• Useful when many possible courses of
action compete for attention• Helps to identify the top 20% of causes to
solve 80% of the problems
An example of a Pareto diagram
0%
80%
60%
40%
20%
100%
Num
ber
of d
efec
ts r
epor
ted
over
a g
iven
tim
e pe
riod
0
40
80
120
160
200 185
142
3020
10
Cum
ulat
ive
Per
cent
age
Insufficient Qualifier Data
Missing Source Information
Missing Benchmark Data
Missing Status Data
Missing Target Level
(Note all percentages are rounded down)
Another Example
Control Charts
• Used to monitor, control and improve process performance over time
• Studies statistical variation and its source• Most commonly used method of Statistical
Process Control (SPC)
Control Chart Population Density Function
An example of a control chart
TimeMean
32
3
2
Upper Control Limit
Lower Control Limit
Violation of control limits
Control charts• Chance causes: tend to lie within the control limits -
only a one in a thousand chance of not doing so
• Special causes (assignable causes or sporadic causes): controlled at the local or operational level. Eliminating these means the process returns to its controlled state. Identified by looking for patterns that suggest non-random behaviour in the control chart. Corrective actions are needed to remove special causes. For example, can detect when manufacturing equipment is becoming defective
• Common causes (endemic causes or chronic causes): inherent in the process, only if the basic process is altered will they change
Walkthroughs
• A walkthrough is a type of peer group review• Informal and lasts 1 to 1.5 hours• Purpose is to enforce standards, detect errors and
improve visibility of the material and overall system quality
• Product author typically describes the structure and logic of the material being reviewed
• A co-ordinator plans and organises the walkthrough• An action list of problems and questions is generated• Outcome is a decision about whether the material can be
accepted as it is or whether it needs revision and even a further walkthrough
Inspections• Carried out on written documentation• Originally developed by Michael Fagan for use
on source code• Now extended by Tom Gilb for use on earlier
system documents (requirements specifications and even contracts)
• Also extended by Robert Mays (IBM) to support continuous process improvement
• Gilb has moved focus from defect finding and fixing towards sampling to determine quality (better to rewrite than fix if a high number of defects)
Inspections
• Inspection must be economic• Can be used for training about standards• Does not replace testing, but can be
argued to find defects earlier
• Checkers are given roles• Use rules and checklists to find ‘issues’ • Check against source and kin documents• Author has final say if ‘issues’ are ‘defects’
QUALITY METRICS
Quality metrics
How can quality be expressed?– Think back to the discussion about success
criteria being picked up as project objectives– Remember Doran’s SMART method– Think back to the discussion about quality
requirements– Many ways of measuring quality
DETOUR: THE QUALITY MOVEMENT
Key contributors to the quality movement
• Walter Shewhart• W. Edwards Deming• Joseph Juran• Philip Crosby• Kaoru Ishikawa
Walter Shewhart
• Encouraged managers to think about problem prevention and process improvement
• Developed:– The control chart (discussed earlier)
– The Plan/Do/Check/Act cycle for process improvement
The Shewhart Cycle or Deming Cycle
Act Plan
DoStudy/Check*Execute Plans
Plan ActionsDecide Actions Needed
Study Results of Actions Taken
* Shewhart used ‘Check’, while Deming preferred ‘Study’
W. Edwards Deming
• Deming popularised the Shewhart cycle
• Came to fame after working in Japan to help the Japanese improve the quality of their manufactured products
• Promoted the concept:– Quality is a management issue: common
causes often beyond the ability of the worker to fix and so require management action
Joseph Juran
• Juran defined quality from a customer’s viewpoint as ‘fitness for use’ (five attributes: quality of design, quality of conformance, availability, safety and field use)
• Juran’s message was that quality must be planned
• The ‘Juran Trilogy’– Quality planning– Quality improvement– Quality control
Philip Crosby
Cost of Quality (COQ) = Price of Conformance (POC) + Price of Non-conformance (PONC)
4 Absolutes of Quality Management:Quality is defined as conformance to requirementsQuality comes from preventionQuality sets the performance standard at ‘zero defects’Quality is measured by the cost of non-conformance
Insisted that ‘Quality is Free’
Kaoru Ishikawa
• Developed:– Quality circles: teams within one or more
organisations meet regularly to discuss how to improve a work process. They devise and try out corrective actions, and report back. Meetings are held until the team decides to disband
– ‘Cause and effect’ diagrams known as Ishikawa diagrams or fishbone diagrams
An Ishikawa diagram or fishbone diagram
Software not delivered on-time
Inexperiencedproject team
New technology
Too many bugs
Too much work Other?
Lack of inspections
Too many last minute changes
Insufficient priorproject experience
Poor training
Too much rework
Unrealistic deadlines
Poor documentation
Lack of knowledge of Evaluation Criteria
New hardwarearrived late
Work rushed
Requirementsspecificationtoo imprecise
Continuous improvement
• Continuously striving to produce better products and improve processes
• Key requirement is that an organisation has stable processes so that the impact of any change can be understood. (Often misinterpreted as that an organisation has to have processes!)
Six Sigma• Holistic approach to quality. An organisation sets
high six sigma goals and uses continuous process improvement
• Six sigma goal is no more than 3.4 defects per million opportunities (think back to control charts)
• Adopted by Motorola and General Electric Company (GEC)
• Builds on the work of the quality gurus such as Deming, Juran and Crosby
• Originally a five step DMAIC improvement process. It has now become eight step
Six Sigma improvement process
• Identify the project• Define the project• Measure current process performance• Analyse/probe the problem• Develop the improved process• Implement the changes• Control - measure and hold the gains• Communicate - exploit the achievement in other
areas
Industry quality standardsISO 9000:2000 Standards for quality management
with respect to improved customer satisfaction and continuous improvement. Focuses on eight principles:– Customer focus– Leadership– Involvement of people– Process approach– System approach to management– Continual improvement– Factual approach to decision making– Mutually beneficial supplier relationships
See http://www.iso.org
Industry quality standardsTickIT• TickIT provides a framework for organisations to
get certification under the ISO 9001:2000 framework
• Develops quality management system certification procedures:– Publishes guidance material for interpreting the
requirements of ISO 9001– Advises on training, selecting and registering auditors
with IT experience and competence– Introduces rules for the accreditation of certification
bodies in the software sector(see http://www.tickit.org)
Maturity models
• Provide frameworks for organisations to assess their overall capability
• Aim to help organisations understand what they need to do to achieve process improvement or enhance organisational capability
Capability Maturity Model Integration (CMMI)
• Developed by the Software Engineering Institute (SEI) at Carnegie Mellon university. See http://www.sei.cmu.edu
• Originally CMM but integrated other models to become CMMI
• Two instantiations:– Staged CMMI: assesses a whole organisation’s
process capability at one of five maturity levels– Continuous CMMI: assesses different process areas
across an organisation individually, so a set of maturity levels results
The CMMI staged diagram:the maturity levels
Level 1Initial
Level 2Managed
Level 4Quantitatively
Managed
Level 5Optimising
Level 3Defined
CMMI • Maturity Level 1: Initial: processes are ad hoc
and chaotic• Maturity Level 2: Managed: processes are
planned, performed, measured and controlled• Maturity Level 3: Defined: Processes are
qualitatively predictable• Maturity Level 4: Quantitatively Managed:
Processes are understood in statistical terms and special causes are addressed. Processes are quantitatively predictable
• Maturity Level 5: Optimising: Focus on improving process performance by removing common causes
Project management maturity models
• Aim to improve and standardise project management processes
• The Project Management Institute (PMI) produced an Organizational Project Management Maturity Model (OPM3). – This builds on the project management processes
described in Lecture 1 (initiating processes, planning processes, executing processes, monitoring and controlling processes, and closing processes)
– Extends into programme and portfolio management
Advantages of CMMI
• It works with all kinds of technologies• Compatible with agile methods and
traditional methods• Good reputation• Cost efficient (saves more than it costs)
Summary
• Discussed:– Current ideas on project quality management mostly
from a traditional PM perspective– Key people who have shaped quality management
and their ideas• Quality:
– Must be designed into a product– Not just about errors and their elimination: it is also
about prevention– Is concerned with the usefulness and acceptability of
a product to its users and the project clients
Next week:
• Agile Quality Management