software quality managementpro.unibz.it/staff/mscotto/sqm/sqm1 - introduction.pdf · 2005-05-30 ·...
TRANSCRIPT
2Software Quality Management
Course Outline
Introduction to Software QualitySoftware quality frameworks• Product-oriented approaches• Process-oriented approaches
Management topics in Software QualityTestingMeasurement methodologiesSoftware inspection
3Software Quality Management
Logistics
Schedule• Lecture
Wednesday 8.30-10:30Thursday 8.30-10:30
• LabThursday 10:30-12:30
Copies of the slides will be available on Teleacademy web site (http://www.teleacademy.it)
4Software Quality Management
Bibliography
Martin Wieczorek, Dirk Meyerhoff, B. Baltus, Software Quality: State of the Art in Management, Testing, and Tools, Springer-Verlag, 2001N. E. Fenton, S. H. Pfleeger. Software Metrics: a rigorous and practical approach second edition, International Thomson Computer Press, UK, 1997Ron. S. Kenett, Emanuel R. Baker, Software Process Quality, Marcel Dekker, Inc., 1999
6Software Quality Management
Introduction (1/3)
Software presents both an opportunity and a threatIn many applications, software is a critical componentExamples:• Elevators• Airlines• Telecommunications• Medical devices
7Software Quality Management
Introduction (2/3)
From a survey of the Software Engineering Institute (SEI):• 60% of software development organizations
are at the lowest level of capability• The assessment is based on the Capability
Maturity Model (CMM) A framework for achieving process improvement
• The survey involved over 600 organizations worldwide
8Software Quality Management
Introduction (3/3)
The situation is even worse The survey involved organizations that have started improvement efforts or made a commitment to do soIf we add in all the organizations that: • have not assessments performed or • have no plans to implement process improvement
the number of organizations at the lowest level is probably 80%
9Software Quality Management
Software Quality (1/3)
What is the consequence of this low level of process capability in most software companies?• The quality of software!
How many users are really happy with the quality of their software products?• Stories about people losing hours of work
because the computer locked up at an inopportune time are very common
10Software Quality Management
Software Quality (2/3)
Definition of quality• “Degree of excellence, relative nature or kind
of character, class or grade of thing as determined by this, general excellence" (The New Oxford Illustrated Dictionary)
11Software Quality Management
Software Quality (3/3)
Software definitions: • Intellectual product consisting of information stored
on a storage device (ISO/DIS 9000: 2000)• Software is "intellectual creation comprising the
programs, procedures, rules and any associated documentation pertaining to the operation of a data processing system"
• A software product is the "complete set of computer programs, procedures and associated documentation and data designated for delivery to a user" (ISO 9000-3)
12Software Quality Management
Why does software have errors?
When executing complex tasks, human beings do commit errors • This cannot be avoidedExperienced programmers commit 1 error in every 10 lines of code (average) About 50% of these errors are corrected during code compilationFurther errors are corrected in testingAbout 15% of the errors are still in the system when delivering it to the customer
13Software Quality Management
Errors and faults in software
And what if the software was (life) critical? How many errors and faults is it acceptable it has? How can we handle errors and faults? What does software quality really mean?
16Software Quality Management
Different focuses over time (1/4)
In the “heroic age of computer science”:
• A program was "good" if it was able to run - at least once-, to terminate running after a time - the length of running-time was not really important -, and to produce results similar to those expected
Around 1960-1975: “micro-efficiency”• A program was "good" if its use was cheap in a certain hardware-
software environment, and it handled the given resources of the computer in an optimal way
Beginning with the ’80’s: ''macro-efficiency''• A "good" computer program is portable, its use can be learned by
common people, and it is highly user-friendly
17Software Quality Management
Different focuses over time (2/4)
Nowadays• Correctness, reliability, integrity, interoperability etc.
of the whole system is important
• We usually think of a good system if its documentation is complete and understandableif it interacts with other - computer aided or classic - data-processing systems in an optimal wayif its maintenance can be done at low costsif it is able to cooperate with new software-hardware tools and so on
• Complex organizational frame, well-organized team-work are needed
18Software Quality Management
Different focuses over time (3/4)
• Nowadays:
Structured systems' analyzing and developing methods and methodologies have been worked out and complex computer-aided tools have been developed
The use of embedded intelligence
Total Quality Management"to create a system of management procedures that focus on customer satisfaction"
19Software Quality Management
Different focuses over time (4/4)
Nowadays:• Software
“drives” airplanes and cars“moves” our money“executes” surgical interventions… interferes with all aspects of our life!
• Bad software/ software with errors: Can kill peopleCan cause financial crashCan lead to worldwide disasters
20Software Quality Management
Importance of Software Quality (1/5)
Several historic disasters attributed to software• 1988 shooting down of Airbus 320 by the USS
VincennesCryptic and misleading output displayed by tracking software
• 1991 patriot missile failure Inaccurate calculation of time due to computer arithmetic errors
• London Ambulance Service Computer Aided Dispatch System
Several deaths• On June 3, 1980, the North American Aerospace
Defense Command (NORAD) reported that the U.S. was under missile attack
21Software Quality Management
Importance of Software Quality (2/5)
Several historic disasters attributed to software• First operational launch attempt of the space
shuttle, whose real-time operating software consists of about 500,000 lines of code, failed
Synchronization problem among its flight-control computers
• 9 hour breakdown of AT&T's long-distance telephone network
Caused by an untested code patch
22Software Quality Management
Importance of Software Quality (3/5)
Ariane 5 crash June 4, 1996• Maiden flight of the European Ariane 5 launcher
crashed about 40 seconds after takeoff• Lost was about half a billion dollars• Explosion was the result of a software error• Uncaught exception due to floating-point error:
conversion from a 64-bit integer to a 16-bit signed integer applied to a larger than expected number
• Module was re-used without proper testing from Ariane 4
Error was not supposed to happen with Ariane 4No exception handler
23Software Quality Management
Importance of Software Quality (4/5)
Mars Climate Orbiter - September 23, 1999• Mars Climate Orbiter, disappeared as it began to orbit
Mars• Cost about $US 125-million• Failure due to error in a transfer of information
between a team in Colorado and a team in CaliforniaOne team used English units (e.g., inches, feet and pounds) while the other used metric units for a key spacecraft operation
24Software Quality Management
Importance of Software Quality (5/5)
Internet viruses and worms• Blaster worm ($US 525 millions)• Sobig.F ($US 500 millions – 1billions)
Exploit well known software vulnerabilities• Software developers do not devote enough effort to
applying lessons learned about the causes of vulnerabilities
• Same types of vulnerabilities continue to be seen in newer versions of products that were in earlier versions.
Usability problems
25Software Quality Management
Garvin’s quality definitions (1984) (1/5)
Transcendent definition
• Quality is absolute and universally recognizable, despite the fact that it cannot be defined precisely
• Only experience can teach to recognize quality
26Software Quality Management
Garvin’s quality definitions (1984) (2/5)
User-based definition
• Quality is "fitness for use”
• This definition starts from the assumption that the individual customers have different needs and those goods that best satisfy their needs are considered to have the highest quality
27Software Quality Management
Garvin’s quality definitions (1984) (3/5)
Value-based definition
• Defines quality in relation to costs
• A quality product provides performance at an acceptable price or conformance at an acceptable cost
28Software Quality Management
Garvin’s quality definitions (1984) (4/5)
The product-based definition • Regards quality as a precise and measurable
variable, as an inherent characteristic of goods
29Software Quality Management
Garvin’s quality definitions (1984) (5/5)
Manufacturing-based definition • Identifies quality as conformity with
specifications• It focuses on the supply side
30Software Quality Management
The elements of software quality (1/5)
• Product• Process• Resource
• Definition• Quality attribute• Metric
31Software Quality Management
The elements of software quality (2/5)
Product • Use cases• Design artifacts• Code• ..
Process• Duration• Costs• …
Process• Duration• Costs• …
32Software Quality Management
The elements of software quality (3/5)
Software quality attributes are useful to more precisely define customer’s quality requirements Examples• Reliability• Performance• Security• Functionality• Usability• Maintainability• Portability• Reusability• Integrability• Testability
33Software Quality Management
The elements of software quality (4/5)
Quality attributes• Attributes may have different measures
describing them from different perspective • Example
Complexity is an internal attribute of products
• There are many complexity measures: to trace the complexity in structure we may consider the cyclomatic complexity of McCabe
• CC cannot track the algorithmic complexity
34Software Quality Management
The elements of software quality (5/5)
Quality Attributes• It easier that a a company collects measure of
length and complexity of the code (internal attribute of product) than measure of quality (external attribute) of the product
• Internal attribute: internal characteristics of product, process, and human resources
• External attributes: due to external environment
35Software Quality Management
Measurement
Definition• The process through which we attach
numbers or symbols to the attributes of the real world’s elements, with the scope to describe them according to some, well defined rule (Fenton)
36Software Quality Management
Type of measures
Direct: from a direct process of measuringIndirect: from a mathematical equation in the world of symbolsObjective: the same each time they are taken (e.g. automated collected by some device) Subjective: manually collected by direct human evaluationExample: temperature of coffee - taste of a coffee
37Software Quality Management
The quality framework
Characteristics
Process
Product
Resource
Definition
Qualityattribute
Metric
Objects
38Software Quality Management
Different approaches towards software quality
ISO 9126(Boehm, McCall)...
Process
Product
Resource
Definition
Qualityattribute
Metric
Objects
CharacteristicsGQM
ISO 9001:2000CMM, SPICE
CMMIISO 15504TSP, PSP
Methodologies (RUP, Prince...), Life-cycle support
P-CMM, Weinberg...
39Software Quality Management
What does software quality definition depend on?
…the personality / value-perception / point of view of the evaluator …the type of software production …the development life cycle
…the type of the system / the application domain of the software …business policy
40Software Quality Management
Quality framework, objects and perspectives
objectperspectives
Product Process Resource
user user satisfaction based
participation based
interaction based
engineer system based design based application based
project manager project based control based allocation based
41Software Quality Management
Quality – as depending on the life cycle
Prod
uct r
evisi
on Product transition
Product operation
maintainabilityflexibilitytestability
portabilityreusabilityinteroperability
correctnessreliabilityusability
efficiencyintegrity
(McCall)
42Software Quality Management
Quality as depending on the evaluator (1/4)
User• Software does what I wish
it to do • Quick response time • Ease of use • Should be ready as soon as
possible• Able to follow business
changes• Able to handle erroneous
data• …
Developer• Well specified• Technically correct • Easy to maintain• Easy to develop • Stable• Receives adequate data• ...
43Software Quality Management
Quality as depending on the evaluator (2/4)
User is interested in:• Using the software, its performance, effects of
using the software
User not interested in the internal aspects of software or how it was developedFunctionality, reliability, efficiency, usability, portability
44Software Quality Management
Quality as depending on the evaluator (3/4)
Developer is interested in:• Intermediate product quality and final product
quality• Quality characteristics and metrics might
differ according to the phases of the life cycle• Maintainability• Final product quality characteristics have to
be agreed with the user
45Software Quality Management
Quality as depending on the evaluator (4/4)
Manager is interested in:• Overall quality• Not a specific quality characteristic• Quality improvement with management
criteria (schedule delay, cost overrun)• Will need to assign weights, reflecting
business requirements, to the individual characteristics
46Software Quality Management
Quality according to software application domain
Spaceshuttle orairplaneSatellite
Detectionand alarm
ManagementsystemTest bench
corr
ectn
ess
relia
bilit
y
effic
ienc
y
inte
grity
usab
ility
mai
ntai
nabi
y
flexi
bilit
y
test
abili
ty
porta
bilit
y
reus
abili
ty
inte
rope
rabi
.
47Software Quality Management
Software quality and business policy
Rapid developmentHonoring schedule constraints100% conformity to the specificationComplete coverage during testing...
48Software Quality Management
Total Quality Management (TQM) (1/2)
It represents a style of management aimed at achieving long-term success by linking quality and customer satisfaction
The approach include the creation of a culture in which all members of the organization participate in the improvement of processes, products, and services
49Software Quality Management
Total Quality Management (TQM) (2/2)
Despite variations in its implementation, the key elements of a TQM system can be summarized as follows:• Customer Focus• Process• Human side of quality• Measurement and analysis
50Software Quality Management
TQM Key Elements (1/2)Customer focus• The objective is to achieve total customer satisfaction• Customer focus includes studying customers’ wants
and needs, gathering customer’ requirements, and measuring and managing customers’ satisfaction
Process• The objective is to reduce process variations and to
achieve continuous process improvement• This element includes both the business process and
the product development process• Through process improvement, product quality will be
enhanced
51Software Quality Management
TQM Key Elements (2/2)
Human side of quality• The objective is to create a companywide quality
culture• Focus areas include leadership, management
commitment, total participation, employee empowerment, and other social, psychological, and human factors
Measurement and analysis• The objective is to drive continuous improvement in
all quality parameters by the goal-oriented measurement system
52Software Quality Management
Total Quality Management
Metrics, Models, Measurement, and Analysis
Customer Focus
Process Improvement
Human Side of Quality
Total Quality ManagementContinuous Improvement
53Software Quality Management
Total Quality Management
Various organizational frameworks have been proposed to improve quality Specific examples include:• Plan-Do-Check-Act (PDCA)• Quality Improvement Paradigm/Experience
Factory Organization • Capability Maturity Model (SEI-CMM)• Lean Enterprise Management
54Software Quality Management
Plan-Do-Check-Act (PDCA)
PDCA is based on a feedback cycle for optimizing a single process or product lineCreated by W. Edwards Deming in the 50’s It uses techniques such as feedback loops and statistical quality control • to experiment with methods for improvement• to build predictive models of the product
55Software Quality Management
PDCA BasicsPlan• Plan your improvement, also setting clear numerical
goalsDo• Do your improvement plan, recording the different
stepsCheck• Check what you have done and how closely you have
reached your goalsAct• Act on the basis of the result to get ready to the next
circle of improvement
57Software Quality Management
Quality Improvement Paradigm/Experience Factory Organization (1/2)
It aims at building a continually improving organization based on its evolving goals and assessment of its status relative to those goalsIt uses such techniques as Goal-Question-Metric (GQM)
58Software Quality Management
Quality Improvement Paradigm/Experience Factory Organization (2/2)
The six fundamentals steps are:1. Characterize the project and its environment2. Set the goals3. Choose the appropriate processes4. Execute the processes 5. Analyze the data 6. Package the experience for reuse
59Software Quality Management
Capability Maturity Model (CMM)
The SEI CMM is a staged process improvement, based on assessment of key process areas until you reach level 5, which represents a continuous process improvement
60Software Quality Management
Lean Enterprise Management (1/2)
It is based on the principle of concentration of production on “value-added” activities and the elimination or reduction of “not-value-added”activitiesThe approach has been used to improve factory outputThe goal is to build software with the minimum necessary set of activities and then to tailor the process to the product’s requirements
61Software Quality Management
Lean Enterprise Management (2/2)
The approach uses such concept as:• Technology management• Human-centered management• Decentralized organization• Quality management• Supplier and customer integrationThe implementation of Lean Management in Software Engineering is represented by Agile Methodologies
62Software Quality Management
Summary
Software quality is a complex issueThe need to be familiar with basic concepts and definitions There is no “always good” solution There is no solution “good for everyone”There are many possible approaches • It is not easy to know them all • It is not easy to choose the right one