measuring and estimating software defects - loren stroup

8/4/2019 Measuring and Estimating Software Defects - Loren Stroup

1/20

Measuring and Estimating

Software Defects

Loren Stroup

UCFEEL6883: Software Engineering II


2/20

Outline Introduction To Topic

Current Measures for Estimating Defects

Preventative Methods

Management Decisions

Capability Maturity Model and Defect

Prevention Personal Work Experience

Conclusion


3/20

Introduction Software is in demand

As software complexity has increased sohas the need for software quality

The goal of software reliability is tomake software behave the way it isintended


4/20

Introduction:

Software Defects Over Time

Figure shows number of defects over time

Most defects found during debugging, unit testing, and systemlevel testing

At some point, software defects will continue to rise and anupgrade is needed

IBM studies have shown software defects found after coding are10x more costly to correct than found during testing


5/20

Introduction:

Software Defect Costs

Driven by the amount an organization invests insoftware development

U.S. businesses were investing roughly $250 billion insoftware development annually in 1996

A single post-release defect can cost between$20,000 and $40,000

Improving quality and reducing software defects has

a conservative return of $7.50 to every $1 Software quality budget of $40,000 can return a

savings of $300,000 in preventing most softwaredefects


6/20

Current Measures For

Estimating Defects Defect Detection Rate

Ratio of the number of defects found per the sizeof the work product

The number of defects per page of code or per1000 SLOC

Compared to previous projects and establishedthresholds

A high DDR could mean experienced coders doingthe peer review or latent defects from previousrelease of software

A low DDR could mean inexperienced coders arereview the software or more experienced codersare doing the development


7/20


Estimating Defects (cont)

Defect Removal Efficiency

Percentage of software defects that areremoved by code reviews, inspections, andtests

In use since the 1960s

Current U.S. average is about 85% Companies such as IBM, AT&T, and

Hewlett-Packard have gotten to 99%


8/20



Table shows defect removal efficiency for severalactivities

No one activity has the best efficiency overall

All the activities working together help softwarequality


9/20



Defect Density

Number of defects per size of the softwareproject

Size of project measured in LOC

Useful metric in determining if a software

release is ready for delivery to thecustomer


10/20

Preventative Methods Software Inspections (peer reviews)

Introduced by Michael Fagan in the 1970s at IBM

Involves reviewing a software work product by agroup of trained individuals

Reviewers review the software code changes andrelay comments back to the author for clarification

or further work Initial costs are in training individuals on the

process

Software developers are reviewing instead of

coding


11/20

Preventative Methods Pair Programming

Two software developers work together at

one computer One developer is coding, the other

developer is reviewing

Potential to find defects earlier because the

code is under constant review On average, PP leaves 15% fewer defects

than conventional methods


12/20

Management Decisions Implement fixes prior to delivery

Depends on severity and schedule

Deliver a new software release Hold off the fixes until another software release,

depends on severity Operation and Maintenance work

Deliver a software patch release out-of-cycle releases Release accepted by customer, but patches will be

delivered for any defects found during additionaltesting


13/20

Capability Maturity Model and

Defect Prevention CMM level 2

Key process area is software qualityassurance

Starting to be process-oriented

Main goals are planning quality assurance

guidelines and enforcing them Beginning foundation to defect prevention


14/20


Defect Prevention CMM level 3

Key process area is peer reviews

Focus is on the cause of software defects

Processes begin to be repeatable

CMM level 5

Defect prevention is a key process area Defect prevention is a strategy applied to

the software lifecycle that identifies rootcauses of defects and prevents them from

recurring


15/20


Defect Prevention

Figure is a cost analysis of defect prevention

Clearly, defects found earlier saves money

Team can be set up dedicated to the defect

prevention activities


16/20

Personal Work Experience Software development experience of 2 years Software test experience of 3 years Use peer reviews to minimize software

defects Run unit tests and integration tests with test team

witness

Been a part of 2 major software releases andseveral patch releases Tough to anticipate all behaviors of system

Must pay special attention to code changeswhen dealing with older code


17/20

Conclusion Defects are unavoidable but can be limited

with the right tools

Special metrics make it easier to plan releasesand diagnose problems more quickly

Preventing defects is a key process area inmany of the CMM levels

It is very beneficial in the long run for acompany to invest money in processes toreduce defects for the time of release of thesoftware


18/20

Cited Works Pan, Jiantao. Software Reliability. Website, 1999,

http://www.ece.cmu.edu/~koopman/des_s99/sw_reliability/#concepts. Wikipedia. Software quality. Website, 2008,

http://en.wikipedia.org/wiki/Software_quality#Software_reliability. Subramaniam, Bala. Eective Software Defect Tracking. Website, 1999,

http://www.stsc.hill.af.mil/crosstalk/1999/04 /subramaniam.asp. Lett, Steven H. Using Peer Review Data to Manage Software Defects

Web Document, 2008, http://www.compaid.com/caiinternet/ezine/lett-defects.pdf.

Jones, Capers. "Software Defect Removal Efficiency", in Computer,April 1996, Software Productivity Res. Inc., IEEE.

McConnel, Steve. "Gauging Software Readiness With Defect Tracking",in IEEE Software, May/June 1997, Construx Software, IEEE.

Software Engineering Institute. Software Inspections. Website, 2007,http://www.sei.cmu.edu/str/descriptions/inspections_body.html.


19/20

Cited Works (cont) Wikipedia. Software inspection. Website, 2008,

http://en.wikipedia.org/wiki/Software_inspection. Wikipedia. Pair programming. Website, 2008,

http://en.wikipedia.org/wiki/Pair_programming. Padberg, Frank and Muller Matthias M. "Analyzing the Cost and

Benet of Pair Programming", in Proceedings of the NinthInternational Software Metrics Symposium, 2003, The ComputerSociety, IEEE.

Wikipedia. Capability Maturity Model. Website, 2008,http://en.wikipedia.org/wiki/Capability_Maturity_Model.

Paulk, Mark C., Weber, Charles V., Garcia Suzanne M., Chrissis,Mary Beth, Bush, Marilyn. Software Quality Assurance. Website,2008 http://www2.umassd.edu/swpi/sei/tr25f/tr25_l2e.html.

Narayana, Purushotham. "Software Defect Prevention - In aNutshell", in iSixSigma, 2000-2008, iSixSigma LLC, WebDocument:

http://software.isixsigma.com/library/content/c030611a.asp.


20/20

Questions! If you have any questions, feel free to

contact me

[email protected]

measuring and estimating software defects - loren stroup

Documents