galorath - it data collection, analysis and benchmarking: from processes and benchmarks into real...
TRANSCRIPT
![Page 1: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/1.jpg)
IT Data Collection, Analysis and Benchmarking: From Processes and
Benchmarks into Real Business Value
Dan Galorath
Copyright 2013 Galorath Incorporated
![Page 2: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/2.jpg)
Key Points
Measurement needs to think like business
leaders… provide
information understandable to management
and other stakeholders
Measurement needs to produce value not cost in the eyes of management and other stakeholders
Measurement results should include confidence and risk so leaders can understand and include in decisions
![Page 3: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/3.jpg)
Context
© 2011 Copyright Galorath Incorporated
3
![Page 4: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/4.jpg)
•Cutter Consortium Software Project Survey:• 62% overran original schedule by more than 50%
• 64% more than 50% over budget
• 70% had critical product quality defects after release
• Standish Group CHAOS Report• 46% challenged
• 19% failed
• 35% successful
~$875 billion spent on IT~$300 billion spent on IT projects ~$57 billion wasted annually
ROI of better planning huge
IT Failures Are Pervasive: And Even Successful Projects May Not Be
Some suggest 80% of systems NEVER produce a positive ROI
![Page 5: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/5.jpg)
Human Nature: Humans Are Optimists
HBR Article explains this Phenomenon:
• Humans seem hardwired to be optimists
• We routinely exaggerate benefits and discount costs
Delusions of Success: How Optimism Undermines Executives' Decisions (Source: HBR Articles | Dan Lovallo, Daniel Kahneman | Jul 01, 2003)
5
Solution - Temper with “outside view”:Past Measurement Results, traditional forecasting,
and statistical parametrics can help
Don’t remove optimism, but balance optimism and realism
![Page 6: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/6.jpg)
Example of Tempering: German Mark V Tanks (Source: How To Measure Anything)
• Allies estimated production by analyzing serial numbers
• Used the captured tanks as a random sample and predicted probability of various production levels
![Page 7: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/7.jpg)
Some of Dan’s Heroes
• Frederick Taylor: The Principals of Scientific Management 1901 “Let data and facts do the talking”
• W. Edwards Demming: “In God We Trust… All Others Bring Data”
• Frederick Brooks: “There is an incremental person when added to a software project that makes it take longer”
• Ed Yourdon: “Avoiding Death Marches in Software Projects”
• Steven Covey: “Sharpen the Saw” Focus on improvement
![Page 8: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/8.jpg)
Measurement Manifesto For Software & IT
Measurement Perspective
• We measure to ultimately produce business value to the organization
• MEASURED Measurement should not ultimately be a cost
• The analysis of measurements produces decisions that produce business value
Management & Stakeholder Perspective
• Speak so I can understand
• Give me actionable items
• Don’t just give me problems… Give me solutions
• Help me make the best decisions so we can produce business value
© 2011 Copyright Galorath Incorporated 8
Measurement itself is N ENABLER….
Management decision making, better performance, quality improvements and better serving stakeholders is
![Page 9: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/9.jpg)
Example: Project Cost Alone Is not The Cost of IT Failure (Source: HBR)
• Case Study: Levi Strauss
• $5m ERP deployment contracted
• Risks seemed small
• Difficulty interfacing with customers systems
• Had to shut down production
• Unable to fill orders for 3 weeks
• $192.5M charge against earnings on a $5m IT project failure
© 2011 Copyright Galorath Incorporated 9
“IT projects touch so many aspects of organization they pose a new singular risk”
http://hbr.org/2011/09/why-your-it-project-may-be-riskier-than-you-think/ar/1
![Page 10: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/10.jpg)
Exuding Confidence or Concern With Measurement
© 2009 Copyright Galorath Incorporated 10
![Page 11: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/11.jpg)
Frederick Brooks Classic Paper “No Silver Bullets”• “There is no single development, in either
technology or management technique, which by itself promises even one order-of magnitude improvement within a decade in productivity, in reliability, in simplicity.”
• -- Fred Brooks, 1986
• i.e. There is no magical cure for the “software crisis”
• Not software measurement
• Not better tools
• Not ---- (fill in the blank… the current great hope)
![Page 12: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/12.jpg)
Size & Productivity
© 2011 Copyright Galorath Incorporated
12
![Page 13: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/13.jpg)
Fundamental Metrics For Estimation, Planning & Control
• Size
• AKA Volume, Mass
• Units: Source Lines of Code (SLOC); Function Points (FP) Use Cases
• New versus rework
• COTS & Packages
• Technology
• AKA Productivity Potential, Efficiency
• Units: none
• Time
• AKA Duration, Schedule
• Units: Calendar Months, Calendar Weeks
• Effort
• AKA Work, Labor
• Units: Staff Months, Staff Hours
• Cost
• AKA Budget, Money
• Units: $, other currencies
• Staffing
• AKA Manpower Loading
• Units: FTE People
• Defects
• AKA Reliability, Quality
• Units: Defect Count
![Page 14: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/14.jpg)
Many Size Measures Are Viable
• We know about:
• Lines of Code &
• COSMIC Both provide a method for Sizing Software
• Story Points (different since their values are team defined) provide the same capability for User Stories or Features or anything that can sized
• Many others
• Function Points
© 2012 Copyright Galorath Incorporated 14
![Page 15: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/15.jpg)
A1-15
Size Is More Than Just Total Application
NewNew
Effective Effort Units
Lines, Functions, Objects, Use Cases, Etc.Lines, Functions, Objects, Use Cases, Etc.
Pre-existingPre-existing
![Page 16: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/16.jpg)
Reuse: Watch Out For Low Cost Assumptions on “Heritage”
• Reuse or Heritage: applying existing software to a new mission (or additional innovation in its current mission)
• Effort to reuse software is routinely under estimated
Design
Test
ImplementationWhy should we care: Heritage is often underestimated and causes major schedule / cost overruns
![Page 17: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/17.jpg)
Sample Effort Factors for Reuse
© 2013 Copyright Galorath Incorporated 17
![Page 18: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/18.jpg)
Software Design For Reuse (Lower Cost Heritage)
• Designed for reuse• Reusability requirements during original development
• Significant extra coding and documentation effort during original development to insure reusability
• Minor to moderate work (mostly retest) required when reused IF NOT MODIFIED
• Not designed for reuse• Developed without extra effort to ensure reusability
• Sources include any legacy code not specifically designed for reuse
• Other projects & programs
• Prior builds of the current program
• Moderate to major rework required• Redesign
• Reimplementation
• Retest
![Page 19: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/19.jpg)
Rework Causes Software Project Issues
• Rework: Doing the same work over again because it was incorrect the first time
• Not prototyping
• Not refactoring (tuning to make software better)
• Between 40% and 50% of the total effort on software projects is spent on rework.. Barry Boehm
• Initiatives to reduce rework can save significantly
Why should we care: Unplanned rework can drive cost up dramatically AND reducing rework can be a boon to projects
![Page 20: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/20.jpg)
Customization
COTS or GOTS Hoped For Reuse
• Developmental Software:
• Functionality developed specifically for the project at hand
• May include customization of COTS
• “Glue” Code:• Code written to bind
COTS to developmental software
• Development effort must be captured
• COTS Software:• Purchased functionality
• Direct Cost component of COTS integration
• COTS Cognition: • Required functionality within the COTS
software that must be understood
• Effort component of COTS integration
DevelopmentalSoftware
COTS Software
“COTS Cognition”
Glue
Why should we care: COTS promises are often not accurate meaning additional development cost & Schedule
![Page 21: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/21.jpg)
Packaged Applications Are Costly To Organizations
• “Commercial application program or collection of programs developed to meet the needs of a variety of users, rather than custom designed for a specific organization”
• Many are enterprise applications
• Often allows / requires customization
• Examples: SAP; Rational PPM, SEER for Software; Microsoft Excel, CA Clarity, Oracle Business Suite
Why should we care: Packages sometimes comprise solutions for parts of complicated systems and can be trouble
"One-third [of the budget] has to go to testing. Don’t ever short change testing. Everyone always underestimates it, and says it’s the last thing to worry about. Don’t do that!“
- Jim Larson, consultant for communications solutions provider
![Page 22: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/22.jpg)
Software Productivity Laws
• LAW 1 – Smaller teams are more efficient: The smaller the team the higher the productivity of each individual person.
• LAW 2 – SOME schedule compression can be bought: Adding people to a project, to a point, decreases the time and increases the cost.
• LAW 3 – Every project has a minimum time: There is an incremental person that consumes more energy than he/she produces. Team size beyond this point decreases productivity and increases time. (aka Brooks’ Law--”Adding staff to a late software project makes it later.”)
• LAW 4 – Productivity is scalable: Projects of larger software size can use larger teams without violating LAW 3.
• LAW 5 – Complexity limits staffing: As complexity increases, the number of people that can effectively work on the project and the rate at which they can be added decreases.
![Page 23: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/23.jpg)
Software Productivity Laws (continued)• LAW 6 – Staffing can be optimized: There exists an optimal
staffing function (shape) that is effectively modeled by the Rayleigh function. Flat (level load) staffing is rarely optimal.
• LAW 7 – Projects that get behind, stay behind: It is extremely difficult to bring a project that is behind schedule back on plan.
• LAW 8 – Work expands to fill the available volume: It is possible to allow too much time to complete a project (aka Parkinson’s Law).
• LAW 9 – Better technology yields higher productivity: More capable teams, better tools, and advanced, stable processes yield higher productivity.
• LAW 10 – No “silver bullets”: There is no methodology, tool, or process improvement strategy out there that yields revolutionary improvements in project efficiency.
![Page 24: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/24.jpg)
The Bottom Line
• Effort for reuse and rework must be considered
• Standard definitions of productivity e.g.
• 18 hours per function point…
• Is necessary but not sufficient for detailed estimates that must be viable
• Use a range
• Identify key cost drivers that impact productivity
• Ensure you know what is IN and NOT IN the hours
© 2013 Copyright Galorath Incorporated 24
![Page 25: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/25.jpg)
Relative Sizing Can Provide Quick ROM Numbers E.G. ISBSG
• War and Peace is 587,287 words
• How long will it take you to read it?
• Using a different reference try…
• Girl with the Dragoon Tattoo =144,000
• The Bourne Identity = 80,000
• Harry Potter and the Philosopher's Stone = 76,944
© 2012 Copyright Galorath Incorporated 25
![Page 26: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/26.jpg)
Data Collection, Normalization, Usage, Caution
© 2013 Copyright Galorath Incorporated 26
![Page 27: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/27.jpg)
Before You Start - Some Basic Rules
1. Know from where the data comes (total, effective, new, etc?)
2. Know what is included (Phases, People, Processes, …)
3. Know the different sizing schemas and how applied
4. Know what was intentionally left out (and why)
5. Consider the hindsight – if things happen they should be accounted for in the data collection
6. Data needs to be consistent or normalized
![Page 28: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/28.jpg)
Galorath’s Data Review Process
Management is excluded from actual
Concept is excluded
from actual
CM / QA is excluded
from actual
…management from the estimate
…concept from the estimate
…CM / QA from the estimate
THE ACTUAL You should exclude...
Productivity Reasonable ?
Is Effort Accounting By Phase Probably
Complete?
Summary EffortMatch Accounting
By Phase?No No No
Adjust For Missing Phases
Is There Effort Accounting By
Phase?
Yes
YesYesYes
No
Do Not Adjust
Do Not Adjust
Do Not Adjust
Is“Project Activity
Scope” fieldComplete?
No
Do Not AdjustYes
Productivity Reasonable ?
Is Effort Accounting By Phase Probably
Complete?
Summary EffortMatch Accounting
By Phase?No No No
Adjust For Missing Phases
Is There Effort Accounting By
Phase?
Yes
YesYesYes
No
Do Not Adjust
Do Not Adjust
Do Not Adjust
Is“Project Activity
Scope” fieldComplete?
No
Do Not AdjustYes
Normalization
Mapping
![Page 29: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/29.jpg)
29
Data Must Be Used With Caution
• Run sanity checks on data
• 10,000 Function Points can’t be developed in 3 months
• Ongoing issue between statisticians and engineers
• Some Statisticians claim.. “That is what the data says so it must be right”
• Sometimes even if it is obviously wrong
![Page 30: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/30.jpg)
Data Doesn’t Have To Be Perfect To Be Useful: But Is Has To Be Viable
• 80 Calories per serving
• 2.5 Servings per can
• 4 Ounces, Condensed, 8 Ounces With Water
![Page 31: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/31.jpg)
31
Data Improves Estimates For New Programs Source: John Vu, Boeing SEPG 1997
.
0 %
140%
-140%
..
..
.
..
..
.
..
.
. .
. . . .
.
. . .
. .
.
.
. . . .. . . . . . . ...
. . . .. .
.
. .
. ..
.
.
.
. .. .. ...... . .. . ... . ...
.. .
.
Without Historical Data With Historical Data
Variance between + 20% to - 145% Variance between - 20% to + 20%
(Efforts = Labor Hours)
(Mostly Level 1 & 2) (Level 3)
Ove
r/Und
er P
erce
ntag
e
.
(Based on 120 projects in
Boeing Information Systems)
.
. . .
.
.
..
..
.
. .
..
.
.
. .
.
.
.. .
. . .. . . . . .. . . . . .. .
..
. . .. . .. . . .
. . . .. . . . .
. . . . .
. . . . .. . . . . .
. . . . .. . . . . .
. . . . . .. . . . . . . .
. . .. . . . .
. . . . .. . . .
. . . . . .
. . . . . .
. . . . . .
John Vu, Boeing, keynote talk at SEPG ‘97,
“Software Process Improvement Journey (From Level 1 to Level 5)”
![Page 32: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/32.jpg)
Just the Top Level Data Isn’t Sufficient For Estimation
Hours Versus UFPs
1
10
100
1000
10000
100000
1000000
1 10 100 1000 10000 100000UFPs
Hou
rs
![Page 33: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/33.jpg)
Use Historical Data to evaluate your estimate!
![Page 34: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/34.jpg)
The Error of Causal Analysis Is often encountered in data analysis!
NORMALIZE:There is no correlation that estimate of size ‘X’ is an
equivalent of historical projects of about ‘X’.
Code and Unit Test Only
System and SW Requirements
were done separately
This point does not include SW Testing Effort
![Page 35: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/35.jpg)
Fallacy of Silent EvidenceWhat about what we don’t know?
How confident would you feel if the Silent Evidence was visible?
![Page 36: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/36.jpg)
36
Software Has Additional Characteristics That Should Be Considered
Track defect discovery and removal rates
against expected rates
Increased defect reporting rate
shows a worsening trend
Heath and Status Indicator shows status and trends from
the previous snapshot
Thresholds are user definable
![Page 37: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/37.jpg)
Watch Total System Requirements: IT Systems Total Ownership Costs; 60+% Can Be Infrastructure & Services
• Software Development
• Software Maintenance
• IT Infrastructure
• IT Services
• Other
Measurement Results must provide real analysis
![Page 38: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/38.jpg)
Risk and Uncertainty
© 2013 Copyright Galorath Incorporated 38
![Page 39: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/39.jpg)
39
Statistician Drowns in River with Average Depth of 3 Feet!
![Page 40: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/40.jpg)
Software Is A Critical Component of Military & Aerospace Systems Failures Ariane 5
• Ariane 5 video
• Note: $500 Million payload
• Failure due to reused software from (Ariane 4)
• with incompatible assumptions for exception condition that was not required
ariane_5_explosion_video.mp4
“the culture within the Ariane program…” only addressed “random hardware failures… which can quite rationally be handled by a backup system”“the view had been taken that software should be considered correct until it is shown to be at fault”!
Why should we care: Software cost & schedule should be sufficient for successful missions
![Page 41: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/41.jpg)
Minimum
Most Likely:
Maximum
If everything goes well (optimistic
estimate)
If everything goes badly
(pessimistic estimate)
Using Ranges To Address Uncertainty
pert beta Incremental Likelihood
Pert Beta
![Page 42: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/42.jpg)
Adding Reality to Schedules Example (Source: SEI)
Process DurationsStep Expected1 302 503 804 505 906 257 358 459 7010 25
500
What would you forecast the schedule duration to be?
![Page 43: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/43.jpg)
Adding Reality to Schedules – Example - 2
Process DurationsStep Best Expected Worst1 27 30 752 45 50 1253 72 80 2004 45 50 1255 81 90 2256 23 25 637 32 35 888 41 45 1139 63 70 17510 23 25 63
500
What would you forecast the schedule duration to be now?
![Page 44: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/44.jpg)
Adding Reality to Schedules – Example - 3
With 90% confidence, the project will be under 817 days duration.
The project is almost guaranteed to miss the 500 days duration 100% of the time.
![Page 45: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/45.jpg)
Adding Reality to Schedules – Example - 4
With only 50% confidence, the project will be under 731 days duration.
![Page 46: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/46.jpg)
Typical Growth Range From Initial Sizing To Delivery
• Functional sizing for estimates should include growth range to mitigate risk
© 2011 Copyright Galorath Incorporated 46
![Page 47: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/47.jpg)
Dealing With the “Problem of Assumptions”• Assumptions are essential but…
• Incorrect assumptions can drive an estimate to uselessness
• Use an assumption verification process
© 2012 Copyright Galorath Incorporated 47
Estimates must have assumptions defined but…Bad assumptions should not be justification for
bad estimatesASSUMPTIONS MAY BE RISKS
![Page 48: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/48.jpg)
Business Value Is Part of Measurement & Estimation
© 2011 Copyright Galorath Incorporated
48
![Page 49: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/49.jpg)
Software Projects Must Return Business Value“Economics is primarily a science of choice…
software economics should provide methods for analyzing the choices software projects must make.” Leon Levy
“Base choices on those providing the maximum business value to the organization” Eli Goldratt
Measurement and its uses such as estimating and defect analysis help this science of choice
• Some say business value is not our problem
• While others generally need to perform benefit analysis
• We need to build systems that optimize the business
• Make IT part of the solutionIf IT & measurement don’t generate sufficient profit money will go elsewhere
![Page 50: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/50.jpg)
The Gap Between IT & Management Needs (Source: Fraunhofer)
© 2011 Copyright Galorath Incorporated 50
Financial
Customer Business Process
Innovation / Growth
Vision & Strategy
Goals
Data Collection
MeasurementMetricsAnswersQuestions
Goal Attainment
Software Development & Maintenance
6σ
GQM
PSM
CoBIT…
Business/OrganizationLevel
ITGovernance
ITGovernance
CustomerFocus
![Page 51: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/51.jpg)
Software & IT Systems Are About Business Value
© 2009 Copyright Galorath Incorporated 51
![Page 52: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/52.jpg)
Generating the Business Value Side of the Equation (Benefits)• The business owns benefit calculations
• IT should participate
• Exception: projects solely improving internal IT
• Beware of subjectivity translating soft benefits
• Use probability and risk
© 20011 Copyright Galorath Incorporated 52
![Page 53: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/53.jpg)
Quantify Intangible Software Benefits & Costs When Possible … But Be Realistic; Examples
• Improved Customer Satisfaction: May yield Improved Customer Retention
• Higher Quality Product: May yield lower rework and higher customer satisfaction
• Improved Decision Making: May yield many things
• Improved Win Ratio: Yields additional revenue
• Time to Market: May improve profit/market share
© 2009 Copyright Galorath Incorporated 53
![Page 54: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/54.jpg)
An ROI Analysis of A New System
© 2013 Copyright Galorath Incorporated
54
Cost of capital 8.0%
Initial Investment Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7Total
Ownership
Investment $100,000 $100,000
Increase/(dec.) in revenue ($40,000) $60,000 $110,000 $100,000 $100,000 $150,000 $150,000 $630,000
Increase/(dec.) in op. exp. $90,000 $70,000 $70,000 $22,000 $24,000 $27,000 $28,000 $331,000 Cash Flow ($100,000) ($130,000) ($10,000) $40,000 $78,000 $76,000 $123,000 $122,000 $199,000
PV of Cash Flow ($100,000) ($120,370) ($8,573) $31,753 $57,332 $51,724 $77,511 $71,186 $60,563
NPV 60,563 $60,563
IRR 13.5% 13.5%
ROI 121% 121.1%
![Page 55: Galorath - IT Data Collection, Analysis and Benchmarking: From Processes and Benchmarks Into Real Business Value](https://reader038.vdocuments.site/reader038/viewer/2022110203/55d0c9e4bb61eb7a358b4787/html5/thumbnails/55.jpg)
The Rule of Measurement
When performance is measured Performance improves…
When performance is measured and reported the rate of improvement accelerates..
Thomas Monson
The work you do is a root cause of program success