business value of agile methods: using return on investment

Business Value of

Agile MethodsUsing Return on Investment

Dr. David F. Rico, PMP, CSM

2

AgendaIntroductionSources of Business ValueSurveys of Business ValueMeasures of Business ValueModels of Business ValueEstimation of Business ValueComparison of Business ValueSummary of Business Value

3

AuthorDoD contractor with 25+ years of IT experienceB.S. Comp. Sci., M.S. Soft. Eng., D.M. Info. Tech.Large NASA & DoD programs (U.S., Japan, Europe)

* Published five textbooks and over 15 articles on various topics in return on investment, information technology, agile methods, etc.

4

PurposeProvide an overview of the business value of Agile Methods using return on investment:

Business value is an approach for estimating the tangible and intangible worth of organizational assetsBusiness value is an appraisal of intellectual assets such as knowledge, experience, and skillsBusiness value is a technique for determining the complete worth of an investment to an enterpriseBusiness value is a method of determining the health and well-being of a firm in the long-runBusiness value includes employee, customer, supplier, alliance, management, and societal value

5

What is Agility?A-gil-i-ty (ə-'ji-lə-tē) Quickness, lightness, and ease of movement; nimbleness

Agility is the ability to create and respond to changein order to profit in a turbulent business environmentAgility is reprioritizing for maneuverability because of shifting requirements, technology, and knowledgeAgility is a very fast response to changes in customer requirements through intensive customer interactionAgility is the use of adaptability and evolutionary delivery to promote rapid customer responsivenessAgility is a better way of developing products using collaboration, teamwork, iterations, and flexibility

6

What are Agile Methods?‘Adaptable’ software development methodologies‘Human-centric’ method for creating business value‘Alternative’ to large document-based methodologies

Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org

7

Essence of Agile MethodsHigh degree of customer & developer interactionHighly-skilled teams producing frequent iterationsRight-sized, just-enough, and just-in-time process

Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.

8

Why use Agile Methods?Adaptability to changing market/customer needsBetter cost efficiencies and fastest time-to-marketImproved quality, satisfaction, and project success

Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org

9

Antecedents of Agile MethodsRooted in management evolution from early 1900sEvolved from software methods from 1950s/1960sSpinoffs of NPD/RAD approaches from the 1980s

Rico, D. F., Sayani, H. H., & Field, R. F. (2008). History of computers, electronic commerce, and agile methods. In M. V. Zelkowitz (Ed.), Advances in computers: Emerging technologies, Vol. 73. San Diego, CA: Elsevier.

10

AgendaIntroduction

Sources of Business ValueSurveys of Business ValueMeasures of Business ValueModels of Business ValueEstimation of Business ValueComparison of Business ValueSummary of Business Value

11

Types of Agile MethodsCrystal Methods and Scrum 1st Agile MethodsExtreme Programming swept the globe by 2002Scrum/Extreme Programming hybrids are popular

Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.

Feature Lists (Customer Needs), Domain Model (Object Orientation), Inspection (Peer Review)

Use Cases, Domain Models, Frequent Delivery, Reflection Workshops, Risk Management

DSDM

Easel

IBM

Beck

De Luca

Millington

Sutherland

Cockburn

Author

Release Planning, Onsite Customers, Iterations,Pair Programming, Test-Driven Development

User Involvement, Time Boxes and Prototypes (Iterations), Testing and Quality Assurance

Backlogs (Feature Lists), Daily Scrums, Sprints (Iterations), Retrospectives (Post Mortems)

Major Features

Chrysler

Nebulon

Firm

1998

1997

1993

1993

1991

Year

Feature-Driven Development

CrystalMethods

Extreme Programming

Dynamic Systems

Development

Scrum

Method

12

Crystal MethodsCreated by Alistair Cockburn in 1991Has 14 practices, 10 roles, and 25 productsScalable family of techniques for critical systems

Cockburn, A. (2002). Agile software development. Boston, MA: Addison-Wesley.

13

ScrumCreated by Jeff Sutherland at Easel in 1993Has 5 practices, 3 roles, 5 products, rules, etc.Uses EVM to burn down backlog in 30-day iterations

Schwaber, K., & Beedle, M. (2001). Agile software development with scrum. Upper Saddle River, NJ: Prentice-Hall.

14

Dynamic Systems Develop.Created by group of British firms in 199315 practices, 12 roles, and 23 work productsNon-proprietary RAD approach from early 1990s

Stapleton, J. (1997). DSDM: A framework for business centered development. Harlow, England: Addison-Wesley.

15

Feature Driven DevelopmentCreated by Jeff De Luca at Nebulon in 1997Has 8 practices, 14 roles, and 16 work productsUses object-oriented design and code inspections

Palmer, S. R., & Felsing, J. M. (2002). A practical guide to feature driven development. Upper Saddle River, NJ: Prentice-Hall.

16

Extreme ProgrammingCreated by Kent Beck at Chrysler in 1998Has 28 practices, 7 roles, and 7 work productsPopularized pair programming and test-driven dev.

Beck, K. (2000). Extreme programming explained: Embrace change. Reading, MA: Addison-Wesley.

17

Extreme Programming (cont’d)RELEASE PLANNING — Best Practice

Created by Kent Beck at Chrysler in 1998Used for managing both XP and Scrum projectsLight, flexible, and adaptable project mgt. framework

Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.

18

PAIR PROGRAMMING — Best PracticeTerm coined by Jim Coplien in 1995Consists of two side-by-side programmersHighly-effective group problem-solving technique

Williams, L., & Kessler, R. (2002). Pair programming illuminated. Boston, MA: Pearson Education.

Extreme Programming (cont’d)

19

TEST-DRIVEN DEV. — Best PracticeTerm coined by Kent Beck in 2003Consists of writing unit tests before codingSubject to automated testing/continuous integration

Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley.


20

REFACTORING — Best PracticeTerm coined by William Opdyke in 1990Improve readability, maintainability, and qualitySoftware is continuously rewritten for every iteration


Fowler, M. (1999). Refactoring: Improving the design of existing code. Boston, MA. Addison-Wesley.

21

AgendaIntroductionSources of Business Value

Surveys of Business ValueMeasures of Business ValueModels of Business ValueEstimation of Business ValueComparison of Business ValueSummary of Business Value

22

Surveys of Agile MethodsNumerous surveys of Agile Methods since 2003AmbySoft and Version One collect annual dataGenerally include both hard and soft benefits

Rico, D. F. (2008). What is the return-on-investment of agile methods? Retrieved February 3, 2009, from http://davidfrico.com/rico08a.pdf

23

Shine TechnologiesSurvey of 131 international respondentsExtreme Programming (58%) and Scrum (8%)85% of respondents were experts in Agile Methods

Johnson, M. (2003). Agile methodologies: Survey results. Victoria, Australia: Shine Technologies.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Productivity

Quality

Satisfaction

Cost

Improvement

24

Agile JournalSurvey of 400 international respondentsExtreme programming (28%) and Scrum (20%)80% using Agile Methods to deliver maximum value

Barnett, L. (2006). And the agile survey says. Agile Journal, 1(1).

0% 10% 20% 30% 40% 50%

Time to Market

Quality

Alignment

Cost

Improvement

25

MicrosoftSurvey of 492 Microsoft respondentsScrum (65%) and Extreme Programming (5%)65% using Agile Methods in virtual distributed teams

Begel, A., & Nagappan, N. (2007). Usage and perceptions of agile software development in an industrial context: An exploratory study. Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, Madrid, Spain, 255-264.

0% 10% 20% 30% 40% 50% 60% 70%

Communication

Time to Market

Flexibility

Quality

Satisfaction

Productivity

Improvement

26

UMUCSurvey of 250 international respondents70% of respondents using Agile Methods83% of were from small-to-medium sized firms

Rico, D. F., Sayani, H. H., Stewart, J. J., & Field, R. F. (2007). A model for measuring agile methods and website quality. TickIT International, 9(3), 3-15.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Satisfaction

Productivity

Cycle Time

Quality

Cost

Improvement

27

AmbySoftSurvey of 642 international respondents69% of firms had adopted an Agile Method62% were from firms with less than 1,000 people

Ambler, S. W. (2008). Agile adoption survey. Retrieved October 17, 2008, from http://www.ambysoft.com/downloads/surveys/AgileAdoption2008.ppt

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Productivity

Satisfaction

Project Success

Quality

Cost

Virtual Success

Improvement

28

IT AgileSurvey of 207 respondents in GermanyScrum (21%) and Extreme Programming (14%)97% of respondents are satisfied with Agile Methods

Wolf, H., & Roock, A. (2008). Agile becomes mainstream: Results of an Online Survey. Object Spektrum, 15(3), 10-13.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Flexibility

Job Satisfaction

Learning on the Job

Productivity

Project Status

Quality

Improvement

29

Version OneSurvey of 3,061 respondents from 80 countriesScrum (49%), Scrum/XP (22%), and XP (8%)68% from small firms and 57% distributed

Version One. (2008). The state of agile development: Third Annual Survey. Alpharetta, GA: Author.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Cycle Time

Project Visibility

Productivity

Quality

Maintainability

Cost

Improvement

30

AgendaIntroductionSources of Business ValueSurveys of Business Value

Measures of Business ValueModels of Business ValueEstimation of Business ValueComparison of Business ValueSummary of Business Value

31

Measures of Business ValueA major principle of Agile Methods is creating valueROI is the measure of value within Agile MethodsThere are seven closely related ROI measures

Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.

Metric Definition Formula Costs

Sum of Costs Total amount of money spent ∑

=

n

iiCost

1

Benefits Sum of Benefits

Total amount of money gained ∑=

n

iiBenefit

1

B/CR Benefit to Cost Ratio

Ratio of benefits to costs Costs

Benefits

ROI Return on Investment

Ratio of adjusted benefits to costs %100×−

CostsCostsBenefits

NPV Net Present Value

Discounted cash flows ∑=

−+

Years

iYears

i CostsRateDiscount

Benefits1

0)1(

BEP Breakeven Point

Point when benefits exceed costs 1−CostsNewCostsOldCostsNew

ROA Real Options Analysis

Value gained from strategic delay ( ) ( ) YearsRateeCostsdNBenefitsdN ×−××−× 21

d1 = [ln(Benefits ÷ Costs) + (Rate + 0.5 × Risk2) × Years] ÷ Risk × √ Years, d2 = d1 − Risk × √ Years

32

CostsTotal amount of money spent on Agile MethodsIncludes training, coaching, automated tools, etc.Minimally, includes the dev. effort of Agile Methods


∑=

n

iiCost

1

33

BenefitsTotal amount of money gained from Agile MethodsIncludes economic benefit from using new systemMinimally, includes maintenance rework savings


∑=

n

iiBenefit

1

34

Benefit to Cost RatioRatio of total benefits to total costs of Agile MethodsIncludes development, maintenance, and businessMinimally, benefits should be larger than all costs


CostsBenefits

35

Return on InvestmentRatio of adjusted benefits to costs of Agile MethodsBenefits are adjusted downward using total costsMinimally, benefits should be larger than costs


%100×−

CostsCostsBenefits

36

Net Present ValueDiscounted benefits of using Agile MethodsFuture benefits are discounted due to inflationMinimally, future benefits should exceed all costs


∑=

−+

Years

iYears

i CostsRateDiscount

Benefits1

0)1(

37

Breakeven PointPoint when benefits exceed costs of Agile MethodsPoint where slope of benefits and costs intersectMinimally, old costs should exceed new costs


1−CostsNewCostsOldCostsNew

38

Real Options AnalysisIterative benefits gained from using Agile MethodsFuture benefits are increased because of risksMinimally, benefits should exceed costs


( ) ( ) YearsRateeCostsdNBenefitsdN ×−××−× 21

d1 = [ln(Benefits ÷ Costs) + (Rate + 0.5 × Risk2) × Years] ÷ Risk × √ Yearsd2 = d1 − Risk × √ Years

39

AgendaIntroductionSources of Business ValueSurveys of Business Value

Models of Business ValueMeasures of Business ValueEstimation of Business ValueComparison of Business ValueSummary of Business Value

40

Software Lifecycle Costs1:10:100 is a classical ratio of dev. to maint. hoursDefects have negative multiplicative effect on costA conservative and contemporary ratio is 1:6:30

Boehm, B. W. (1981). Software engineering economics. Englewood Cliffs, NJ: Prentice-Hall.Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.

Rel

a tiv

e C

o st

to F

ix E

rror

41

Software Cost ModelsCost estimation models still in use todayUsed to estimate effort of Traditional MethodsAdjusted average of 5,088 used for ROI estimation

Benediktsson, O., & Dalcher, D. (2005). Estimating size in incremental software development projects. Journal of Engineering Manufacture, 152(6), 253-259.

42

Total Lifecycle Costs0.51 hours/line of code for Traditional Methods10% defect inject rate (1,000 defects/10 KLOC)67% of defects in test (33% in maintenance)

Rico, D. F. (2004). ROI of software process improvement: Metrics for project managers and software engineers. Boca Raton, FL: J. Ross Publishing.In, H. P., et al. (2006). A quality-based cost estimation model for the product line life cycle. Communications of the ACM, 49(12), 85-88.McCann, B. (2007). The relative cost of interchanging, adding, or dropping quality practices. Crosstalk, 20(6), 25-28.

43

Agile Productivity StudiesProductivity data found in 26 Agile Methods studiesStudies conducted from 2002 to the present timeAverage productivity 21.2374 LOC per hour


Abrahamsson 2003 XP Case Study 19.2550

Abrahamsson & Koskela 2004 XP Case Study 16.9000

Back, Hirkman, & Milovanov 2004 XP Experiment 8.0000

Cohn 2008 Scrum Case Study 5.9050

Jones 2008 Scrum Case Study 5.7400

Sutherland 2006 Scrum Case Study 4.6858

Average 21.2374

44

Agile Productivity ModelsBased on 13 studies of Extreme Programming (XP)Also based on 7 studies of pair programming (PP)“Pair programming” had highest productivity


45

Agile Quality StudiesDefect density found in 21 studies of Agile MethodsStudies conducted from 2002 to the present timeAverage quality 1.7972 defects per KLOC


Abrahamsson 2003 XP Case Study 2.1450

Abrahamsson & Koskela 2004 XP Case Study 1.4300

Back, Hirkman, & Milovanov 2004 XP Experiment 0.7000

Cohn 2008 Scrum Case Study 2.9000

Jones 2008 Scrum Case Study 8.5000

Schatz & Abdelshafi 2005 Scrum Case Study 0.4350

Average 1.7972

46

Quality ModelsBased on 10 studies of Extreme Programming (XP)Also based on 6 studies of pair programming (PP)“Extreme Programming” had the highest quality


47

AgendaIntroductionSources of Business ValueSurveys of Business ValueMeasures of Business ValueModels of Business Value

Estimation of Business ValueComparison of Business ValueSummary of Business Value

48

Agile Lifecycle CostsCosts based on productivity and quality modelsDevelopment costs based on LOC ÷ productivity rateMaintenance costs based on defects × KLOC × MH


49

Agile Lifecycle BenefitsBenefits based on total traditional less agile costsTraditional costs based LOC × dev. + maint. effortTraditional costs credited testing effort applied


1. XP

2. TDD

3. PP

4. Scrum

5. Agile

No. Method

$1,708,039

$1,714,690

$1,573,096

$1,695,146

Benefits

$1,667,079

Agile Lifecycle Benefit Models

(10,000 3.51 – 3,651.48 4.47) 100 – $208,069

(10,000 3.51 – 3,651.48 4.47) 100 – $167,109

(10,000 3.51 – 3,651.48 4.47) 100 – $160,459

(10,000 3.51 – 3,651.48 4.47) 100 – $302,052

(10,000 3.51 – 3,651.48 4.47) 100 – $180,002

50

Extreme ProgrammingCosts based on avg. productivity and qualityProductivity moderated from 16.1575 to 5.3858Costs were $208,069, benefits were $1,667,079


∑ =

5

1i

51

Test Driven DevelopmentCosts based on avg. productivity and qualityProductivity moderated from 29.2800 to 9.7600Costs were $167,109, benefits were $1,708,039


∑ =

5

1i

52

Pair ProgrammingCosts based on avg. productivity and qualityProductivity moderated from 33.4044 to 11.135Costs were $160,459, benefits were $1,714,690


∑ =

5

1i

53

ScrumCosts based on avg. productivity and qualityProductivity data remained the same at 5.4436Costs were $302,052, benefits were $1,573,096


∑ =

5

1i

54

Agile MethodsCosts based on avg. productivity and qualityProductivity data resulted in average of 7.9311Costs were $180,002, benefits were $1,695,146


∑ =

5

1i

55

AgendaIntroductionSources of Business ValueSurveys of Business ValueMeasures of Business ValueModels of Business ValueEstimation of Business Value

Comparison of Business ValueSummary of Business Value

56

Data for Agile MethodsAgile Methods were ranked based on ROIAgile Methods with high quality had lower ROIAgile Methods with high productivity had high ROI

Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls

57

ROI of Agile MethodsAgile Methods were ordered based on ROIAgile Methods had a high ROI value of 969%Agile Methods yielded an average ROI of 842%


58

Data for Traditional MethodsTraditional Methods were ranked based on ROIMethods with good cost and quality had higher ROIAgile Methods had better ROI than heaviest methods


59

ROI of Traditional MethodsTraditional Methods were ordered using ROITraditional Methods had high ROI value of 1,562%Agile Methods had better ROI than heaviest methods


60

Data for All MethodsSoftware methods were ranked based on ROIMethods with good cost and quality had best ROIBest Agile and Traditional Methods had similar ROI


Traditional

Traditional

Agile

Traditional

Agile

Agile

Traditional

Agile

Traditional

Traditional

Type Costs Benefits B/CR ROI NPV BEP ROAMethod

Agile

PSPsm $105,600 $1,755,148 17:1 1,562% $1,414,174 $945 $1,672,907

TSPsm $148,400 $1,706,648 12:1 1,050% $1,329,379 $5,760 $1,591,127

Inspections $82,073 $897,499 11:1 994% $695,067 $51,677 $833,681

PP $160,459 $1,714,690 11:1 969% $1,324,283 $5,919 $1,590,034

TDD $167,109 $1,708,039 10:1 922% $1,311,874 $6,430 $1,578,575

Agile $180,002 $1,695,146 9:1 842% $1,287,817 $7,483 $1,556,997

XP $208,069 $1,667,079 8:1 701% $1,235,446 $10,064 $1,513,332

Scrum $302,052 $1,573,096 5:1 421% $1,060,084 $21,682 $1,389,810

SW-CMM® $311,433 $1,153,099 4:1 270% $687,030 $153,182 $998,013

ISO 9001 $173,000 $566,844 3:1 228% $317,828 $1,196,206 $486,750

CMMI® $1,108,233 $1,153,099 1:1 4% -$109,770 $545,099 $891,412

61

ROI of All MethodsSoftware methods were ordered by ROIAgile Methods had a high ROI value of 969%Traditional Methods had high ROI value of 1,562%


62

Unadjusted ROI of All MethodsAre data based on unrealistic laboratory conditions?Are productivity data from lab studies optimistic?Are total lifecycle costs closer to 1:10:100?


63

AgendaIntroductionSources of Business ValueSurveys of Business ValueMeasures of Business ValueModels of Business ValueEstimation of Business ValueComparison of Business Value

Summary of Business Value

64

Benefit SummaryAgile (138 pt.) and Traditional Methods (99 pt.)Agile Methods fare better in all benefits categoriesAgile Methods 459% better than Traditional Methods


65

Cost of QualityApply traditional reliability and quality theoryDefects are inexpensive to remove early in cycleLate bug removal has negative, multiplicative effect

Rico, D. F. (2000). Using cost benefit analyses to develop software process improvement (SPI) strategies. Rome, NY: DACS.

Inspection Cost (57X PSP)

Soft

war

e De

fect

s

PSP

Test Cost (138X PSP)

Ad Hoc (326X)

Analysis Design CodeUnitTest

System Test

ComponentTest

CustomerUse

PSP Cost (326X lower than Ad Hoc)

66

Real OptionsNPV models losses of Traditional MethodsReal options model profits from Agile MethodsAgile Methods incur less initial risk and higher ROI

Fichman, R. G., Keil, M., & Tiwana, A. (2005). Beyond valuation: Options thinking in IT project management. California Management Review, 47(2), 74-96.

Prob

abili

ty

Prob

abili

ty

67

Agile vs. Traditional MetricsAgile Methods are a fundamentally new paradigmAgile Methods are “not” lighter Traditional MethodsThey should not be viewed through a Traditional lens

Rico, D. F. (2009). Metrics for agile methods. Retrieved February 7, 2009, from http://davidfrico.com/agile-metrics.pdf

68

New BookGuide to Agile Methods for business leadersCommunicates business value of Agile MethodsRosetta stone to Agile Methods for Traditional folks

* Rosetta stone to the business value and culture of Agile Methods for executives, managers, and thought leaders in the field of software methods.

THE BUSINESS VALUE OF AGILE METHODS

Maximizing ROI with Right-Sized, Just-Enough, and Just-in-Time Processes and Documentation

DR. DAVID F. RICO, DR. HASAN H. SAYANI AND DR. SAYA SONE

Forward by Dr. Jeffrey V. Sutherland

Table of Contents

1. Introduction2. Values of Agile Methods3. History of Agile Methods4. Antecedents of Agile Methods5. Types of Agile Methods6. Practices of Agile Methods7. Agile Project Management8. Agile Software Engineering9. Agile Support Processes10. Agile Tools and Technologies11. Comparison of Agile Methods12. Agile Metrics and Models13. Costs of Agile Methods14. Benefits of Agile Methods15. ROI of Agile Methods16. NPV of Agile Methods17. Real Options of Agile Methods18. Conclusion

69

ReferencesRico, D. F. (2000). Using cost benefit analyses to develop software process improvement (SPI) strategies. Rome, NY: DACS.Rico, D. F. (2002). How to Estimate ROI for Inspections, PSP, TSP, SW-CMM, ISO 9001, and CMMI. Software Tech News, 5(4), 23-31.Rico, D. F. (2002). The Return on investment in quality. TickIT International, 4(4), 13-18.Rico, D. F. (2004). ROI of software process improvement: Metrics for project managers and software engineers. Boca Raton, FL: J. Ross Publishing.Rico, D. F. (2005). Practical metrics and models for return on investment. TickIT International, 7(2), 10-16.Rico, D. F. (2006). A framework for measuring the ROI of enterprise architecture. International Journal of End User Computing, 18(2), 1-12.Rico, D. F. (2007). Optimizing the ROI of enterprise architecture using real options. In S. Clarke (Ed.), End user computing challenges and technologies: Emerging tools and applications. Hershey, PA: Information Science Reference.Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? An analysis of extreme programming, test-driven development, pair programming, and scrum (using real options). TickITInternational, 10(4), 9-18.Solingen, R. A., & Rico, D. F. (2006). Calculating software process improvement’s return on investment. In M. V. Zelkowitz (Ed.), Advances in computers: Quality software development, Vol. 66 (pp. 1-41). San Diego, CA: Elsevier.

70

Contact InformationWebsite: http://davidfrico.comBiography: http://www.linkedin.com/in/davidfricoCapabilities: http://davidfrico.com/rico-capability.pdf

business value of agile methods: using return on investment

Documents