business value of agile methods using return on investment dr. david f. rico, pmp, csm

Business Value of

Agile Methods

Using Return on InvestmentDr. David F. Rico, PMP, CSM

2

Agenda

Introduction Sources of Business Value

Planning for Business Value Surveys of Business Value

Measures of Business ValueModels of Business ValueEstimation of Business Value

Comparison of Business Value Summary of Business Value

3

Author DoD contractor with 25+ years of IT experience B.S. Comp. Sci., M.S. Soft. Eng., D.M. Info. Tech. Large NASA & DoD programs (U.S., Japan, Europe)

* Published five textbooks and over 15 articles on various topics in return on investment, information technology, agile methods, etc.

4

Purpose Provide an overview of the business value of

Agile Methods using return on investment: Business value is an approach for estimating the

tangible and intangible worth of organizational assets Business value is an appraisal of intellectual assets

such as knowledge, experience, and skills Business value is a technique for determining the

complete worth of an investment to an enterprise Business value is a method of determining the health

and well-being of a firm in the long-run Business value includes employee, customer,

supplier, alliance, management, and societal value

5

What is Agility? A-gil-i-ty (ə-'ji-lə-tē) Quickness, lightness,

and ease of movement; nimbleness Agility is the ability to create and respond to change

in order to profit in a turbulent business environment Agility is reprioritizing for maneuverability because of

shifting requirements, technology, and knowledge Agility is a very fast response to changes in customer

requirements through intensive customer interaction Agility is the use of adaptability and evolutionary

delivery to promote rapid customer responsiveness Agility is a better way of developing products using

collaboration, teamwork, iterations, and flexibility

6

What are Agile Methods? ‘Adaptable’ software development methodologies ‘Human-centric’ method for creating business value ‘Alternative’ to large document-based methodologies

Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org

alsoknown as

CustomerCollaboration

Individuals &Interactions

WorkingSoftware

Respondingto Change

CustomerInteraction

High-Performance Teams

IterativeDevelopment

Adaptabilityor Flexibility

ContractNegotiation

Processes& Tools

ComprehensiveDocumentation

Followinga Plan

Agile Methods‘Values’

alsoknown as

alsoknown as

alsoknown as

valuedmore than

valuedmore than

valuedmore than

valuedmore than

Agile Methods‘Principles’

Traditional Methods‘Values’

7

Essence of Agile Methods High degree of customer & developer interaction Highly-skilled teams producing frequent iterations Right-sized, just-enough, and just-in-time process

Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.

Adaptability or Flexibility


Iterative Development

Customer Interaction

8

Why use Agile Methods? Adaptability to changing market/customer needs Better cost efficiencies and fastest time-to-market Improved quality, satisfaction, and project success

Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org

9

Antecedents of Agile Methods Rooted in management evolution from early 1900s Evolved from software methods from 1950s/1960s Spinoffs of NPD/RAD approaches from the 1980s

Rico, D. F., Sayani, H. H., & Field, R. F. (2008). History of computers, electronic commerce, and agile methods. In M. V. Zelkowitz (Ed.), Advances in computers: Emerging technologies, Vol. 73. San Diego, CA: Elsevier.

1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004

J oint Application Development

Rapid Systems Development

New DevelopmentRhythm

Rapid Application Development

Partic ipatoryDesign

Synch-n-Stabilize

Internet Time

J udo Strategy

Rational Unified Process

10

Agenda

Introduction

Sources of Business ValuePlanning for Business ValueSurveys of Business ValueMeasures of Business ValueModels of Business ValueEstimation of Business Value


11

Types of Agile Methods Crystal Methods and Scrum 1st Agile Methods Extreme Programming swept the globe by 2002 Scrum/Extreme Programming hybrids are popular

Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.

Feature Lists (Customer Needs), Domain Model (Object Orientation), Inspection (Peer Review)

Use Cases , Domain Models, Frequent Delivery, Reflection Workshops, Risk Management

DSDM

Easel

IBM

Beck

De Luca

Millington

Sutherland

Cockburn

Author

Release Planning, Onsite Customers, Iterations,Pair Programming, Test-Driven Development

User Involvement, Time Boxes and Prototypes (Iterations), Testing and Quality Assurance

Backlogs (Feature Lists), Daily Scrums, Sprints (Iterations), Retrospectives (Post Mortems)

Major Features

Chrysler

Nebulon

Firm

1998

1997

1993

1993

1991

Year

Feature-Driven Development

CrystalMethods

Extreme Programming

Dynamic Systems

Development

Scrum

Method

12

Crystal Methods Created by Alistair Cockburn in 1991 Has 14 practices, 10 roles, and 25 products Scalable family of techniques for critical systems

Cockburn, A. (2002). Agile software development. Boston, MA: Addison-Wesley.

13

Scrum Created by Jeff Sutherland at Easel in 1993 Has 5 practices, 3 roles, 5 products, rules, etc. Uses EVM to burn down backlog in 30-day iterations

Schwaber, K., & Beedle, M. (2001). Agile software development with scrum. Upper Saddle River, NJ: Prentice-Hall.

14

Dynamic Systems Develop. Created by group of British firms in 1993 15 practices, 12 roles, and 23 work products Non-proprietary RAD approach from early 1990s

Stapleton, J. (1997). DSDM: A framework for business centered development. Harlow, England: Addison-Wesley.

15

Feature Driven Development Created by Jeff De Luca at Nebulon in 1997 Has 8 practices, 14 roles, and 16 work products Uses object-oriented design and code inspections

Palmer, S. R., & Felsing, J. M. (2002). A practical guide to feature driven development. Upper Saddle River, NJ: Prentice-Hall.

Develop anOverall Model

Build aFeatures List

Plan byFeature

Design byFeature

Build byFeature

Iteration

16

Extreme Programming Created by Kent Beck at Chrysler in 1998 Has 28 practices, 7 roles, and 7 work products Popularized pair programming and test-driven dev.

Beck, K. (2000). Extreme programming explained: Embrace change. Reading, MA: Addison-Wesley.

UserStories

ArchitecturalSpike

ReleasePlanning

IterationAcceptance

TestsSmall

Releases

Spike

TestScenarios

SystemMetaphor

CustomerApproval

LatestVersion

ReleasePlan

NextIteration

BugsNew

StoriesRequirements

UncertainEstimates

ConfidentEstimates

XPExtreme Programming

17

Extreme Programming (cont’d) RELEASE PLANNING — Best Practice

Created by Kent Beck at Chrysler in 1998 Lightweight project management framework Used for managing both XP and Scrum projects

Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.

Release Plan Iteration Plan

UserWrite

StoriesUser

Estimate

StoriesUser

Split

StoriesUser

Order

StoriesUser

Analyze

StoriesDev.

Create

TasksDev.

Accept

TasksDev.

Estimate

Tasks

18

PAIR PROGRAMMING — Best Practice Term coined by Jim Coplien in 1995 Consists of two side-by-side programmers Highly-effective group problem-solving technique

Williams, L., & Kessler, R. (2002). Pair programming illuminated. Boston, MA: Pearson Education.

MOVE PEOPLE AROUND

REFACTOR MERCILESSLY

CONTINUOUS INTEGRATION

CREATE AUNIT TEST

We NeedHelp

ChangePair

SimpleCode

ComplexCode

New UnitTests

NewFunction-

ality

FailedUnitTest

PassedUnitTest

Extreme Programming (cont’d)

19

REFACTORING — Best Practice Term coined by William Opdyke in 1990 Process of frequently rewriting source code Improves readability, maintainability, and quality


Fowler, M. (1999). Refactoring: Improving the design of existing code. Boston, MA. Addison-Wesley.

20

TEST-DRIVEN DEV. — Best Practice Term coined by Kent Beck in 2003 Consists of writing all tests before coding Ensures all source code is verified and validated

Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley.


21

Extreme Programming (cont’d) CONT. INTEGRATION — Best Practice

Term coined by Martin Fowler in 1998 Process of automated build/regression testing Evaluates impact of all changes against entire system

Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley

BuildIntegration

Server

VersionControlServer

BuildScripts

UsesWatches

BuildStatus

ProvidesDeveloper A

Developer B

Developer C

CommitsChanges

CommitsChanges

CommitsChanges

Compile Source Code

Run Unit Tests

Run Coverage Tests

Static Code Analysis

Build Database

Generate Help Files

Archive and Deploy

22

Agenda

IntroductionSources of Business Value

Planning for Business ValueSurveys of Business ValueMeasures of Business ValueModels of Business ValueEstimation of Business Value


23

Release Planning Deliverables Used in both Extreme Programming and Scrum Lightweight framework of Agile planning products Ranges from release plans down through unit tests


No. Deliverable Description

1. Release Plan Fluid informal roadmap (or program plan) for planning software releases (usually containing one or more iterations).

2. Iteration Plan Informal project plan that divides an iteration into user stories and development tasks (usually spanning two to three weeks)

3. User Story A software requirement that has value to end-users or customers (usually a simple sentence written on an index card)

4. Metaphor A simple narrative about how the whole system works (usually written as a sentence or paragraph with major objects)

5. Development Task A development activity necessary to satisfy a user story (usually a numbered list of software development activities)

6. Acceptance Test An end-user test to determine if an iteration satisfies its acceptance criteria (usually written and executed by customers)

7. Unit Test A development test to determine if software components are working properly (usually written and executed by developers)

24

Release Plan Fluid, informal roadmap for planning releases Includes dates for releases, iterations, and stories Must prioritize, split, estimate, and order user stories


Release Plan

<Release Plan>: <Release 1, 2, n>

<Release>: <Iteration 1, 2, n>

<Iteration>: <Story 1, 2, n>

Release PlanRelease Iteration

1

2

3

4

n

1

1

2

2

n

Story

01 thru 06

07 thru 12

13 thru 18

19 thru 24

25 thru nn

25

Iteration Plan Plan that divides iterations into development tasks Each iteration is one to three weeks in duration Iteration plans updated using daily standups

Iteration Plan

<Iteration Plan>: <Story 1, 2, n>

<Story>: <Task 1, 2, n>

<Task>: <Developer 1, 2, n>

<Status>: <Days Complete>

Iteration PlanStory

1

1

2

2

n

Task

1

2

3

4

n

Developer

Bob

Sue

Mary

John

n

Status

1/3

2/3

3/3

3/3

n/n


26

User Story A function or feature of value to a customer An estimable and testable software requirement Six user stories should be implemented per iteration


<Title of User Story>

As a <Type of User> I can <Goal of User> so that <Objective of User>

Make a Reservation

As a customer, I can make a reservation so that I can

perform personal travel

27

System Metaphor Simple story about how the whole system works Overarching 10,000 foot view of system architecture Pushes the system into a sense of coherent cohesion

System Metaphor

<Metaphor>: <Object 1>, <Object 2>, <Object n>

System Metaphor

Shopping Cart: Item, Description, Barcode, Price, Quantity, Subtotal, Tax, Discount, Total, etc.


28

Development Tasks Detailed steps for implementing user stories User stories are decomposed into technical tasks Brainstormed by developers to last two to three days

<Title of Development Task>

<Action of Developer> a <Software Unit> using

<Technology>

Splash Screen

Design a splash screen using PhotoShop


29

Acceptance Tests Black-box, functional tests to be performed Specified by customers during iteration planning Run when user stories and unit tests are completed

<Title of User Story>

Verify <Type of User> can <Satisfy their Goals and

Objectives>

Make a Reservation· Verify customers can establish a

reservation

· Verify customers can change a reservation

· Verify customers can cancel a reservation


30

Unit Tests A test written from the developer’s perspective Each task is implemented by two programmers Unit tests are developed prior to implementation

<Title of Development Task>

Verify <Type of User> can <Satisfy Task> when <Condition Occurs>

Make a Splash Screen· Verify customers can see splash

screen when they visit website

· Verify customers can see company logo when splash screen executes

· Verify customers can skip splash screen when they want to enter site


31

Agenda

IntroductionSources of Business ValuePlanning for Business Value

Surveys of Business ValueMeasures of Business ValueModels of Business ValueEstimation of Business Value


32

Surveys of Agile Methods Numerous surveys of Agile Methods since 2003 AmbySoft and Version One collect annual data Generally include both hard and soft benefits

Rico, D. F. (2008). What is the return-on-investment of agile methods? Retrieved February 3, 2009, from http://davidfrico.com/rico08a.pdf

Johnson 131

Barnett 400

Begel, et al. 492

Rico, et al. 250

Ambler 642

Wolf, et al. 207

Hanscom 3,061

Author Size

93%

45%

14%

81%

82%

78%

74%

Productivity

88%

43%

32%

80%

72%

74%

68%

Quality

49%

23%

16%

75%

72%

72%

38%

Cost

Shine

Agile Journal

Microsoft

UMUC

AmbySoft

IT Agile

Version One

Organization

2003

2006

2007

2007

2008

2008

2008

Year

67% 65% 49%Average

33

Shine Technologies Survey of 131 international respondents Extreme Programming (58%) and Scrum (8%) 85% of respondents were experts in Agile Methods

Johnson, M. (2003). Agile methodologies: Survey results. Victoria, Australia: Shine Technologies.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Produc tivity

Quality

Satisfac tion

Cost

Improvement

34

Agile Journal Survey of 400 international respondents Extreme programming (28%) and Scrum (20%) 80% using Agile Methods to deliver maximum value

Barnett, L. (2006). And the agile survey says. Agile Journal, 1(1).

0% 10% 20% 30% 40% 50%

Time to Market

Quality

Alignment

Cost

Improvement

35

Microsoft Survey of 492 Microsoft respondents Scrum (65%) and Extreme Programming (5%) 65% using Agile Methods in virtual distributed teams

Begel, A., & Nagappan, N. (2007). Usage and perceptions of agile software development in an industrial context: An exploratory study. Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, Madrid, Spain, 255-264.

0% 10% 20% 30% 40% 50% 60% 70%

Communication

Time to Market

F lexibility

Quality

Satisfac tion

Produc tivity

Improvement

36

UMUC Survey of 250 international respondents 70% of respondents using Agile Methods 83% of were from small-to-medium sized firms

Rico, D. F., Sayani, H. H., Stewart, J. J., & Field, R. F. (2007). A model for measuring agile methods and website quality. TickIT International, 9(3), 3-15.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Satisfac tion

Produc tivity

Cyc le Time

Quality

Cost

Improvement

37

AmbySoft Survey of 642 international respondents 69% of firms had adopted an Agile Method 62% were from firms with less than 1,000 people

Ambler, S. W. (2008). Agile adoption survey. Retrieved October 17, 2008, from http://www.ambysoft.com/downloads/surveys/AgileAdoption2008.ppt

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Produc tivity

Satisfac tion

Projec t Success

Quality

Cost

Virtual Success

Improvement

38

IT Agile Survey of 207 respondents in Germany Scrum (21%) and Extreme Programming (14%) 97% of respondents are satisfied with Agile Methods

Wolf, H., & Roock, A. (2008). Agile becomes mainstream: Results of an Online Survey. Object Spektrum, 15(3), 10-13.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Flexibility

J ob Satisfac tion

Learning on the J ob

Produc tivity

Projec t Status

Quality

Improvement

39

Version One Survey of 3,061 respondents from 80 countries Scrum (49%), Scrum/XP (22%), and XP (8%) 68% from small firms and 57% distributed

Version One. (2008). The state of agile development: Third Annual Survey. Alpharetta, GA: Author.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Cyc le Time

Projec t Visibility

Produc tivity

Quality

Maintainability

Cost

Improvement

40

Agenda

IntroductionSources of Business ValuePlanning for Business ValueSurveys of Business Value



41

Measures of Business Value A major principle of Agile Methods is creating value ROI is the measure of value within Agile Methods There are seven closely related ROI measures

Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.

Metric Definition Formula Costs

Sum of Costs Total amount of money spent

n

iiCost

1

Benefits Sum of Benefits

Total amount of money gained

n

iiBenefit

1

B/CR Benefit to Cost Ratio

Ratio of benefits to costs

Costs

Benefits

ROI Return on Investment

Ratio of adjusted benefits to costs

%100

Costs

CostsBenefits

NPV Net Present Value

Discounted cash flows

Years

iYears

i CostsRateDiscount

Benefits

10)1(

BEP Breakeven Point

Point when benefits exceed costs

1CostsNewCostsOld

CostsNew

ROA Real Options Analysis

Value gained from strategic delay

YearsRateeCostsdNBenefitsdN 21

d1 = [ln(Benefits Costs) + (Rate + 0.5 Risk2) Years] Risk Years, d2 = d1 Risk Years

42

Costs Total amount of money spent on Agile Methods Includes training, coaching, automated tools, etc. Minimally, includes the dev. effort of Agile Methods


n

iiCost

1

43

Benefits Total amount of money gained from Agile Methods Includes economic benefit from using new system Minimally, includes maintenance rework savings


n

iiBenefit

1

44

Benefit to Cost Ratio Ratio of total benefits to total costs of Agile Methods Includes development, maintenance, and business Minimally, benefits should be larger than all costs


Costs

Benefits

45

Return on Investment Ratio of adjusted benefits to costs of Agile Methods Benefits are adjusted downward using total costs Minimally, benefits should be larger than costs


%100

Costs

CostsBenefits

46

Net Present Value Discounted benefits of using Agile Methods Future benefits are discounted due to inflation Minimally, future benefits should exceed all costs


Years

iYears

i CostsRateDiscount

Benefits

10)1(

47

Breakeven Point Point when benefits exceed costs of Agile Methods Point where slope of benefits and costs intersect Minimally, old costs should exceed new costs


1CostsNewCostsOld

CostsNew

48

Real Options Analysis Iterative benefits gained from using Agile Methods Future benefits are increased because of risks Minimally, benefits should exceed costs


YearsRateeCostsdNBenefitsdN 21

d1 = [ln(Benefits Costs) + (Rate + 0.5 Risk2) Years] Risk Years d2 = d1 Risk Years

49

Agenda


Models of Business ValueMeasures of Business ValueEstimation of Business Value


50

Software Lifecycle Costs 1:10:100 is a classical ratio of dev. to maint. hours Defects have negative multiplicative effect on cost A conservative and contemporary ratio is 1:6:30

Boehm, B. W. (1981). Software engineering economics. Englewood Cliffs, NJ: Prentice-Hall.Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.

200

100

50

20

10

5

2

1

Phase in Which Error was Detected and Corrected

Re

lati

ve

Co

st

t o F

ix E

r ro

r

Requirements Design CodeDevelopment

TestAcceptance

TestOperation

51

Software Cost Models Cost estimation models still in use today Used to estimate effort of Traditional Methods Adjusted average of 5,088 used for ROI estimation

Benediktsson, O., & Dalcher, D. (2005). Estimating size in incremental software development projects. Journal of Engineering Manufacture, 152(6), 253-259.

COCOMO-O

COCOMO-S

COCOMO-E

COCOMO-II

Walston-Felix

Bailey-Basili

Doty

Source

Average

Months 2.4 KLOC 1.05





Months 5.5 + 0.73 KLOC 1.15


Model10,000

10,000

10,000

10,000

10,000

10,000

10,000

LOC26.93

39.55

57.06

36.51

42.27

15.81

58.92

39.58

Months4,667.60

6,854.94

9,889.73

6,328.20

7,326.31

2,740.66

10,213.48

6,860.13

Hours2.24

3.30

4.75

3.04

3.52

1.32

4.91

3.30

Years

* (6,854.94 + 7,326.31 + 2,740.66 + 10,213.48) 4 0.75 5,087.89

52

Total Lifecycle Costs 0.51 hours/line of code for Traditional Methods 10% defect inject rate (1,000 defects/10 KLOC) 67% of defects in test (33% in maintenance)

Rico, D. F. (2004). ROI of software process improvement: Metrics for project managers and software engineers. Boca Raton, FL: J. Ross Publishing.In, H. P., et al. (2006). A quality-based cost estimation model for the product line life cycle. Communications of the ACM, 49(12), 85-88.McCann, B. (2007). The relative cost of interchanging, adding, or dropping quality practices. Crosstalk, 20(6), 25-28.

Step1.

2.

3.

4.

5.

6.

7.

Total Lifecycle Cost Model0.51 Size + 30 IR Size – 30 IH – 5.47 TH + IH + TH

0.51 Size + 30 10% Size – 30 IH – 5.47 TH + IH + TH

0.51 Size + 3 Size – 30 IH – 5.47 TH + IH + TH

0.51 Size + 3 Size – 29 IH – 4.47 TH

3.51 Size – 29 IH – 4.47 TH

3.51 10,000 – 0 – 4.47 3,651.48

18,751.48* hours or $1,875,148* 5,087.89 Development Hours + 3,651.48 Test Hours + 10,012.11 Maintenance Hours

53

Agile Productivity Studies Productivity data found in 26 Agile Methods studies Studies conducted from 2002 to the present time Average productivity 21.2374 LOC per hour


No. Author(s)1. Abrahamsson 2003 XP Case Study 19.2550

2. Abrahamsson & Koskela 2004 XP Case Study 16.9000

3. Back, Hirkman, & Milovanov 2004 XP Experiment 8.0000

24. Cohn 2008 Scrum Case Study 5.9050

25. Jones 2008 Scrum Case Study 5.7400

26. Sutherland 2006 Scrum Case Study 4.6858

Average 21.2374

Year Method Type LOC/Hour

54

Agile Productivity Models Based on 13 studies of Extreme Programming (XP) Also based on 7 studies of pair programming (PP) “Pair programming” had highest productivity


1. XP 13

2. TDD 2

3. PP 8

4. Scrum 3

5. Agile 26

No. Method Pts.

3.5000

12.3800

15.4667

4.6858

3.5000

Low

16.1575

29.2800

33.4044

5.4436

21.2374

Median

43.0000

46.1800

86.4502

5.9050

86.4502

High

LOC 16.1575

LOC 29.2800

LOC 33.4044

LOC 05.4436

LOC 21.2374

Productivity

55

Agile Quality Studies Defect density found in 21 studies of Agile Methods Studies conducted from 2002 to the present time Average quality 1.7972 defects per KLOC


No. Author(s)1. Abrahamsson 2003 XP Case Study 2.1450

2. Abrahamsson & Koskela 2004 XP Case Study 1.4300

3. Back, Hirkman, & Milovanov 2004 XP Experiment 0.7000

19. Cohn 2008 Scrum Case Study 2.9000

20. Jones 2008 Scrum Case Study 8.5000

21. Schatz & Abdelshafi 2005 Scrum Case Study 0.4350

Average 1.7972

Year Method Type Def/KLOC

56

Quality Models Based on 10 studies of Extreme Programming (XP) Also based on 6 studies of pair programming (PP) “Extreme Programming” had the highest quality


1. XP 10

2. TDD 2

3. PP 6

4. Scrum 3

5. Agile 21

No. Method Pts.

0.0032

0.6100

0.3250

0.4350

0.0032

Low

0.7466

2.1550

2.3550

3.9450

1.7972

Median

2.1450

3.7000

5.8500

8.5000

8.5000

High

0.7466 KLOC 30

2.1550 KLOC 30

2.3550 KLOC 30

3.9450 KLOC 30

1.7972 K LOC 30

Quality

57

Agenda


Measures of Business ValueModels of Business Value

Estimation of Business ValueComparison of Business Value

Summary of Business Value

58

Agile Lifecycle Costs Costs based on productivity and quality models Development costs based on LOC productivity rate Maintenance costs based on defects KLOC MH


1. XP

2. TDD

3. PP

4. Scrum

5. Agile

No. Method Agile Lifecycle Cost Models

(10,000 05.3858 + 0.7466 10 30) 100

(10,000 09.7600 + 2.1550 10 30) 100

(10,000 11.1350 + 2.3550 10 30) 100

(10,000 05.4436 + 3.9450 10 30) 100

(10,000 07.9311 + 1.7972 10 30) 100

$167,109

$160,459

$302,052

$180,002

Costs

$208,069

* XP, TDD, and PP reduced by two-thirds to moderate effects of laboratory conditions

59

Agile Lifecycle Benefits Benefits based on total traditional less agile costs Traditional costs based LOC dev. maint. effort Traditional costs credited testing effort applied


1. XP

2. TDD

3. PP

4. Scrum

5. Agile

No. Method

$1,708,039

$1,714,690

$1,573,096

$1,695,146

Benefits

$1,667,079

Agile Lifecycle Benefit Models

(10,000 3.51 – 3,651.48 4.47) 100 – $208,069

(10,000 3.51 – 3,651.48 4.47) 100 – $167,109

(10,000 3.51 – 3,651.48 4.47) 100 – $160,459

(10,000 3.51 – 3,651.48 4.47) 100 – $302,052

(10,000 3.51 – 3,651.48 4.47) 100 – $180,002

60

Extreme Programming Costs based on avg. productivity and quality Productivity moderated from 16.1575 to 5.3858 Costs were $208,069, benefits were $1,667,079


Formula

(10,000 5.3858 0.7466 10 30) 100

$1,667,079 $208,069

($1,667,079 – $208,069) $208,069 100%

($1,667,079 5) 1.055) – $208,069

$208,069 ($1,875,148 $208,069 – 1)

NORMSDIST (2.24) $1,667,079 –NORMSDIST (0.59) $208,069 EXP (–5% 5)

(10,000 3.51 – 3,651.48 4.47) 100 – $208,069

Value

$208,069

8:1

701%

$1,235,446

$10,064

$1,529,066

$1,667,079

Metric

Costs

B/CR

ROI

NPV

BEP

ROA

Benefits

5

1i(

61

Test Driven Development Costs based on avg. productivity and quality Productivity moderated from 29.2800 to 9.7600 Costs were $167,109, benefits were $1,708,039


Formula

(10,000 9.7600 + 2.1550 10 30) 100

$1,708,039 $167,109

($1,708,039 – $167,109) $167,109 100%

($1,708,039 5) 1.055) – $167,109

$167,109 ($1,875,148 $167,109 – 1)


(10,000 3.51 – 3,651.48 4.47) 100 – $167,109

Value

$167,109

10:1

922%

$1,311,874

$6,430

$1,580,490

$1,708,039

Metric

Costs

B/CR

ROI

NPV

BEP

ROA

Benefits

5

1i(

62

Pair Programming Costs based on avg. productivity and quality Productivity moderated from 33.4044 to 11.135 Costs were $160,459, benefits were $1,714,690


Formula

(10,000 11.1350 + 2.3550 10 30) 100

$1,714,690 $160,459

($1,714,690 – $160,459) $160,459 100%

($1,714,690 5) 1.055) – $160,459

$160,459 ($1,875,148 $160,459 – 1)


(10,000 3.51 – 3,651.48 4.47) 100 – $160,459

Value

$160,459

11:1

969%

$1,324,283

$5,919

$1,590,927

$1,714,690

Metric

Costs

B/CR

ROI

NPV

BEP

ROA

Benefits

5

1i(

63

Scrum Costs based on avg. productivity and quality Productivity data remained the same at 5.4436 Costs were $302,052, benefits were $1,573,096


Formula

(10,000 5.4436 + 3.9450 10 30) 100

$1,573,096 $302,052

($1,573,096 – $302,052) $302,052 100%

($1,573,096 5) 1.055) – $302,052

$302,052 ($1,875,148 $302,052 – 1)

NORMSDIST (1.97) $1,573,096 –NORMSDIST (-0.27) $302,052 EXP (–5% 5)

(10,000 3.51 – 3,651.48 4.47) 100 – $302,052

Value

$302,052

5:1

421%

$1,060,084

$21,682

$1,441,741

$1,573,096

Metric

Costs

B/CR

ROI

NPV

BEP

ROA

Benefits

5

1i(

64

Agile Methods Costs based on avg. productivity and quality Productivity data resulted in average of 7.9311 Costs were $180,002, benefits were $1,695,146


Formula

(10,000 7.9311 + 1.7972 10 30) 100

$1,695,146 $180,002

($1,695,146 – $180,002) $180,002 100%

($1,695,146 5) 1.055) – $180,002

$180,002 ($1,875,148 $180,002 – 1)


(10,000 3.51 – 3,651.48 4.47) 100 – $180,002

Value

$180,002

9:1

842%

$1,287,817

$7,483

$1,562,126

$1,695,146

Metric

Costs

B/CR

ROI

NPV

BEP

ROA

Benefits

5

1i(

65

Agenda



Comparison of Business ValueSummary of Business Value

66

Data for Agile Methods Agile Methods were ranked based on ROI Agile Methods with high quality had lower ROI Agile Methods with high productivity had high ROI

Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? Retrieved September 3, 2008, from http://davidfrico.com/agile-benefits.xls

Method Costs Benefits B/CR ROI NPV BEP ROA

PP $160,459 $1,714,690 11:1 969% $1,324,283 $5,919 $1,590,927

TDD $167,109 $1,708,039 10:1 922% $1,311,874 $6,430 $1,580,490

Agile $180,002 $1,695,146 9:1 842% $1,287,817 $7,483 $1,562,126

XP $208,069 $1,667,079 8:1 701% $1,235,446 $10,064 $1,529,066

Scrum $302,052 $1,573,096 5:1 421% $1,060,084 $21,682 $1,441,741

67

ROI of Agile Methods Agile Methods were ordered based on ROI Agile Methods had a high ROI value of 969% Agile Methods yielded an average ROI of 842%


842%

701%

421%

0%

200%

400%

600%

800%

1,000%

PP TDD Agile XP Scrum

922%969%

68

Data for Traditional Methods Traditional Methods were ranked based on ROI Methods with good cost and quality had higher ROI Agile Methods had better ROI than heaviest methods


Method Costs Benefits B/CR ROI NPV BEP ROA

PSP sm $105,600 $1,755,148 17:1 1,562% $1,414,174 $945 $1,672,907

TSP sm $148,400 $1,706,648 12:1 1,050% $1,329,379 $5,760 $1,591,127

Inspections $82,073 $897,499 11:1 994% $695,067 $51,677 $833,681

Agile $180,002 $1,695,146 9:1 842% $1,287,817 $7,483 $1,556,997

SW-CMM® $311,433 $1,153,099 4:1 270% $687,030 $153,182 $998,013

ISO 9001 $173,000 $566,844 3:1 228% $317,828 $1,196,206 $486,750

CMMI® $1,108,233 $1,153,099 1:1 4% -$109,770 $545,099 $891,412

69

ROI of Traditional Methods Traditional Methods were ordered using ROI Traditional Methods had high ROI value of 1,562% Agile Methods had better ROI than heaviest methods


1,562%

1,050%994%

842%

270% 228%

4%0%

300%

600%

900%

1,200%

1,500%

1,800%

PSPsm TSPsm Inspections Agile SW-CMM® ISO 9001 CMMI®

70

Data for All Methods Software methods were ranked based on ROI Methods with good cost and quality had best ROI Best Agile and Traditional Methods had similar ROI


Traditional

Traditional

Agile

Traditional

Agile

Agile

Traditional

Agile

Traditional

Traditional

Type Costs Benefits B/CR ROI NPV BEP ROAMethod

Agile

PSP sm $105,600 $1,755,148 17:1 1,562% $1,414,174 $945 $1,672,907

TSP sm $148,400 $1,706,648 12:1 1,050% $1,329,379 $5,760 $1,591,127

Inspections $82,073 $897,499 11:1 994% $695,067 $51,677 $833,681

PP $160,459 $1,714,690 11:1 969% $1,324,283 $5,919 $1,590,034

TDD $167,109 $1,708,039 10:1 922% $1,311,874 $6,430 $1,578,575

Agile $180,002 $1,695,146 9:1 842% $1,287,817 $7,483 $1,556,997

XP $208,069 $1,667,079 8:1 701% $1,235,446 $10,064 $1,513,332

Scrum $302,052 $1,573,096 5:1 421% $1,060,084 $21,682 $1,389,810

SW-CMM® $311,433 $1,153,099 4:1 270% $687,030 $153,182 $998,013

ISO 9001 $173,000 $566,844 3:1 228% $317,828 $1,196,206 $486,750

CMMI® $1,108,233 $1,153,099 1:1 4% -$109,770 $545,099 $891,412

71

ROI of All Methods Software methods were ordered by ROI Agile Methods had a high ROI value of 969% Traditional Methods had high ROI value of 1,562%


1,562%

1,050%994% 969%

922%842%

701%

421%

270% 228%

4%0%

300%

600%

900%

1,200%

1,500%

1,800%

PSPsm TSPsm Inspections PP TDD Agile XP Scrum CMM® ISO 9001 CMMI®

72

Unadjusted ROI of All Methods Are data based on unrealistic laboratory conditions? Are productivity data from lab studies optimistic? Are total lifecycle costs closer to 1:10:100?

4,133%

3,272%3,103%

2,826%

1,788%1,607% 1,499%

871%

580%

229% 173%

0%

500%

1,000%

1,500%

2,000%

2,500%

3,000%

3,500%

4,000%

4,500%

PSPsm Inspect XP TSPsm Agile TDD PP SW-CMM® Scrum ISO 9001 CMMI®


73

Agenda


Measures of Business ValueModels of Business ValueEstimation of Business ValueComparison of Business Value

Summary of Business Value

74

Benefit Summary Agile (138 pt.) and Traditional Methods (99 pt.) Agile Methods fare better in all benefits categories Agile Methods 459% better than Traditional Methods

Agile Methods Traditional MethodsLow Median HighCategory

ROI 240% 2,633% 8,852%

Satisfaction 70% 70% 70%

Quality 10% 70% 1,000%

Productivity 14% 122% 712%

Schedule 11% 71% 700%

Cost 10% 26% 70%

Low Median HighCategory

ROI 200% 470% 2,770%

Satisfaction -4% 14% 55%

Quality 7% 50% 132%

Productivity 9% 62% 255%

Schedule 2% 37% 90%

Cost 3% 20% 87%


75

Cost of Quality Apply traditional reliability and quality theory Defects are inexpensive to remove early in cycle Late bug removal has negative, multiplicative effect

Rico, D. F. (2000). Using cost benefit analyses to develop software process improvement (SPI) strategies. Rome, NY: DACS.

Inspection Cost (57X PSP)

Softw

are

Defe

cts

PSP

Test Cost (138X PSP)

Ad Hoc (326X)

Analysis Design CodeUnitTest

System Test

ComponentTest

CustomerUse

PSP Cost (326X lower than Ad Hoc)

76

Real Options NPV models losses of Traditional Methods Real options model profits from Agile Methods Agile Methods incur less initial risk and higher ROI

Fichman, R. G., Keil, M., & Tiwana, A. (2005). Beyond valuation: Options thinking in IT project management. California Management Review, 47(2), 74-96.

0.00

0.05

0.10

0.15

0.20

0.25

- $9M - $7M - $5M - $3M - $1M $1M $3M $5M $7M $9M

Prob

abili

ty

NPV of Traditional Methods = $1.1 Million

0.00

0.05

0.10

0.15

0.20

0.25

- $9M - $7M - $5M - $3M - $1M $1M $3M $5M $7M $9M

Prob

abili

ty

Real Options of Agile Methods = $1.4 Million

77

Agile vs. Traditional Metrics Agile Methods are a fundamentally new paradigm Agile Methods are “not” lighter Traditional Methods They should not be viewed through a Traditional lens

Rico, D. F. (2009). Metrics for agile methods. Retrieved February 7, 2009, from http://davidfrico.com/agile-metrics.pdf

Customer Interaction

Iterative Development


Adaptability or Flexibility

valuedmore than

valuedmore than

valuedmore than

valuedmore than

Agile

Met

rics

Traditional Metrics

Contracts

Documentation

Processes

Project P lans

· Interaction frequency· Communication quality· Strength of Relationship

· Customer trust· Customer loyalty· Customer satisfaction

· Team skills· Team motivation· Team cooperation

· Team trust· Team cohesion· Team communications

· Iteration size· Iteration number· Iteration frequency

· Productivity· Defect density· Software reliability

· Organizational flexibility· Management flexibility· Developer flexibility

· Process flexibility· Design flexibility· Technology flexibility

· Fixed-fee contracts· Engineering changes· Contract change orders

· Standards compliance· Process Maturity Level· Manufacturing practices

· Document standard· Lifecycle/phase reviews· Quality assurance audits

· Cost Compliance· Scope Compliance· Schedule Compliance

78

New Book Guide to Agile Methods for business leaders Communicates business value of Agile Methods Rosetta stone to Agile Methods for Traditional folks

* Rosetta stone to the business value and culture of Agile Methods for executives, managers, and thought leaders in the field of software methods.

THE BUSINESS VALUE OF AGILE METHODS

Maximizing ROI with Right-Sized, Just-Enough, and Just-in-Time Processes and Documentation

DR. DAVID F. RICO, DR. HASAN H. SAYANI

AND DR. SAYA SONE

Forward by Dr. Jeffrey V. Sutherland

Table of Contents

1. Introduction2. Values of Agile Methods3. History of Agile Methods4. Antecedents of Agile Methods5. Types of Agile Methods6. Practices of Agile Methods7. Agile Project Management8. Agile Software Engineering9. Agile Support Processes10. Agile Tools and Technologies11. Comparison of Agile Methods12. Agile Metrics and Models13. Costs of Agile Methods14. Benefits of Agile Methods15. ROI of Agile Methods16. NPV of Agile Methods17. Real Options of Agile Methods18. Conclusion

79

References Rico, D. F. (2000). Using cost benefit analyses to develop software process improvement (SPI)

strategies. Rome, NY: DACS.

Rico, D. F. (2002). How to Estimate ROI for Inspections, PSP, TSP, SW-CMM, ISO 9001, and CMMI. Software Tech News, 5(4), 23-31.

Rico, D. F. (2002). The Return on investment in quality. TickIT International, 4(4), 13-18.

Rico, D. F. (2004). ROI of software process improvement: Metrics for project managers and software engineers. Boca Raton, FL: J. Ross Publishing.

Rico, D. F. (2005). Practical metrics and models for return on investment. TickIT International, 7(2), 10-16.

Rico, D. F. (2006). A framework for measuring the ROI of enterprise architecture. International Journal of End User Computing, 18(2), 1-12.

Rico, D. F. (2007). Optimizing the ROI of enterprise architecture using real options. In S. Clarke (Ed.), End user computing challenges and technologies: Emerging tools and applications. Hershey, PA: Information Science Reference.

Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? An analysis of extreme programming, test-driven development, pair programming, and scrum (using real options). TickIT International, 10(4), 9-18.

Solingen, R. A., & Rico, D. F. (2006). Calculating software process improvement’s return on investment. In M. V. Zelkowitz (Ed.), Advances in computers: Quality software development, Vol. 66 (pp. 1-41). San Diego, CA: Elsevier.

80

Contact Information Website: http://davidfrico.com Biography: http://www.linkedin.com/in/davidfrico Capabilities: http://davidfrico.com/rico-capability.pdf

business value of agile methods using return on investment dr. david f. rico, pmp, csm

Documents

business valuealternative

skillsbusiness value

enterprisebusiness value

software methods

agile methodsrooted

longrunbusiness value

customer requirements

use of adaptability