agile vs. industry - path to agility conference · agile assessment — staffing build phase...

Copyright QSM Associates, Inc.

@MichaelCMahManaging Partner

QSM Associates, Inc.75 South Church Street

Pittsfield, MA 0120100 1 413-499-0988

e-mail: [email protected]: www.qsma.com

Blog: www.optimalfriction.com

The Path To a*gil*i*tyCentral Ohio Agile Association

May 27, 2010

Agile vs. Industry: How Does Agile Compare?

mailto:[email protected]

http://www.qsma.com/

http://www.optimalfriction.com/


Are You Agile or Not?


“Frothy eloquence neither convinces nor satisfies me. I am from Missouri. You have got to show me.”

- Missouri Congressman Willard Duncan Vandiver, 1899


Who We Are:QSM maintains the world’s largest benchmarking database of 8,000+ completed software projects collected worldwide. We put industryproductivity statistics on the desktop.The QSM SLIM database contains projects in all industries, waterfall, Agile, offshore/outsourced, in-house, new development, and maintenance.SLIM tools enable managers to measure and estimate Agile and/or waterfall projects, and determine ROI.


Partial List of Clients

British TelecomSAP MicrosoftIntel AT&T/BellSouth BMC Software Bank of New York MellonMotorola VerizonWirelessRoche Diagnostics

Fiserv CorpIBM GlobalMisys HealthcareJPMorganChaseBoeing

Lockheed MartinProgressive InsuranceDirecTV

13Copyright QSM Associates, Inc.

Co-Located XP - Follett SoftwareTeam size

24 Developers7 Testers3 Customers3 Project Leaders

Code Base1,000,000 lines of code7,000 automated unit test10,000 automated acceptance test


3 - DETERMINE PROCESSMETRICS & PROJECT POSITIONING

1 - COLLECT ANDVALIDATE PROJECT DATA

2 - ANALYZE PROJECTS USINGQSM REFERENCE DATABASE

4 - PRESENT RESULTS

Agile Release Retrospectives


The S

LIM-S

uite

SLIM-Estimate:

Size, Schedule, Cost & Quality

Estimating

SLIM-DataManagerSoftware Project Metrics Repository

SLIM-Control:Variance Analysis

&Adaptive

ForecastingSLIM-Metrics:

Industry Benchmarking &

Process Improvement

SLIM-MasterPlan: Incremental

Development & Project

AggregationThe SLIM-Suite


Four Core Metrics: Your HistoryDuration

Effort

DiscoveredDefects

ProducedSoftware

(Size)

How long?

How much?

How good?


Input to SLIM

Size

Time

Defects

Effort


SLIM Replica – Destiny 5.0Staffing & Probability Analysis

Avg Staff (people)<Destiny Release 5.0>

1 2 3 4 5 6 7 8 9Oct'03

Nov Dec Jan'04

Feb Mar Apr May Jun Jul Aug Sep

0

5

10

15

20

25

30

Avg Staff (people)

876543 21

STORYBUILD

Milestones 0 - CSR 1 - SRR 2 - HLDR 3 - LLDR 4 - CUT 5 - IC 6 - STC 7 - UAT 8 - FCR 9 - 99R 10 - 99.9R


SOLUTION PANEL - <Destiny Release 5.0>

DurationEffortCost

Peak StaffMTTD

Start Date

BUILD6.5151

1283.523.2

0.6751/5/2004

Life Cycle7.2162

1377.223.2

0.67512/15/2003

MonthsPM$ (K)peopleDays

PI=23.1 MBI=5.6 Eff SLOC=218,531

CONTROL PANEL - <Destiny Release 5.0>

PI 18.5 27.7

23.1

Peak Staff 18.5 27.8

23.2

Eff SLOC (K) 175 262

219

How Does This ReleaseCompare Against Industry Data?


Industry Average

Current Performance

Delta

Project Cost $3.5 Million $2.2 Million -$1.3M

Schedule 12.6 months 7.8 months -4.8 mos

QA Defects 242 121 -50%

Staffing 35 35 n/a

Follett vs. Industry Average


Industry Average

Offshore Average

Delta

Project Cost $3.5 Million $3.2 Million -$0.3M

Schedule 12.6 months 9.6 months -3.0 mos

QA Defects 242 677 +280%

Staffing 35 50 +15

* Normalized to same project size

Offshore vs. Industry Average


Defects Found

6 12 18 24 30 36 42 48 54 60 66 72 78 84 9005/17

'0306/28 08/09 09/20 11/01 12/13 01/24

'0403/06 04/17 05/29 07/10 08/21 10/02 11/13 12/25 02/05

'05

0

50

100

150

200

250

300

350

Defects

109876543101098765

Current Plan Actuals Green Control Bound Yellow Control Bound Project: CEP

Offshore Case Study Defects vs. Industry Average


Defects Found

6 12 18 24 30 36 42 48 54 60 66 72 78 84 9005/17

'0306/28 08/09 09/20 11/01 12/13 01/24

'0403/06 04/17 05/29 07/10 08/21 10/02 11/13 12/25 02/05

'05

0

100

200

300

400

500D

efects

109876543101098765

Current Plan Actuals Green Control Bound Yellow Control Bound Project: CEP

Scaled Up Defect Curve – Outsourced Project


Trendline Assessment – Build Phase ScheduleMain Build Phase Duration vs Size

100 1,000Effective SLOC (thousands)

1

10

100

Time (M

onths)Rel 5.0

Rel 6.0

Rel 6.5

Rel 7.5Rel 7.0

Rel 8.0Rel 5.0

Rel 6.0

Rel 6.5

Rel 7.5Rel 7.0

Rel 8.0

Business Sy stems Av ionic Sy stems Command & Control Microcode Sy stems Process Control QSM 2005 BusinessAv g. Line Sty le 1 Sigma Line Sty le

Schedules are Half Industry


Trendline Assessment – Defects/QualityDefects During Test

100 1,000Effective SLOC (thousands)

10

100

1,000

10,000

Errors (S

ysInt-Del)

Rel 5.0

Rel 6.0 Rel 6.5Rel 7.5 Rel 7.0

Rel 8.0

Rel 5.0

Rel 6.0 Rel 6.5Rel 7.5 Rel 7.0

Rel 8.0

Business Sy stems Av ionic Sy stems Command & Control Microcode Sy stems Process Control QSM 2005 BusinessAv g. Line Sty le 1 Sigma Line Sty le

Far Fewer Defects: 50% - 66% Below Industry

34Copyright QSM Associates, Inc.

Follett and XP: It has worked incredibly well…Destiny Library Manager:

Award of Excellence 2004, presented by Technology and Learning magazine (December 2004). Awards Portfolio 2004, presented by Media and Methodsmagazine (May/June 2004).Technology & Learning Award of Excellence 2006, 2007

Destiny Textbook ManagerAwards Portfolio 2005, presented by Media and Methodsmagazine (May/June 2005).Technology & Learning Award of Excellence 2007

Destiny Enriched ServicesTechnology & Learning Award of Excellence 2007

Follett Software provides Library Automation Solutions to 52% of the K12 market. Destiny Library Manager: Single largest product market share in K12 with 19% of the total market and continues to outpace the competition in market growth.


Domain Knowledge

Smart people, experienced peopleCoding is moving knowledge from mind into the machineInexperience costs money


Short Feedback Loops

Paired programmersInstantaneous code reviewsAccelerated learning and executionFace to face communication channel


Transparency

The best teams hide nothingRelease PlanningDaily stand-ups/scrum sessionsIteration DemosVelocity and burn-down charts


Distributed SCRUM – BMC Software


Whiteboard Sketch – Performance Mgr R2.3


Release 2.3 Defect Rate

0

20

40

60

80

100

120

140

160

Defect Type (All)

Count of Severity*

Create Date Status Mode Status Severity* TR-Version

Product+*


Input to SLIM-DataManager

Size

Time

Defects

Effort

Size

Time


SLIM Replica – Rel 2.3Staffing & Probability Analysis

Avg Staff (people)<Performance Manager Rel 2.3>

1 2 3 4 5 6 7 8 9Apr'06

May Jun Jul Aug Sep Oct Nov Dec Jan'07

Feb Mar

0

20

40

60

80

100

120

Avg Staff (people)

109876543 21

R&DC&TP_Mnt



SOLUTION PANEL - <Performance Manager Rel 2.3>

DurationEffortCost

Peak StaffMTTD

Start Date

C&T5.3488

4880.092.8

0.1047/2/2006

Life Cycle7.0556

5561.292.8

0.2326/1/2006

MonthsPM$ (K)peopleDays

PI=28.3 MBI=8.3 Eff SLOC=844,710

CONTROL PANEL - <Performance Manager Rel 2.3>

PI 22.6 33.9

28.3

Peak Staff 74.2 111.3

92.8

Eff SLOC (K) 676 1014

845

How Does This ReleaseCompare Against Industry Data?


Industry Average

Current Performance

Delta

Project Cost $5.5 Million $5.2 Million -$.3M

Schedule 15 months 6.3 months -8.7 mos

QA Defects 713 635 -11%

Staffing 40 92 +52

BMC vs. Industry Average


Agile Assessment — StaffingBUILD Phase Staff ing

10 100 1,000STORIES (thousands)

1

10

100

1,000

C&T Peak Staff (People)

BMC Rel 2.4BMC Rel 2.3


Agile Companies Company B SCRUM Company A - Agile XP QSM 2005 Business Avg. Line Style1 Sigma Line Style

Agile Projects’ team sizes are fairly typicalBMC (red) uses large teams. Follett (green)uses average sized teams.

Larger Teams


Agile Assessment — ScheduleBUILD Phase Schedule


1

10

100C&T D

uration (Months)




BMC and Follett projects are very fast

Faster Schedules


Agile Assessment – QualityBugs


1

10

100

1,000

10,000

Errors (SysInt-Del)




BMC and Follett Bugs are low

Fewer Defects


BMC “Secret Sauce”


BMC “Secret Sauce” (con’t)Buy-In

VP-Level (or higher) Senior Executive SponsorshipScrum Master TrainingCore Group Energized and Passionate

Staying “Releasable”Nightly Builds/Test2-week Iteration DemosFrequent, Rigorous Peer Code Review

Dusk-to-Dawn TeamworkCommunication Techniques for Information FlowWikis, Video-conferencing, Periodic On-Site MeetingsCo-Located Release PlanningScrum of Scrum Meetings (US Time)


BMC “Secret Sauce” (con’t)Backlogs

One Master Backlog AND Multiple Backlog ManagementOne Setup for User Stories Across TeamsAdded “Requirements Architect” to Interface Product Mgt with R&D

“Holding Back the Waterfall”Test Driven DevelopmentRetrospective Meetings to Not Regress into old Waterfall HabitsOutside Source to Audit the Process


Agile at Quick Solutions


Agile Assessment — StaffingAverage Staff

1 10 100 1,000

New + Modified Size (thousands)

1

10

100

1,000H

eadcount

All Systems Special Project QSM 2008 Business Avg. Line Style 1 Sigma Line Style

Smaller Teams


Time-to-Market

1 10 100 1,000


1

10

100M

onths


Faster Schedules

Agile Assessment — Schedule


Agile Assessment – QualityBugs During QA

1 10 100 1,000


1

10

100

1,000

10,000

Defects


Fewer Defects


Agile Database Creates Agile Trends


New Agile Benchmark Trends


Get The Cutter Report by Contacting Me:

Michael Mahemail: [email protected]: www.qsma.comblog: www.optimalfriction.comtwitter: @michaelcmahTel: 1 413-499-0988

mailto:[email protected]

http://www.qsma.com/

http://www.optimalfriction.com/


www.qsma.com


Who We Are:QSM maintains the world’s largest benchmarking database of 8,000+ completed software projects collected worldwide. We put industryproductivity statistics on the desktop.The QSM SLIM database contains projects in all industries, waterfall, Agile, offshore/outsourced, in-house, new development, and maintenance.SLIM tools enable managers to measure and estimate Agile and/or waterfall projects, and determine ROI.


Industry Data from the QSM SLIM-Metrics Database

Spans 20+ yearsLarge, worldwide heterogeneous database contains over 8,000+ projectsRepresents over 685+ million SLOC, 7+ million function points, over 600 languages, from 500+ organizations in 18 countriesAdding 200 – 400 projects/year


Partial List of Clients

British TelecomSAP MicrosoftIntel AT&T/BellSouth BMC Software Bank of New York MellonMotorola VerizonWirelessRoche Diagnostics

Fiserv CorpIBM GlobalMisys HealthcareJPMorganChaseBoeing

Lockheed MartinProgressive InsuranceDirecTV


The S

LIM-S

uite

SLIM-Estimate:

Size, Schedule, Cost & Quality

Estimating

SLIM-DataManagerSoftware Project Metrics Repository

SLIM-Control:Variance Analysis

&Adaptive

ForecastingSLIM-Metrics:

Industry Benchmarking &

Process Improvement

SLIM-MasterPlan: Incremental

Development & Project

AggregationThe SLIM-Suite

agile vs. industry - path to agility conference · agile assessment — staffing build phase...

Documents