agile metrics for senior managers and executives
DESCRIPTION
In this webinar, find out about agile appropriate metrics at the customer, portfolio and project levels. Presented by LitheSpeed, LLC. Want to check out the full webinar? Visit http://pm.versionone.com/Webinar_MetricsExecs.htmlTRANSCRIPT
Agile Metrics for Senior
Managers & ExecutivesManagers & ExecutivesSanjiv Augustine and Roland Cuellar, LitheSpeed
Annandale, VAApril 15, 2009
1. Conceptual Background
• Defining Key Metrics
• Metrics as Indicators
• The Balanced Scorecard
2. Agile Balanced Scorecard
• Sample Scorecard
Agenda
• Sample Scorecard
• Product: Business Value Delivered
• Team: Customer Satisfaction
• Financial: Net Present Value
• Schedule: Earned Value for Agile
• Other Leading Indicators
3. Q&A
22
• Beware the Hawthorne Effect
o We affect what we measure
• Knowledge workers need to first define:
o What is our business?
Who is our customer?
Defining Key Metrics
o Who is our customer?
o What does our customer consider valuable?
33
Drucker: Measures of quality must flow from the above.
• Business Metrics are often Lagging Indicators
o They often measure what happened in the last financial period
o Do not always give a forward view into what is to come and how
to make corrections
• Managers need Leading Indicators to help keep projects on track
o Are we delivering according to schedule and budget?
o Are we practicing the appropriate agile methods that will lead to
success?
Metrics Frameworks
success?
o Are we being efficient with resources?
• Effectiveness Metrics
o Measures the extent to which we met our business goals
o Tend to be business metrics such as revenue generated, cost
savings, new customer acquisition, customer retention, etc.
• Efficiency Metrics
o Measures the efficiency of our solution in terms of cost, time,
resources, etc
44
The metrics that you
choose may need to
cover all 4 facets.
The Balanced Scorecard
CustomerCustomer Satisfaction / NPS
FinancialCost per Unit
Project Savings
Activity Based Costing
Innovation & LearningLessons Learned
55
Customer Satisfaction / NPS
Customer Acquisition & Retention
Quality
Lessons Learned
Training
% Revenue from NPD
Internal ProcessesCycle Times
Volumes Shipped
On Time Delivery Kaplan, Norton
Sample Agile Balanced Scorecard
ScheduleEVA
Product Business Value Delivered
Remaining Feature Backlog
Quality
TeamCustomer & Associate Satisfaction
66
EVA
Release Burndown
Days per Release
Customer & Associate Satisfaction
Iteration Velocity Variance
Audit Pass Rate
FinancialNPV
Cost Savings or Revenue
Dollars per Release
Fast account registration process
.4
User account registration Account processing
Product: Business Value
• Goal represents 40% of business case
• “User account registration” Feature represents
70% of Goal value
• “Enter billing information” User Story realizes
60% of Feature value
• % Value Delivered= .4 x .7 x .6 = 16.8%
Go
al
Fe
atu
re
User account registration
.7
As a User, I can enter personal information.
.4
As a User, I can enter billing information.
.6
Account processing
.3
As an Account Processor, I can activate a User account.
1
77
Fe
atu
reU
ser
Sto
ry
Landmark Honda: 6x
HondaPresident’s Award
winner:
• “…demonstrate superior achievement
in customer satisfaction, new-car-
unit sales volume and business
management.”
Team: Customer Satisfaction
• “…it is the highest award that a Honda
dealership can receive because it
demands the highest personal
investment and stimulates the
greatest results – satisfied, loyal
customers.”
88
“Although some of them seem to rate our entire operation, in fact they all rate my
personal performance and the scores become part of my permanent record.”
Financial: Net Present Value
• C = net cash in period t
• R = cost of capital
• NPV > 0; accept the project
• NPV < 0; decline the project
NPV:� Takes time value of money into account
� Takes cost of capital into account
� Values projects same as stock
� Positive NPV adds value to the firm
� Leads to better investment decisions
than other criteria such as IRR, ROI*
NPV = ∑ Ct
(1+r)t
99
• NPV < 0; decline the project
* From “Principles of Corporate Finance” by Brealey &
Myers
Leading and Lagging:� For leading indicators of success, measure the assumptions that went into
the model
� For lagging indicators, measure actual investment performance
Schedule: Earned Value for Agile
• Does not really measure business value; measures cost delivery
• Tries to measure both schedule performance and cost performance against plan
EVM Definition Calculation
PPC Planned % Complete Iteration # / Total Iterations
APC Actual % Complete Points Delivered / Total Points
1010
APC Actual % Complete Points Delivered / Total Points
PV Planned Value PPC * BAC (budget at completion)
EV Earned Value APC * BAC
CPI Cost Perf Index EV / Actual Cost
SPI Schedule Perf Index EV / PV
Sulaiman, Barton, and Blackburn
Other Leading Indicators - Use With Caution!
• Change in velocity
• Change in Team Capacity
• Product Owner Time with Team
• Team co-location time
• Backlog growth
• Defect generation and resolution rates
They Want It All:�“I want it fast”
�“I want it cheap”
�“I want good quality”
�“It must pass governance”
1111
• Process QA Checklist–
o Are stand-ups happening?
o Are demos happening?
o Are retrospectives happening?
o Is working software delivered in every sprint?
o Is testing occurring in every sprint?
Sample Agile Balanced Scorecard Revisited
ScheduleEVA
Product Business Value Delivered
Remaining Feature Backlog
Quality
TeamCustomer & Associate Satisfaction
1212
EVA
Release Burndown
Days per Release
Customer & Associate Satisfaction
Iteration Velocity Variance
Audit Pass Rate
FinancialNPV
Cost Savings or Revenue
Dollars per Release
Contact Us for Further Information
Sanjiv Augustine
Roland Cuellar
Vice President
1313
Vice [email protected]
On the Web:
http://www.lithespeed.com
http://www.sanjivaugustine.com
"I only wish I had read this book when I started my career in
software product management, or even better yet, when I was
given my first project to manage. In addition to providing an
excellent handbook for managing with agile software development
methodologies, Managing Agile Projects offers a guide to more
effective project management in many business settings."
John P. Barnes, former Vice President of Product Management at
Emergis, Inc.