measuring product management/ownership effectiveness in a lean/agile world
DESCRIPTION
Effective Product Ownership is key to enabling an effective agile delivery. What goals should we set? How would we measure them?TRANSCRIPT
Metrics for the LEAN/Agile Product Manager
Yuval Yeret
Main things we want to pay attention to
• Performance of the Production floor – covered elsewhere (Simple KPIs Slides)
• Performance of the Product Management group: – Business Value–Wastes related to PM– Technical Debt
Business value
• We care about outcome – features delivered, adopted, used, paid for
• How can we measure this? • Manage a kanban at the high level features level,
that tracks when features are adopted, and upon first paying customer.
• Then see how much WIP of features not yet adopted we have, LEAd/Cycle time to adoption, features that we dropped on the way.
Debt
• A lot of time debt is taken due to PM decision
• We want to track how much debt we have, and take action to minimize it.
• E.g. we need to release ASN1 now, so we don’t “automate tests”/”code it correctly”, so every work on ASN1 is slower, until this is fixed
Tracking debt in kanban
• Have debt card type that is created when debt is taken on
• Track amount of debt versus overall WIP/Backlog
• See whether stable, improving, worsening trend
• Decide on policy for dealing with debt – WIP Limit, etc.
• Track the cycle time and WIP for debt cards to see whether they get the SLA they deserve
Wastes related to PM
• Waiting for PM• PM related Churn / Context switching /
Expediting• Sunk Costs• Rework due to late feedback by PM
Waiting for PM
• Look at the CFD, observe the size of the PM-related queues over time. – Especially Pending PM Review which is in the
middle of DEV/Test– And Ready-MMFs as well as DEV Ready in some
cases which depend on PM approval– Advanced – in the cycle time performance
report, focus on PM areas
• When looking at exceptions to Cycle time, participate in the root cause analysis, and see if interaction with PM was part of the long cycle time.
PM related Churn / Context switching / Expediting
• Add Expedited class of service– Can be used by PM to
override priorities in DEV WIP – just to top of queue, don’t override current WIP
• Add emergency class of service– Can also override current
WIP
• Assumption – – This is value trumps flow.
We give up efficiency when we use these COSs
Measuring the effect of value over flow COS
• Look at cycle times for different kinds of classes of service
• Look at distribution of different COS in the WIP
Look at amount of changes in scope
• Replace – need visualization that shows scope changes in content
• Add – can simply look at total scope for a “Release” and observe whether its growing
Case Study – Typical release behavior
1 2 3 4 5 60
20
40
60
80
100
120
140
160
180
200
Total Required WorkLinear (Total Required Work)Actual DonePlanned ScopePlanned DoneOriginal + Dark MatterLinear (Original + Dark Matter)
Added Scope
Growth in Feature
Cost / Dark Matter
• Dark Matter – Is where we thought a feature costs X
• But then, during breakdown, analysis, creating iteration stories, we understand it actually costs X+D
• Then, PM decides whether to scope to fit down to X again, or D is worth it.
• Worthwhile tracking our behaviour on this, and learning from it.
• What is the right D number/percent? Good question!
• Can be observed in the CFD for a release.
Sunk Costs
• Add a LANE that collects features/stories that are “ON HOLD” – the Recycle Bin in the archive area
• The amount of work done on them is the sunk cost
• Amount of work hard to measure, so use alternative:– CYCLE TIME – look at cycle time for end
lane being the recycle bin
Rework due to late feedback by PM
• Will appear as high cycle times• Will appear as moving back cards on
the board (need to find way to measure)
• Can use special Class of service / card type to identify these kinds of stories better for measurement/tracking purposes
Workload compared to DEV
• See how much workload is in PM compared to DEV
• Look for trends and major changes that can indicate:– Bottleneck in PM– Idle and slack capacity – expect to see PM seeing
clients/customers at those times