software development and delivery metrics that matter
TRANSCRIPT
What are you interested in?
New Product Construction
Product Maintenance
Manage Cost
IT Visibility
SuccessEmployee
ManagementCustomer
Satisfaction
What’s in it for you?• Techniques• Managing technical product delivery• Managing product maintenance
• Metrics• Measure Constraints for Visibility
• Philosophy• The big picture has tons of little details
About Me
Software Developer
Agile Team Member
Team Lead
Agile Advocate
SDLC Implementer
Why do we want to measure?
Product
Development/QA
Physical Infrastructure
Project Management
COST
What are your variable costs?
Requirements Quality
Defects / Testing
Technical Skill
Infrastructure
These are under your control
The measurement process
Take ActionIdentify ProblemsMeasure
What to measure?
Development
•Code Coverage•Team Velocity•Developer Monitoring•Defect Kill Rate
QA•Defects Found•Testing Time : Manual vs. Automated•Kickback Rate•Quality Tracing
Business•Story Creation Rate•Story Unblock Rate•Release Progress
Actionable Stories
Itera
tion 1
Itera
tion 2
Itera
tion 3
Itera
tion 4
Itera
tion 5
Itera
tion 6
Itera
tion 7
Itera
tion 8
Itera
tion 90
20
40
60
80
100
120
140
160
180
Actionable
Added Unblocked
Itera
tion 1
Itera
tion 2
Itera
tion 3
Itera
tion 4
Itera
tion 5
Itera
tion 6
Itera
tion 7
Itera
tion 80
20
40
60
80
100
120
140
160
Blocked
Added Blocked
Unblock Rate
Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 5 Iteration 6
-1.5
-1
-0.5
0
0.5
1
1.5
Unblock Score Unblock Rate
New Story Count Story Blocked Story Unblocked10 3 710 10 010 6 410 0 1010 5 510 7 3
The Hawthorne Effect
• Knowing you are being monitored will change your behavior
Quality Tracing• Add findable defects in product
each week• Track how long it takes to find all
defects• Reveal the find/miss rate each
week
• Automated UI Testing• QA Define Requirements
1 2 3 4 5 6 7 8 9 10 11 12 13 14 150
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Trace Defects Found
Iteration Flow (QA Constraint)
1 2 3 4 5 6 70%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Defined In Progress Completed Accepted
1 2 3 4 5 6 70%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Defined In Progress Completed Accepted
Team Velocity
Developer Monitoring
1 2 3 4 5 6 7 8 9 10 11 12 13 140
200
400
600
800
1000
1200
Developer #1
Code Added Code Modified
1 2 3 4 5 6 7 8 9 10 11 12 13 140
200
400
600
800
1000
1200
Developer #2
Code Added Code Modified
Defect Kill Rate𝐴𝑣𝑒𝑟𝑎𝑔𝑒(𝐷𝑒𝑓𝑒𝑐𝑡𝐶𝑙𝑜𝑠𝑒𝑑−𝐷𝑒𝑓𝑒𝑐𝑡𝑂𝑝𝑒𝑛𝑒𝑑)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160
1
2
3
4
5
Difference
1 2 3 4 5 6 7 8 9 10 11 12 13 140
2
4
6
8
10
12
14
Average
Average
Implementation of Unit Testing!
SDLC
Code Coverage
Velocity versus Coverage
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250%
10%
20%
30%
40%
50%
60%
70%
0
2
4
6
8
10
12
14
16
18
Coverage Velocity Linear (Velocity)
Putting it all together