How do we Measure Software?RUBEN-DARIUS MOLDOVAN13 MAY 2015 DEVTALKS

HDWM Software Questions• Who measures?

• Why measure?

• Where?

• When?

• How?

• What is ? Why would I invest in your product?To improveTo express qualitative factor by a numberTo estimate costs, schedule, complexity, effort requirements

HDWM Software Agenda – Definitions

Software Definitio

ns

QuestionsTypesComplexityStatus

Metrics

PlanningDeveloping

TestingSupport

Summary &

Questions

HDWM Software Type

Price List

Internal• Shared

HDWM Software Complexity• Business• No. of People working on it

• No. of Customers

• No. of Supported Releases

• Size?

• Technology• Lines of code

• Modules

• Languages

• Symbiosis

• Customization

• Size?

Weighted Methods?

HDWM Software Status• Value• Sales

• Customers

• Quality• Health

• Performance

• Direction• Market

• Competitors

HDWM Software Life• Calendar• Months

• Weeks

• Fiscal• Years

• Quarters

• Cycle• Releases

Planning Specificatio

ns

Development

Design

Testing Validation

SupportMaintenanc

e

HDWM Software Agenda – Metrics

Software Definitio

ns

QuestionsTypesComplexityStatus

Metrics

PlanningDeveloping

TestingSupport

Summary &

Questions

HDWM Software Metrics

Support

Incoming Cases

Days to Resolution

Elevations

Customer Satisfaction

Index

Solved without Help

Development

Requirements

Defects

Enhancements

Escalations

Patents

Lines of Code

Testing

Automation

Performance

Code Coverage

Defect Escape Rate

Security

Financial

Sales / Licenses

Costs

Customers

Net Promoter Score

Other

Ease to Run

Time to Value

Usage Statistics

Maintainability

Mismatch / Consistency

HDWM Software Meaningful• Deliverables

• Process Effectiveness

• Development Time

• Product Quality

HDWM Software Reports

HDWM Software Indicator ModelNo. Meaning

1 This is the metric's value.

2The background color of the cell indicates the status of the metric which can be: "red", "yellow" or "green" color, based on the Thresholds

3 A blue rectangle means that the status changed from red to yellow/green or vice versa.

4

This is the trend of the metric. The arrows can be colored in red, or green if the difference was higher than 20% of the previous quarter. The trend is up if the product improved over previous quarter from a business perspective, not from a mathematical point of view.

5This is the difference from the previous quarter's value in percent. The number is green if the metric improved, or red if it declined.

6 This is the absolute difference from the previous quarter's value.

HDWM Software Indicator ExampleLet's suppose a product has 8 Escalations for the current quarter and 5 Escalations for the previous quarter. The Incident Escalations cell would have the following values:

• 8 as the cell value (count), 2 is the grade • Background color will be red, because Esc grade is 2. • The blue border is present because last quarter the Esc Grade was 3.0 and was

yellow. • The trend arrow is down (current quarter value is worse than previous quarter),

but the difference is more than 20%, so red color is applied. • The percentage difference rounded up to integer number is 60%

and is colored in red because the metric is worse than previous quarter. • Finally, the absolute difference is +3 Incident Escalations

More detailed information is available on every number's mouse over tooltip: values for the selected timeframes, grades, including metric numerators and denominators and history charts.

HDWM Software Support• Incoming Cases• Increase / decrease why?

• Seasonal?

• Solved without help

• Days To Resolution• What cases do we analyze?

• Average over Time? Median?

• Elevations

• Customer Satisfaction Index• Top 2, Top x

• Overall, Product Quality, Support, Documentation, Ease to.. etc

How an improvement looks?How do we improve?What’s the cost of improvement?

HDWM Software Support - DTR• Days To Resolution (DTR)

DTRAvg =

• For Current Backlog = • Considering the Urgent / High severity

• Customer Interaction

• What would you show in a current month?

• High volume vs. Low volume

• Is this seasonal?

HDWM Software Planning• Agile Metrics• Burn Up / Downs

• Velocity

• Feature Status

• Traceability• Defects

• Features

• Tests

HDWM Software Development• Cost per Defect• Critical / Serious Defects• Customer Visible / Customer Encountered• Backlog of Defects / Enhancements

• Escalations

• Defect Escape Rate (Leakage)

• Code Analysis• Length (LOC)• Cost per 1k / Errors per 1k / LOC per engineer

• Complexity

• Re-usability

HDWM Software Testing• Automated Tests• 100% automated tests?

• Ease to (automated) test

• Analysis of Test Results• Performance

• HW/SW Requirements

• Security

HDWM Software Financial• Sales• Licenses

• Support

• Market Share• Net Promotor Score

• Double blind surveys

HDWM Software OtherProduct Management

• Traceability (between Requirements, Test and Defects)

Improvement Programs

• Time 2 Value (T2 Demo, POC, T2 Install, T2 See Results, T2 Profit of Usage, T2 Customization)

• Ease 2 Run (Upgrades, HC to Operate)

HR

• Attrition

• Current (T) Shape or Engineers, Trainings

HDWM Software Usage Statistics• Know the top Features / Modules• and their performance

• Know the top Users• their environment

• their roles

• Attrition

• Current (T) Shape or Engineers, Trainings

HDWM Software Agenda – Summary

Software Definitio

ns

QuestionsTypesComplexityStatus

Metrics

PlanningDeveloping

TestingSupport

Summary &

Questions

HDWM Software Summary• Pre-Release / Post-Release Metrics

• Process / Product - Project / Program Metrics• Size / Function Oriented Metrics

• Customer Metrics• Private and Public Metrics

• What is improvement?• How do we simulate a 5%-20% improvement?

• Usage of the Product Logs• Install base history

• Ecological (Predictive) Metrics• The substance to anchor into the future

HDWM Software Questions

?

HDWM Software

Thank

you!