the metrics that matter using scalability metrics for project planning of a distributed workforce ...

The Metrics that MatterProject Planning using Scalability Metrics for Distributed Development

Dilber Mann, Development DirectorRelic Entertainment Inc.

• Establishing reliable metrics to help make informed decisions about organizing a distributed & scalable workforce.

Overview

• Context from multiple perspectives– Publishing / Central Group– Onsite Partner/Vendor management.– Studio Development

Understanding Content Source

Discussion Points

• Distributed Trends

• Establishing your own system of metrics

• Incorporating scalability metrics

• Discussing Real-World examples.

• Converting your Metrics into KPI’s

Identifying the Audience

• Two distinct groups under Distributed Development

1. Consolidated Resource Structure• scalability through Outsourcing.• development outsources when needed

2. Distributed Resource Structure• scalability through distributed workforce.• development is always distributed.

Distributed Trends

• Deciding between Distributed or Consolidated Organization.

– Option (Distributed) is available when region has a wide range for cost of living.

– Choice (for both) is driven by cost savings and scalability needs.

ExternalPartner

Distributed Trends

• Relation between Consolidated & Distributed Teams

Development Studio

Location A

Location B

Location C

Location D

Outsource

Distributed Workforce

• Advantages– Manage burn rate at lower tier/cost

locations.

– Maintain consistent man-day rates.

Distributed Workforce

• Wide Range for cost of living in Asia/Europe region.

• Tier 0/1: modern and Large Cities• Tier 2: modern but developing cities• Tier 3: developing cities• Tier 4: under developed, but progressively

developing Cities.

Distributed Workforce

• How to plan the distribution?– by using reliable metrics to make

informed decisions.

Establishing Foundation Metrics

• Cost Savings Analysis– Man-Day Rate comparisons– Scope– Impact of Studio Support

Cost Savings Analysis: Man-Day RateVendor Rate Studio Rate

Avg Studio Man-Day Rate $198.00 $700.00Mgm Overhead $22.00 $70.00Man-Day Rate $220.00 $770.00

Vendor Costs Studio CostsScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00

Vendor Costs Studio RateScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00

Studio Support $250,000.00

TOTAL $800,000.00 $1,925,000.00

****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****

71% SAVINGS

VENDORRATE

STUDIO RATE

Cost Savings Analysis: ScopeVendor Rate Studio Rate

Avg Studio Man-Day Rate $198.00 $700.00Mgm Overhead $22.00 $70.00Man-Day Rate $220.00 $770.00

Vendor Costs Studio CostsScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00

Vendor Costs Studio RateScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00

Studio Support $250,000.00

TOTAL $800,000.00 $1,925,000.00

71% SAVINGS

VENDORRATE

STUDIO RATE

****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****

Cost Savings Analysis: Studio SupportVendor Rate Studio Rate

Avg Studio Man-Day Rate $198.00 $700.00Mgm Overhead $22.00 $70.00Man-Day Rate $220.00 $770.00

Vendor Costs Studio CostsScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00

Vendor Costs Studio RateScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00

Studio Support $250,000.00

TOTAL $800,000.00 $1,925,000.00

71% SAVINGS

VENDORRATE

STUDIO RATE

****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****

Vendor Rate Studio RateAvg Studio Man-Day Rate $198.00 $700.00Mgm Overhead $22.00 $70.00Man-Day Rate $220.00 $770.00

Vendor Costs Studio CostsScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00

Vendor Costs Studio RateScope (Man-Days) 2500 2500Rate * Scope $550,000.00 $1,925,000.00

Studio Support $250,000.00

TOTAL $800,000.00 $1,925,000.00

58% SAVINGS

VENDORRATE

STUDIO RATESTUDIO

SUPPORT

Cost Savings Analysis: Studio Support

****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****

Resource Ramp-Up / Down

5% 9% 14% 17% 17% 17% 14% 9% 9%

0%2%4%6%8%

10%12%14%16%18%

1 2 3 4 5 6 7 8 9

• Approx. 50% of Manpower spread across 3 months during peak time.

****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****

Resource Compression

12% 23% 28% 23% 12% 12%

-5%

0%

5%

10%

15%

20%

25%

30%

1 2 3 4 5 6 7 8 9

• Approx. 50% of Manpower spread

across 2 months during peak time.

****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****

Resource Compression

12% 41% 41% 25%

-20%

-10%

0%

10%

20%

30%

40%

50%

1 2 3 4 5 6 7 8 9

• Approx. 50% of Manpower

spread across 1.25 months during peak time.

• Disclaimer: The sum of Month to Month % of work may exceed 100% total due to contingency buffer used for planning.

****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****

Resource Productivity

• 40 peak resources

• 50% efficient at start

• 80% efficient at peakResource Efficiency & Savings

Pilot Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7

Group 1 50% 70% 80% 80% 80% 80% 80% 80%Group 2 50% 70% 80% 80% 80% 80% Group 3 50% 70% 80% 80% Group 4 50% 70% 80%

****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****

Importance of Cost Savings

• 35% - 50% Good Range for Cost Savings.• Is Cost Savings the best decision making metric?• Is it a metric that speaks to scalability?

Building a System of Scalability Metrics

• Man Effort Established– Internal Man Effort established externally.

• Added Value– Man Effort added to project.

• Budget Savings– Look at Internal cost of doing the same scope.

Scalability Metrics

****Rates, scope, costs, and savings listed here are a sample used for comparison and do not reflect specific data used by any studio or external vendor.****

Case Study

• Improving Performance over time.– Step 1: Conventional OS– Step 2: Distributed Development– Step 3: Dissolving the centralized role

Conventional OS

Project A(Conventional)

19% 8% 13%

24% 7% 19%

40% 20% 25%

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Distributed DevelopmentProject A

(Conventional)

19% 8% 13%

Project B(Distributed Dev)35% less scope than

Project A

24% 7% 19%

40% 20% 25%

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Dissolving the OS role

19% 8% 13%

24% 7% 19%

Project C(Optimum)

40% 20% 25%

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Internal

External

Converting Metrics into KPI’s

• Why?– Keep track of your planned goals.

• How?• Continue to update your data.

– (planned & actuals).

• Convert Metrics into KPI’s.

Converting Metrics into KPI’s

• KPI’s (Key Performance Indicators)

– Metrics scaled to a 0 – 100% range.• +100% Out performing • 75% - 100% Normal Performance • >50% or <75% Under performing • <=50% Need Escalation

– Easy for multiple stakeholders to interpret.

Converting Metrics into KPI’s

• 20% Added Value Metric (Max Threshold)– 20% x 5 (Multiplier) = 100% KPI

• 25% Budget Savings Metric (Max Threshold)– 25% x 4 (Multiplier) = 100% KPI

• 40% Man-Effort Metric (Max Threshold)– 40% x 2.5 (Multiplier) = 100% KPI

Converting Metrics into KPI’s

ManEffort Established Added Value Metric Budget Savings Metric

Project A(Conventional) 47% 38% 54%

Project B(Distributed Dev)35% less scope than

Project A

59% 33% 76%

Project C(Optimum) 100% 100% 100%

• To establish reliable metrics that can help us make informed decisions about how we plan our distributed workforce.

Our Purpose

• Distributed Trends– Manage the impact of idle/burn rate by using Lower Tier/Cost locations.– Maintain consistent man-day rates by creating balanced distributed teams.– Identify reliable metrics to plan the distribution of resources.

• Cost Savings Analysis– Include Scalable Management & Studio Support.– Know that Schedule Compression has an impact.

• Decision Making KPI’s– Know what your KPI’s are telling you.– Agree as an organization on the KPI goals.– Track KPI’s throughout production.

Summary

THANK YOU!

• dmann@relic.com

Q&A