linda esker kathleen dangle fraunhofer center maryland

© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering

Developing An Early Project Cost Estimate:

An Applied Case Study From Project Concept through Contract Award

Linda EskerKathleen Dangle

Fraunhofer Center Maryland


2

Topics Covered

• Project Overview

• The Early Project Cost Estimate

• Estimate Phases– Challenges– Strategy– Approach: Methodology and

Models Used– Decisions to go forward

• Summary / Lessons Learned


3

Project Overview

• A large Ground Space Communication System

• COTS-intensive– For both HW and SW

• 5-year project duration

• Many areas highly scientific– Need appropriate technical expertise as well as those who

understand what is involved in the cost estimate, especially SW– Would require a blended estimation team

• Development of the estimate began in the earliest initial concept phase

© 2012 Fraunhofer USA, Inc. Center for Experimental Software Engineering 4

Cost Estimate Began Before Typical 1st Milestone

RFP

Life-Cycle Phases

Reviews


5

The Early Project Cost Estimate

• Cost estimate developed by the project to estimate government and contractor project costs for budget/funding approval

• Activity played a significant role in the development of the project– Value of the activity included not just the resulting estimate, but also drove

understanding, evolution, and “selling” of the project concept

• State of the practice for early project estimates:– Typically use general, high-level, “personal” heuristics/ rules of thumb by

subject matter experts

– Not able to effectively use estimation tools because details required as inputs are not known early in the project

– Process followed not as disciplined or structured (or repeatable) as expected in estimates during later phases of project lifecycle


6

The Early Project Cost Estimate (cont.)• The good news: With government emphasis on

better project estimates, projects now need a basis of estimate and confidence levels

• The bad news: Solid estimates face challenges that make use of estimation tools an uphill battle

• Politics — Organizational budgets have a life of their own, are highly competitive, and defy comprehension for those not in the political know

• Personal opinion and aggressive (or naive) optimism, “This should be easy…”

• Distrust of what they do not understand


7

Early Project Estimate Phases

“In the Beginning…”

•Forming the concept

“Let there be light…”

•Maturing the project’s architecture and requirements

•Defining a project lifecycle & WBS

•Defining the estimation process and models

“And it was good…”

• Iterating on the architecture, schedule, models, and estimate

•Evaluating options and “what ifs”

1 2 3 4

Lessons Learned

Lessons Learned

Lessons Learned

“And it was better…”

•Formalizing Estimate Confidence


• Started with:– Top-down approach– Fully understanding of the scope – Understanding SW and HW requirements of the project– Examined use of a commercial estimation tool

• Found it was difficult to use at this very early stage– Needed more details than what was available

– Input was constantly changing as project investigated technologies and needs and tool data cumbersome to update

• Management did not trust estimation tool results– Wanted greater insight into and control on how estimate was determined

– Wanted costs broken down into their areas of interest

Early Project Estimate Phase 1

8



• Concept studies performed to develop notional architecture

• Estimate used: • General parametric models

• Expert judgment

• COCOMO

• Spreadsheets

• HW: Developed MEL (Master Equip. List)

• SW: Used analogies/LOC• Percentages used to estimate many

areas:• Management

• Contingency, reserve & inflation

• Spread of labor, HW, SW by fiscal year

• Other technical unknowns

9

Approach

Challenge:Create initial estimate with

minimum information

Strategy:Formulate

concept

Lessons Learned

Initial informal “off-the-cuff” estimates can derail a project from the start

Do not count on Mgmt. acceptance of tool estimate results

Phase Start: Blank page; immaturerequirements; forming team; gatheringhistorical data; top-down approach

Decision:Next use

bottoms-up approach for

greater accuracy


• Enhanced the Estimate with some structure: a notional architecture, schedule, and bottoms up-analysis

• Stabilized requirements to refine the HW and SW estimate

• COCOMO schedule and effort distributions allowed us to distribute costs over the timeframe to allocate budget to fiscal years

• COSYSMO could now be used to validate engineering SME estimateo Needed to use requirement expansion factors

• Project still going through definition and needed to adjust for changes


10



• Migrated approach from parametric overlay to bottoms up

• Established physical notional architecture to organize costs

– Approach helped drive engineer thought process

– Estimate used:

– Expert judgment

– Analogous estimation - some historical data

– COCOMO for SW effort & duration

– % still used to estimate

– Management (based historical data)

– Contingency, reserve, inflation, & unknowns

– Spread of labor, HW, SW by fiscal year

11

Approach

Challenge:Establish

sound basis for good estimate

Strategy:Use bottoms- up approach; define models;HW/SW by arch

Lessons Learned

Hybrid of bottoms-up & parametric modeling works best

Everyone understood what was known (justifiable) and unknown

Phase Start: Immature requirements; notional architecture beginning to evolve

Decisions:Pursue 2

independent paths (dev., deploy.); focus

on things missing


Estimation Model

Iterative Development of the Estimate

12

WBS(By Notional Architecture)

RESOURCESHardware– Antenna

o Antenna part 1o Antenna part 2

– Computerso Server o Workstation

• • •

Software– Software item 1– Software item 2

• • •

Engineering tasksIntegration TasksTransition tasks

COCOMO Model Effort,

Duration

Items,Costs

SLOC

Effort

Notional System Architecture

Project Lifecycle Schedule

Master Resource Sheet of

• Hardware• Software• Engineering Effort• Integration Effort• Transition Effort

$0

$10,000,000

$20,000,000

$30,000,000

$40,000,000

$50,000,000

$60,000,000

$70,000,000

Labor, Material, Cost Phasing


• Areas of Focus– Identifying a good system model on which to base confidence in

the estimate—having a solid Basis of Estimate (BOE)

– Ensuring Implementation schedule is realistic

– Understanding changing expectations and being flexible to deal with change

► Filling voids in analogous systems by using SW SMEs to provide sizing information

► Using a notional system to obtain sufficient information for an early project estimate and trying to avoid over-engineering the perfect system

• Key Accomplishments– Met goals of project: Early Project Estimate for RFP; presentation

to HQ; understandable/supportable basis for budget


13

– SW:

– HW:



• Incorporated inputs from Trade Studies and another independent cost estimate

• Aligned cost structure, schedule, WBS, and notional architecture

– PM worked with all technical teams to understand basis of estimate (BOE)

– Loaded Resources by skill level & labor rates– Spread labor, HW, SW costs by fiscal year based

on schedule– Only PM remained %– Enhanced estimate model to allow:

• Many different views (architecture elements, high cost drivers)

• Investigation of options ("what if" a ground station were eliminated?)

14

Approach

Challenges:Honing the estimate;

Strategy:Use 2 separateTeams (dev. &

deploy.); merge results

Lessons Learned

BOE review sessions need a strong driver

The better the estimate fidelity the more useful for “holes” & “what ifs”

Working as a team creates buy-in and project team is much smarter

Phase Start: Matured requirements & team; near complete notional architecture

Decisions:Ready to go;

freeze estimate for RFP



• Performed Risk Cost Analysis• Focused on high cost drivers

– Optimistic– Most likely – Pessimistic estimate

• Used triangular distribution and Monte Carlo to simulate confidence ranges

• Compared dispersions with expected ranges

• Detailed resource items enabled analysis of procurement long lead items & phasing

15

Approach

Challenge:Understand

Confidence in Estimate

Strategy:Examine Estimate

Cost Risks

Lessons Learned

A solid early Project cost estimate enables developing confidence levels and understanding where the estimate falls within the levels before contract start

Phase Start: RFP Released; Project estimate frozen; fewer distractions

Decisions:Project has

confidence with expected

range


COCOMO Lessons Learned

• At this phase, the summary COCOMO tables are very helpful. However,

– They do not give consistent information

– Percentages do not add up to 100%

– Total effort has two different values

• This makes them appear to be unreliable

– Had to make adjustments

Module Name SampleTotal Size 37975Total Effort 230.308313

Overall Schedule (%)Schedule (Months) Effort (%) Effort Staff

Plans And Requirements 22.53% 14.763379 7.00% 16.121582 1.091998Product Design 27.27% 17.864799 17.00% 39.152413 2.191595Programming 42.93% 28.130345 54.20% 124.82886 4.437516Integration and Test 29.80% 19.524292 28.80% 66.327036 3.397155TOTALS 122.53% 80.282815 107.00% 246.4299 11.118264

EFFORTPlans and

RequirementsProduct Design Programming

Integration and Test TOTALS

Requirements Analysis 7.211762 4.894052 4.993155 1.658176 18.757145Product Design 2.842752 16.052489 9.986309 3.316352 32.197902Programming 0.929637 5.337729 70.528308 26.220951 103.016625Test Planning 0.666338 2.401298 7.031867 2.078163 12.177666Verification and Validation 1.230594 2.988584 10.776733 18.63815 33.634061Project Office 1.972248 3.810935 7.323452 4.554541 17.661176CM/QA 0.462173 0.926657 7.947597 5.217811 14.554238Manuals 0.806079 2.740669 6.241443 4.642893 14.431084TOTALS 16.121583 39.152413 124.828864 66.327037 246.429897


COCOMO Lessons Learned

• Modular approach of COCOMO products is ideal

• Project had definite need for a COCOTS-like tool, but – Seemed too immature

– Decisions on COTS were not controlled by government

• Overlap of COSYSMO and COCOMO II makes management wary of using them together– Need some better quantification of the overlap in the

estimate


Start early: Plan for iterations of the estimate– The project estimate will iterate through phases

and different estimation techniques as more information is learned

– The estimation process can aid in the maturation of the project concept, requirements, and schedule

Summary / Lessons Learned

18

Be Adaptable/Flexible: Standard tools cannot handle all of management’s needs

– Early on there is not a clear definition/agreement on how to proceed and implement the project; need to change and adapt

– Be sure your cost estimation tools are flexible• Need to handle what is known about the project at early and also later phases

of the evolution• COTS may not match all your needs—a hybrid approach works well


Be Mindful of Management: Need to work with high-level management and keep them in the loop

– Early “off-the-cuff” promises can derail a thorough estimation process

• Don’t promise too much too soon—early promises cannot be rescinded

Summary/Lessons Learned

19

– It is imperative that the higher management teams have faith in the estimation work

• You need to thoroughly understand (and justify) how a tool’s model handles all aspects of the estimate

– Even if not accepted for an end result, use tools to also give a sanity check for any ad hoc estimation practices


WBS is Critical: Projects need a WBS that can help them collect data and learn

– Lobby for better WBS to facilitate data and information collection when necessary

Summary / Lessons Learned

20

– Current WBS templates/guidelines make it difficult to extract software from other parts of the system development work

• Project was afraid to specify that WBS differentiate between software and other activities

• Would love to see this addressed as a SW best practices


For more information contact us at:

Linda [email protected]

Kathleen Dangle [email protected]

21

Questions

linda esker kathleen dangle fraunhofer center maryland

Documents

early estimate

early project estimates

project conceptstate

better project estimates

basis of estimate

resulting estimate

fraunhofer usa

contractor project costs