11th Annual Rocky Mountain Project Management SymposiumRoadmap to the Future: Leadership Strategies for Local to Global Success

Deliverables Based Planningsm

Providing Actionable Information To The Program

Manager

1

Glen B. AllemanVP, Program Planning and Controls

Lewis & Fowlerwww.lewisandfowler.com

Ronald Powell, PSPManager, Program Planning and Controls

Ball Aerospace & Technologies Corporationwww.ballaerospace.com

What are the Primary Measures of Success for a

Program?

Are We Done?When will we be Done?What will it cost to be

Done?

What does Done look like for the customer?

How can we recognize Done when it arrives?

How can we be sure we can get to from here to Done?

What are the impediments to getting to done?

2

Why Are Deliverables

the Best Measure of Progress?

Because the customer

contracted for a

“Capability”

We need the capability to fly to

LOE, with 4 on board, dock at ISS,

stay for 90 days, and return home safely3

Lack of predictive variance analysis

Untimely and unrealistic Latest Revised Estimates (LRE)

Progress not monitored in a regular and consistent manner

Lack of vertical and horizontal traceability of cost and schedule data for corrective action

Lack of internal surveillance and controls

Managerial actions not demonstrated using Earned Value

Inattention to budgetary responsibilities

Work authorizations that are not always followed

Issues with budget and data reconciliation

Lack of an integrated management system

Baseline fluctuations and frequent replanning

Current period and retroactive changes

Improper use of management reserve

EV techniques that do not reflect actual performance

We all know the train wreck starts when there is…

Mary K. Evans Picture Library

So how can we address these issues in a cohesive way? 4

a program management method that provides a clear and concise definition of Done in units of measure meaningful to the customer.

based planning and measuring of increasing maturity are the basis of

The Starting Point for Programmatic Success is to Define with the needed mission or system capabilities

5

1. Identify Needed Systems Capabilities – Define the set of capabilities needed to achieve the program objectives or a particular end state for a specific scenario.

2. Establish the Requirements Baseline – Define the technical, organizational, and operational requirements that must be in place for the system capabilities to be fulfilled. Define these requirements in terms isolated from the implementation details. Only then, bound the requirements with technology alternatives.

3. Establish the Performance Measurement Baseline (PMB) – Build a time–phased network of scheduled activities describing the work to be performed, the budgeted cost for this work and the organizational elements that produce the deliverables. Define the technical performance measures showing how this work proceeds according to the technical and programmatic plan.

4. Execute the Performance Measurement Baseline – Execute the Performance Management Baseline Work Packages, while assuring that all performance assessments represent measures of Physical Percent Complete. Execute the risk mitigation or retirement activities.

Four Core Processes of Deliverables Based Planningsm

6

Four Critical Process Areas for Successfully Managing Progress Through the Deliverables

How WhereWhenWho

Why

What

Identify System Capabilities

Establish a Performance Measurement

Baseline

Execute the Performance Measurement

Baseline

Capabilities Based Plan

Operational Needs

Earned Value Performance

0% /100%

Technical Performance

Measures

System Value Stream

TechnicalRequirements

Identify Requirements

Baseline

1

2

3

4

Technical Performance

Measures

PMB

Continuous Risk Management Process

Changes to strategy

Changes to requirements

Changes to program plan

1

2

3

4

5

6

7

8

9

1010 O

rgan

izin

g P

rinci

ples

of

Del

iver

able

s B

ased

Pla

nnin

gsm

7

The Five Process Areas of Deliverables Based Planningsm

88/152

Define the set of capabilities needed to achieve the program objectives or the particular end state for a specific scenario. Using the ConOps, define the details of who, where, and how it is to be accomplished, employed and executed.

Identify Needed System

CapabilitiesWhat capabilities are needed to fulfill the ConOps and System Requirements?

1

Define the technical and operational requirements that must be in place for the system capabilities to be fulfilled. First, define these requirements in terms that are isolated from any implementation technical products. Only then bound the requirements with technology.

Establish the Requirements

BaselineWhat technical and operational requirements are needed to fulfill these capabilities?

2

Build a time–phased network of schedule activities describing the work to be performed, the budgeted cost for this work, the organizational elements that produce the deliverables, and the performance measures showing this work is proceeding according to plan.

Establish the Performance Measurement

Baseline What is the schedule that delivers product or services that meet the requirements?

3

Execute work packages, while assuring all performance assessment are 0%/152% complete before proceeding. No rework, no forward transfer of activities to the future. Assure every requirement is traceable to work and all work is traceable to requirements.

Execute the Performance Measurement

Baseline What are the periodic measures of physical percent complete?

4

Perform the 6 process areas of Continuous Risk Management for each Deliverables Based Planningsm process area to Identify, Analyze, Plan, Track, Control, and Communicate programmatic and technical risk

Performance Continuous Risk

ManagementWhat risk will be impediments to success and what are the mitigations ?

5

These principles have specific relationships

Capabilities Drive Requirements

Requirements are Fulfilled by Work

Packages

Work Packages Define Deliverables

Measure Physical Percent Complete of

Each WP

Perform OnlyAuthorized Work

Earned Value Defines Progress

Adjust EV for TechnicalPerformance

Measure

Use Past Performance to Forecast Future

Performance

1

2

3

5

6

7

8

9

10

PMB Describes Work Sequence

4

9

Assess the capabilities being provided through the deliverables

Fulfill the requirements through effort held in the Work Packages

Produce deliverables from Work Packages

Planned BCWSPhysical % Complete

WP’s contain deliverables that fulfill

requirements

Capabilities topology

defines requirements

flow down

WP flow must describe the increasing maturity of

the product or service

Producing the deliverables in the planned sequence maintains the

value stream to the customer

9

Here are some of the tools involved in building a description of the increasing maturity of the program

Product Development Kaizen’s Sticky’sProduct Development Kaizen’s Sticky’s

A Mind Map™ of the Sticky’sA Mind Map™ of the Sticky’s

MS Project™ export of the Mind Map

MS Project™ export of the Mind Map

PERT Chart Expert™ on the Wall of TruthPERT Chart Expert™ on the Wall of Truth

10

Making deliverables based plan is interactive, not a static production of a unreadable specification

11

Capabilities And Their Enabling Requirements Are Flowed Down To A Work Breakdown Structure As Deliverables Produced By Work Packages

Deliverables

WBS

Tasks and Schedule

Business NeedProcess Invoices for Top Tier Suppliers

1st LevelElectronic Invoice Submittal

1st LevelRouting to Payables Department

2nd LevelPayables Account Verification

2nd LevelPayment Scheduling

2nd LevelMaterial receipt verification

2nd Level “On hand” balance Updates

Work Package

1 2

3

4

6

5 A

B

Deliverables defined in WP

Terminal Node in the WBS defines the products or services of the project

Terminal node of the WBS defined by a Work Package

Tasks within the Work Package produce the

Deliverables

100% Completion of deliverables is the measure of performance for the Work

Package

Management of the Work Package Tasks is the

responsibility of the WP Manager. These are not held in the master plan

A decomposition of the work needed to fulfill the business requirements

12

Cost, Schedule, Technical Model†

WBS

Task 100

Task 101

Task 102

Task 103

Task 104

Task 105

Task 106

† This is a Key concept. This is the part of the process that integrates the cost and schedule risk impacts to provide the basis of a credible schedule.

Probability Density Function

Understand the Processes Find Past Performances Ask in depth Questions Analyze the Results

What can go wrong? How likely is it to go wrong? What is the cause? What is the consequence?

Monte Carlo Simulation Tool is Mandatory

1.0

.8

.6

.4

.2

0

Days, Facilities, Parts, People

Cumulative Distribution Function

Days, Facilities, Parts and People

All of this comes together in the Integrated Master Schedule, that produces the deliverables that delivers the requirements that enables the capabilities

13

Events / Milestones Define the availabilityof a Capability at a point in time.

AccomplishmentsRepresent requirements that enable Capabilities.

Criteria Represent Work Packages thatfulfill Requirements.

Work Package

WorkPackage

Work Package

Work Package

Work Package

Work Package

Work Package

Work package

Deliverables Based Planningsm describes of the increasing maturing of a product or service through Events or Milestones, Accomplishments, Criteria, and Work Packages.

Each Event or Milestone represents the availability of one or more capabilities. The presence of these capabilities is measured by the Accomplishments and their Criteria. Accomplishments are the pre–conditions for the maturity assessment of the product or

service at each Event or Milestone. This hierarchy decomposes the System Capabilities into Requirements, Work Packages, and

the activities the produce the deliverables. This hierarchy also describes increasing program maturity resulting from the activities contained in the Work Packages.

Performance of the work activities, Work Packages, Criteria, Accomplishments, and Events or Milestones is measured in units of “physical percent complete” by connecting Earned Value with Technical Performance Measures.

The structure of a Deliverables Based Plan

What does a Deliverables Based Plan look like from 30,000 feet?

14

For those familiar with DoD’s IMP/IMS, Deliverables Based Planning is the older brother.

Capabilities, Requirements, PMB, and Execution are all needed for success of the IMP/IMS.

Critical Success Factors for Applying Deliverables Based Planningsm to A Program

15

Concept Of Operations Describes Mission

Success in Deliverable

Terms

Each Requirement Traceable To Concept Of Operations Outcome

Integrated Master Plan

Describes Increasing

Maturity of Deliverables

Risk Mitigation

And Retirement

Embedded In The Master

Plan

Program Performance

Measured With Risk Adjusted

Earned Value

11th Annual Rocky Mountain Project Management SymposiumRoadmap to the Future: Leadership Strategies for Local to Global Success

Enough of the PrinciplesLet’s Look at the Practice on a Real

Program

My Colleague Ron Powell will describe how Deliverables Based Planningsm principles are applied on programs at

Ball Aerospace and Technology

16

Any Program Planning and Controls method must measure Physical Percent Complete

Here’s how it is done at Ball EVM and Schedule Past Issues Recommended Solution QBD Benefits Tie Back to DBP Next Steps

04/12/2023 17

EVM and Schedule Past Issues

Ability to take credit for all work accomplished How to defend what work was earned Identifying EV milestones in the schedule focused attention on those

tasks so that maximum earned value could be taken IMS was too detailed to manage the program CAM’s used other means to manage and assess progress related to

their work; i.e. detailed schedules Detailed schedules did not reflect what was reported in IMS

04/12/2023 18

Solution – Quantifiable Backup Data (QBD)

Quantifiable Backup Data (QBD)

QBD is a detailed listing of tasks necessary to complete all scope in a work package during the defined period of performance.

It is an approach used to objectively measure performance Each task on the list is weighted – total weighting equals 100% of the work

package BAC (weighting is important, should not be equal weighting on every task).

Upon completion of QBD development the QBD is placed under configuration control.

The CAM assesses physical percent complete of each QBD task. The percent complete is calculated from the cumulative assessments.

The purpose of the QBD is to: Help ensure and demonstrate that all contract work is accounted for; Help ensure the schedule and budget are realistic and achievable; Help to mitigate schedule and budget risks; and, Provide a basis for objectively assessing progress for discretely measured

work packages.

04/12/2023 19

Quantifiable Backup Data (QBD)

Most program CAMs build detail working schedules to manage the day to day work.

The QBD list, when developed from the detail working schedule, readily correlates each QBD item with a specific IMS activity. QBD tasks may be one-to-one with the detail schedule activities, however, the

QBD tasks may be further expanded if needed; Dollars are used to weight each QBD task.

The QBD, when completed, is comprised of inch-step tasks that will support the development and completion of specific deliverable.

04/12/2023 20

04/12/2023 21

Relationship between the IMS the Detail Schedule and the Quantifiable Backup Data (QBD)

Activity 101

Activity 102

Activity 201

Activity 103

Activity 104

Activity 202

Activity 203

Activity 204

Activity 301

Activity 302

Activity 303

Activity 304

Activity 401

Activity 402

Activity 403

Activity 404

IMS 1001

IMS 1002

IMS 1003IMS 1004

IMS 1006

IMS 1007

IMS 1008

IMS 1009

IMS 1005

IMS 1010

QBD Task List

Act 101 IMS 1001 $$Act 102 IMS 1002 $$Act 103 IMS 1002 $$Act 104 IMS 1002 $$Act 201 IMS 1003 $$Act 202 IMS 1003 $$Act 203 IMS 1004 $$Act 204 IMS 1005 $$Act 301 IMS 1006 $$Act 302 IMS 1007 $$Act 303 IMS 1008 $$Act 304 IMS 1008 $$Act 401 IMS 1009 $$Act 402 IMS 1009 $$Act 403 IMS 1009 SSAct 404 IMS 1010 SS

IMS Logic Network

Detail Schedule Logic Network

Earned Value Status

The QBD list is the basis for calculating work package earned value. Project physical percent complete is derived from the earned value

data!

04/12/2023 22

Maintained by the CAM

Percent complete assessed monthly

Cumulative earned value provided to Business Management for input into M*PM.

Task Task EarnedSchedule Budgeted Percent Value

UID QBD Task List Value Complete Percent

101 Task Title 1 100 100 3.3102 Task Title 2 200 100 6.7103 Task Title 3 150 100 5.0104 Task Title 4 300 85 8.5201 Task Title 5 100 75 2.5202 Task Title 6 250 50 4.2203 Task Title 7 225 25 1.9204 Task Title 8 100 10 0.3301 Task Title 9 100 5 0.2302 Task Title 10 125 0 0.0303 Task Title 11 350 0 0.0304 Task Title 12 200 0 0.0401 Task Title 13 100 0 0.0402 Task Title 14 125 0 0.0403 Task Title 15 250 0 0.0404 Task Title 16 325 0 0.0

Totals $3,000 32.5%

Earned Value Calculations Based on QBD

Schedule Improvements Result from Applying the DBP Approach

Deliverables Based Planning used in development of the Product Structure.

Product Structure is used to build consistent level of detail in the IMS and in the detail working schedules. IMS contains sub-assembly level of detail, working schedules contain

component level detail. Product Structure allows engineering, procurement, manufacturing, testing,

and Scheduling to relate to each other. IMS contains all givers / receivers from and to each IPT area

Becomes the integrated plan for the program WBS is more aligned with Product Structure Detail schedule tasks identified to an activity in the IMS and are

aligned for consistent reporting Resource load the IMS for development of work package budgets

(BCWS)

04/12/2023 23

04/12/2023 24

Top-level MRP Prerequisite: Aligned Program Data

MRPIMS Project

Schedule Data

Demand Dates

Costpoint WBS

xxxxx

xxxxx xxxxx xxxxx

Agile PDMProduct Collections

& Product Structures

ChargeNumbers

Mfg

. BO

MP

art Da

ta

302270- 500

L.T. = 46 days

302267- 500

L.T. = 46 days

Qty Per = 1

302249- 501*

L.T. = 54 days

Qty Per = 1

303006- 500

L.T. = 46 days

Qty Per = 1

302248- 500

L.T. = 46 days

Qty Per = 1

2000234- 005 L.T. = 99 days

Qty Per = 1

Customer Order / DemandQty Need Date

8 6/30/09

Qty Start Need8 2/21/09 4/25/09

Qty Start Need6 12/27/08 2/11/09

Qty Start Need6 11/2/08 2/11/09

Qty Start Need8 - 30 in stock

Qty Start Need8 2/11/09 4/25/09

Inv = 2Inv = 2

* 3 Levels Combined

Detail Schedule

QBD

04/12/2023 25

QBD Measures Earned Value Performance in units of Physical Percent Complete

Capabilities Drive Requirements

Requirements are Fulfilled by Work

Packages

Work Packages Define Deliverables

Measure Physical Percent Complete

of Each WP

Perform OnlyAuthorized Work

Earned Value Defines Progress

Adjust EV for TechnicalPerformance

Measure

Use Past Performance to Forecast Future

Performance

1

2

3

5

6

7

8

9

10

PMB Describes Work Sequence

4

9

Assess the capabilities being provided through the deliverables

Fulfill the requirements through effort held in the Work Packages

Produce deliverables from Work Packages

Planned BCWSPhysical % Complete

WP’s contain deliverables that fulfill

requirements

Capabilities topology

defines requirements

flow down

WP flow must describe the increasing maturity

of the product or service

Producing the deliverables in the planned sequence maintains the

value stream to the customer

QBD Task ListAct 101 IMS 1001 $$Act 102 IMS 1002 $$Act 103 IMS 1002 $$Act 104 IMS 1002 $$Act 201 IMS 1003 $$Act 202 IMS 1003 $$Act 203 IMS 1004 $$Act 204 IMS 1005 $$Act 301 IMS 1006 $$Act 302 IMS 1007 $$Act 303 IMS 1008 $$Act 304 IMS 1008 $$Act 401 IMS 1009 $$Act 402 IMS 1009 $$Act 403 IMS 1009 SSAct 404 IMS 1010 SS

5

Next Steps in the Deployment of QBD

Integrate all levels of schedules into common database – Project Server

Automate the development and maintenance of QBD – SharePoint Services

04/12/2023 26

27

28

Glen B. AllemanVP, Program Planning and Controls

Lewis & Fowlerwww.lewisandfowler.com

Ronald Powell, PSPManager, Program Planning and Controls

Ball Aerospace & Technologies Corporationwww.ballaerospace.com