1

Jairus Hihn

Leigh Rosenberg

November 01, 2011

Jairus Hihn

Leigh Rosenberg

November 01, 2011

Team X Risk Tool Team X Risk Tool AndAnd

Team X Cost Risk ToolTeam X Cost Risk ToolOverviewOverview

2

Methodology

Team X focuses on risk identification and initial assessment In the early life cycle it is difficult to crisply distinguish between a risk, an issue, or a concern Items on the Team X Risk List are those items that the team feels are significant enough that the

customers’ or reviewers’ attention is required Many of these items can be addressed by adding detail or specificity to the proposal

Risk process: Prior to the study the Risk Chair reviews the Team X Risk Checklist and identifies potential risk

items Subsystems initiate/revise/reject proposed risks Subsystems score the risk on their subsystem list using the rating scale described in 8.1.2 Risk Chair reviews risks and talks to each subsystems engineer to clarify risk descriptions and

their scores as needed, enters system level scores (what you primarily see) After the study Risk Chair with key subsystems and the facilitator scrubs risks for consistency in

wording and scoring Construct Risk Adjusted Probabilistic Cost Estimate (Also known as S-Curve)

Background: Team X RAP Tool

3

Publications

• “Identification And Classification Of Common Risks In Space Science Missions”, Jairus Hihn, Debarati Chattopadhyay, Robert Hanna, Daniel Port, Proceedings AIAA Space 2010 Conference and Exposition, 1-3 September, Anaheim, CA.

• “Risk Identification and Visualization in a Concurrent Engineering Team Environment”, Jairus Hihn, Debarati Chattopadhyay, Robert Shishko, Proceedings of the ISPA/SCEA 2010 Joint International Conference, June 8-11, 2010, San Diego, CA.

• “Risk Mental Models in Concurrent Engineering Teams”, USC Systems and Software Cost Modeling Workshop, October 2010.

4

Example Risk Checklist: Propulsion

5

• Checklist of common risks developed for each subsystem, through review of a subset of prior Team X studies

• Checklists validated during interviews with Team X subsystem chairs

• Use of checklists during Team X studies revealed: Lists were useful to Risk chair Subsystem chairs felt the general

lists were long, should be tailored to the specific study

Would be easier to use if built into tool

Translation of impact and likelihood ratings into Red-Yellow-Green for NASA 5x5 risk matrix

6

New Risk Scoring Guidance

Risk Chair Master Sheet

7

Step 1

8

Risk Chair Goes through Template and identifies likely risksSelect copy risk tabSelect/import risks from

a template or studyBoth general risk and

subsystem risks

Risk Chair opens up Study Risks Window Initially blank In end summarizes all risks

once populated

Example S-Curve from NASA CEH

9

Go back to Add/Edit TabSelect Add hit edit to add additional

risks not in imported list

Select Add hit edit to add additional

risks not in imported list

Select risk and hit edit to revise, add details and score

Select risk and hit edit to revise, add details and score

All updates done here

10

Risk-Adjusted Probabilistic Cost Estimate Methodology

Estimate/Model Uncertainty

Estimated schedule risk based on inputs from Mission Design

Technical risks based on key risks based on risks identified by Team X

Risk-Adjusted Probabilistic Cost Estimate

Convolve Yields

11

Risk-Adjusted Probabilistic Cost Estimate Methodology

Each WBS level 2 item has its own distributional CER

Each WBS line item has an underlying algorithm such as

Where model error is captured through a normal distribution on the standard errors of the algorithm’s coefficients (the βs) and input uncertainty is captured by ranges on the inputs.

A Monte Carlo is run on these coefficients and variables

Each WBS’s distribution is then convolved using full correlation in a Monte Carlo run to produce a single total cost distribution

This provides a CDF that represents the model and cost driver uncertainty

nnWBS xxxCost

i

21210

12

Risk-Adjusted Probabilistic Cost Estimate MethodologySchedule Risk

Schedule Schedule distribution is derived from analysis and historical data Likelihood of slip is based on analysis of 19 historical JPL in-house and contracted missions Impact is based on Team X effort profiles and mission design determination of months

between launch opportunities Launch opportunities identifed by Mission Design

13

Risk Idenitification and Scoring

EHM Flyby is a relatively low risk mission compared to other similar space science missionsSC has relatively high heritageModerate number of instruments

There is one significant risks that need to be addressed ASRG performance and delivery

date of flight ready is still highly uncertain

Specific mitigations are not identified but the impact is based on a best estimate for the cost impact should the risk manifest.

14

Implementation Risks

L

i

k

e

l

i

h

o

o

d

5

4

3

2 R:1 R:1

1

1 2 3 4 5

Impact

R:1

R:2

Example EHM Orbiter

Estimate uses parametric cost model based on the Team X 50th-percentile estimate

Cost risk analysis indicates that proposed mission has a high likelihood of success Estimated cost with

reserves is 70% to 76%.. Typical NASA goal is 70%.

Identified risks consumes less than 1/3rd of planned reserves leaving sufficient reserves to cover ‘unknown-unknowns’

The 50th percentile team X estimate becomes 36% when the identified risks are taken into account

Risk-Adjusted Probabilistic Cost Distribution (S-Curve)

15