identifying the most probable cost – schedule values from

Identifying the Most Probable Cost –Schedule Values from a Joint Confidence

Level (JCL) AnalysisNASA Cost and Schedule Symposium

Goddard Space Flight Center August 14 -16, 2018

Sam Steiman, P.E., MPR Associates, Inc.David Hulett, Ph.D., FAACE, Hulett & Associates, LLC

Background

• Integrating cost and schedule risk analysis using Monte Carlo simulation yields a scatterplot of internally consistent pairs of cost and finish dates for a project or phaseWhen cost and schedule are not highly correlated the JCL values, say JCL-70, to achieve both objectives will imply later dates and more cost than the P70 values for cost and finish date when examined individually The risk analyst is faced with multiple possible JCL-70 date and cost combinations that yield a JCL-70. The challenge is to pick the right combination of values to show to management

•

•

•

Background

• The question is: “Which combination of cost and finish date which with a JCL-70 value for both objectives is most likely to occur?” Usually the analyst “eye-balls” a selected JCL-70 point, approximating

the densest point in the scatter plot, but that is not convincing for

budgetary purposes

Some people draw the regression line through the time-cost scatter

and pick the intersection of that and the JCL-70 “necklace”

•

•

• Linear regression analysis answers a different question and is not reliable as a

measure of the most likely cost – finish date combination.

Objective

• This presentation shows an innovative way to evaluate scatterplot data to allow the analyst to recommend the most likely cost and schedule targets at a desired JCLProbability density function (histograms) for cost or schedule examined individually show the number of iterations of the Monte Carlo analysis falling into cost or schedule bins. The most iteration results in any bin is arguably the “most likely” result

•

Objective

• In this presentation a 3D histogram of the scatterplot data can be created showing the number of iterations in bins of a combined cost and schedule grid to identify the most likely result for a desired JCLThe intersection of the greatest number of iteration results on the 3-D histogram with the JCL-70 “necklace” clearly shows the cost and schedule values most likely to occur at that JCL Data representing both highly correlated cost and schedule projects as well as non-correlated and bimodal results are presented to illustrate this methodology

•

•

• We produce more defensible most-likely JCL-70 results than using the regression analysis or simply “eye-balling” the scatterplot data

Refresher – JCL-70 versus Individual P70 Values

Inspiration for innovation …

•

•

Avoid the temptation to say “I want to choose an earlier date and will take on a little more cost to achieve that date”Imagine the scatterplot “as a three-dimensional range of hills with the highest points ranging from lower-left to upper-right snaking through the densest part of the scatter. These high points on the 3-dimensional view of this scatter are the most likely results.”

Concept

• Adapt 2-D histogram for a single parameter for cost or schedule to a 3-D histogram for cost andscheduleAllow user to adjust number of bins to optimize data visualization

•

Example 1 – Moderate Cost/Schedule Correlation

• Regression line intersection with JCL-70 “necklace” appears to reasonably predict most likely cost/schedule combinationThis is somewhat appealing when the cost and schedule are highly correlated

•

Visualization – Convert the 2D “Football” into a 3-D “Slug”

Illustrates the “S-curve” nature of the distribution (similar to the CDF curve) that is not evident from the 2-D “football”

Visualization – Convert the 2D “Football” into a 3-D “Slug”

3-D Histogram with 20x20 Bins

3-D Histogram with 50x50 Bins & Regression Line

Overlay JCL-70 “Necklace” on 3-D Histogram


Confirms regression line intersection with JCL-70 necklace is consistent with most likely cost/schedule outcome

3-D Histogram –JCL-70 (±1%), 20x20 Bins• Comparing Most Likely

values to Intersection Values:• Cost: ~$3M higher (for

$800M project)Finish Date: 2 weeks earlier (for 15-year project

•

• Conclusion: Regression line intersection with JCL-70 necklace is a defendable basis for selecting cost & schedule targets

Example 2 – Weak Cost/Schedule Correlation

• Regression line intersection with JCL70 “necklace” appears to be skewed from visual centerline of the distributionNote: Example 2 is the mitigated scenario for the Example 1 project

•

3-D Histogram with 50x50 Bins & Regression Line


JCL-70 (±5%) 25x25 bins

JCL-70 (±1%), 20x20 Bins• Comparing Most

Likely values to Intersection Values:• Cost: ~$10M higher

(for $735M project)Finish Date: ~4 weeks earlier•

• Conclusion: Regression line provides reasonable basis, but Most Likely values are more defendable

JCL-65 to JCL-75, 10x10 Bins

Most likely value is higher cost and shorter duration than predicted by intersection of regression line with the JCL-70 necklace

• Tuning the bin size can influence resultsConsider required resolution when selecting bin sizes and JCL range

•

Example 3 – Bimodal Scatterplot Results

• Regression line is obviously not a meaningful representation of the dataNote: Bimodal behavior is a result of a ~50% likelihood risk with significant cost impact

•

3-D Histogram, 50x50 Bins


JCL-70 (±10%) 10x10 bins

JCL-70 (±1%), 20x20 BinsConclusion• For bimodal results,

regression line intersection is obviously not meaningfulMost Likely values provide the only defendable basis for selecting cost & schedule target valuesFor reference, differences are:

•

•

• Cost: ~$4M higherFinish date: ~3mo earlier

•

Conclusions• For well-correlated cost/schedule scatterplot results, the regression

line intersection with the JCL-70 “necklace” is a reasonable representation of the most likely cost/schedule outcome

As results correlation becomes weaker (or bimodal), the regression line becomes less representative•

• For two scenarios at JCL-70, regression line intersection predicted lower cost and longer duration than most likely values

• 3-D histogram visualization of the cost/schedule scatterplot data can provide the risk analyst with valuable insights into most likely outcomes• “Dialing in” the appropriate bin size can influence the results

identifying the most probable cost – schedule values from

Documents