probabilistic cost, schedule, and risk management
DESCRIPTION
All variables on projects are random variables. Cost, Schedule, and Technical performance interact with each other is statistical ways to produce probabilistic outcomes for their values. Managing a project to a successful outcomes requires not only understanding the underlying statistics, but forecasting outcomes from these interactions in enough time to take corrective actions.TRANSCRIPT
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
1/186
This briefing is an overview of the
probabilistic risk analysis
processes that can be applied to
our program. Although it may not
appear to be a “simple” overview,
this material is the tip of the
iceberg of this complex topic.
Just schedule analysis has been
addressed in detail here. The
cost aspects of forecasting and
simulation must be addressed as
well to complete the connections
between schedule and cost.
Probabilistic cost will be surveyed
here, but an in depth review is for
a later time.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
2/186
An important aspect of education and research in our business domain, is “Fair Use” copyright law.
All the material in this briefing is accessible through the internet. Conference proceedings journal articles, company white papers and other public sources form the basis of much of this material and are referenced in the bibliography.
Some materials in this briefing make references to other copyrighted materials in the course of research, investigation, and analysis. These references are solely intended for non–commercial use and as such have no intent to infringe on the copyright holder. All attempts have been made to acknowledge the original copyright holder in pursuit of fair use laws as currently defined in the United States.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
3/186
The concept that risk and the
management of risk is a
desirable part of our program is
not always appreciated or well
understood
Without risk there can be no
opportunities. The plans for the
program become static and
deterministic.
While risk and opportunity are
related, the management of risk
is not the complement of
opportunity. - even if this is a
popular notions these days.
See the Conrow, AT&L article for
detailed discussion of this
somewhat controversial topic.
The primary opportunity in
Programmatic Risk Management
is the avoidance of being late and
over budget on the planned
launch date.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
4/186
When we use the term “risk tolerant IMS” it means a plan and supporting that can tolerate risks. Technical risks and programmatic risks. These risks are built into the program by its very nature. These risks must be addressed both technically and programmatically.
The real challenge though is not how to address them, but how to recognize that they are being addressed in a manner that actually reduces the level of risk as the program proceeds along its path to final maturity.
A measure of “increasing maturity” is the reduction of risk made visible to the evaluator of the IMS.
The materials here guide us through the process of building a risk tolerant IMS. But putting it to work still requires practice.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
5/186
The credibility of the Integrated
Master Schedule (IMS) is the
critical success factor for both
our proposal and our execution
phase after the win.
Without a credible schedule and
the related cost credibility, there
is a low probability of a win.
The effort put into constructing a
credible schedule during the
proposal phase will pay off
(assuming the program structure
remains intact) during the
execution phase.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
6/186
The skills of creating and
managing a schedule and the
associated cost require special
understanding.
However, the planners are
usually the last in a long line of
“culprits” for finding the cause of
any failure.
This is a “no win” situation.
People skills, project
management skills, and some
level of technical skill is needed.
But most important is the people
skill, since the knowledge of how
to assemble a successful IMS
resides in the minds of others.
Getting this knowledge out and
on paper requires interpersonal
communication as a primary
process, not technical tools and
formal processes.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
7/186
Understanding the difference
between qualitative and
quantitative risk assessment is
important.
Our first approach is usually
qualitative.
But what is needed is
quantitative.
A specific measure of
programmatic risk, is the impact
of the mitigations or risk
retirement activities and measure
of the increasing maturity of the
program deliverables in the
presence of risk.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
8/186
Programmatic risk management
makes visible the technical risk
mitigation steps as well as the
alternative programmatic
processes in the presence of
these risks.
Alternative branching in the IMS
must be defined to a level of
detail that instills confidence that
the IMS properly represents a
“risk tolerant” plan.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
9/186
Since there is quite a bit of
material here, a quick overview
will get us started.
The executive overview should
leave the reader with a sense of
the important topics
• There are no point estimates
allowed in planning. All
estimates must be
probabilistic
• There are core issues with
simple (deterministic) PERT
and it is not to be trusted
• The use of a probabilistic tool
is useful, but understanding
how the underlying statistic
works is critical to its use in
planning and program
execution
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
10/186
When asked “why are we doing
this?” many would answer –
because our customer wants us
to.
This would be too simple an
answer.
The main reason is, most
programs are simply too complex
not to have a better
understanding of how the
programmatic and technical risks
interact.
Not understanding the interaction
between these two types of risk
that creates the biggest risk.
Individually these risk “could” be
managed. But when combined
they behave in unpredictable and
maybe unknowable ways.
This is a core feature of any
system. See Systems Bible
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
11/186
If we get only two concepts out of
this briefing they should be:
• There are multiple critical
paths in any executing
program. Asking “what is the
critical” indicates that the
questioner does not
understand the probabilistic
nature of the program
• PERT is a poor estimating
metric. It has built in biases
which under estimate the total
duration of the program.
Monte Carlo is a better
estimating tool, but it too
needs careful adjustment
before realistic numbers can
be derived.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
12/186
The DID–MGMT–81650
describes the Integrated Master
Schedule.
Integrating Programmatic and
Technical risk identification and
mitigation adds credibility to the
IMS and therefore to the overall
program.
Applying probabilistic risk
analysis to the IMS is mandated,
but care is needed to interpret
the results.
These tools aid in the evaluation,
but they are not replacements for
good program management
processes.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
13/186
The idea that uncertainty and the
risk that it produces can be
“programmed out” of the
schedule is a false hope.
Without understanding the
principles of Deming, the
management and the planning
staff will be “chasing their tail,”
trying to control the naturally
occurring variances in the plan.
The first approach is to set the
error bands wide enough to not
trigger an exception report for
these variances.
This approach is “good enough”
but what is missing is the
knowledge of “how wide is wide
enough?” for a specific set of
tasks or during a specific phase
of a program?
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
14/186
The first step in the process of
adding credibility to the IMS is to
recognize that all task completion
times are random variables.
They are not “point” numbers
(scalars) but are “estimates” of
the completion time drawn from a
probability distribution of the
underlying population of all
completion times possible for the
specific task.
Modeling schedule durations are
random variables does not imply
these durations are “random.” It
reflects how a duration’s
uncertainty is influenced by the
underlying probabilistic nature of
the activity network.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
15/186
Building a credible IMS starts
with identifying the architecture of
the IMP and the supporting tasks
in the IMS.
Although this is restating the
obvious the process to do this is
actually quite hard.
Adding schedule and cost risk
identification and mitigation to the
process is the minimal result for
a winning proposal.
It cannot be emphasized enough
– the architecture of the IMS is
critical to identifying a risk
tolerant schedule. The “rats nest”
approach is simply unacceptable
to the success of any program.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
16/186
The goal of introducing
probabilistic schedule and cost
analysis is to improve the
probability of a “win” on the
proposal.
While winning is important,
executing the program is even
more important.
What ever “credibility” elements
were in the proposed IMS need
to be carried into the execution
schedule.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
17/186
The use of Monte Carlo for
assessing the IMS must be
turned into forecasting
performance.
This is done by identifying the
“hot spots” in the IMS through
sensitivity analysis, interventions
for these “hot spots” and the
measure of change resulting
from the intervention.
The important concept is to
connect metrics to measurable
benefits to the program. Without
this the creation of metrics is just
wasted effort.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 18/186
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
19/186
Using risk and uncertainty as an
integral part of the planning
process is a sign of maturity.
Making decisions on the this risk
information improves maturity.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
20/186
When we speak of risk management, either technical or programmatic, the term usually has a very localized context.
For the planning context risk management must include both technical and programmatic risk.
The technical risk aspects come from external sources but are directly represented in the IMS.
The programmatic impacts of this technical risk must be explicitly addressed.
This is the easy part.
The hard part is determining the implicit programmatic risk that is derived from the technical risk and the risks that are derived from the “architecture” of the program itself.
This is where the true “risk tolerant” IMS adds value.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
21/186
There are many approaches to
building a risk tolerant IMS. Our
current approach is to add risk
factors and margin to specific
areas of the IMS
The current approach to use a
Monte Carlo tool to assess where
this margin should be placed.
There are several other steps
along the way. Which steps to
take, how much effort to invest
and how to recognize the value
of this investment are some of
the management challenges as
well as the technical challenges.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
22/186
The difference between risk and
uncertainty needs to be
understood at some level.
For the most part the differences
are not important in the
beginning.
But once decisions start to be
made about mitigation steps,
branching probabilities for failure
modes, these differences
become more important.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
23/186
When we use the term
uncertainty or risk it means at
least 4 things.
First let’s sort out “uncertainty”
There are two classes of
uncertainty in large complex
programs.
• Static uncertainty emerges
from the natural variations in
the completion times of tasks.
This is a Deming uncertainty.
http://webserver.lemoyne.edu/
~wright/deming.htm is an
example of this type of
uncertainty
• The dynamic uncertainty is
about the unknowns and the
unknowable
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
24/186
The static uncertainty in a
program can be addressed
directly in the plan with mitigation
tasks.
The dynamic uncertainty arises
from the dynamic interactions
between the tasks of the plan.
This interaction and the
outcomes to the end date cannot
be modeled with static
paradigms.
Monte Carlo simulation is an
approach to modeling these
interactions and their impact on
other elements of the plan
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
25/186
Managing risk in the schedule requires anticipation to identify the risks, but also requires understanding of the source of risk, the impacts of these risks, and the interaction between the risks and the plan.
A process is needed to guide the risk management activities. This process must address both the programmatic as well as technical risk. The interaction between programmatic and technical risks must also be managed.
These interactions must be considered a “first order” interaction.
The common approach is to consider the technical risk as first order and the programmatic risks secondary.
The combination becomes a first order interaction.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
26/186
As planners our goal must be to
produce a plan that has credibility
and integrity.
Credible plans are believable
plans
Integrity plans are trustworthy
plans.
Both attributes are needed for a
winning proposal and the follow
on execution.
The successful assessment of
the IMS during a proposal or
during execution by the customer
or DCMA depends on how
believable the plan is and how
well it can be assessed to
confirm this believability
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
27/186
The assessment of the credibility
and integrity of the IMS can take
place by asking some questions.
These and similar questions
shine light on the underlying
attributes of the IMS in ways that
simple assessments do not.
These are not technical
assessment, like counting data in
the predecessors field, but are
architectural questions about the
“quality” of the IMS independent
of the technical details.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
28/186
NASA does risk management in
a specific way. We need to
understand their way as a
starting point.
Reading the NASA materials is a
start, but there is other research
available from conferences and
vendor web sites that needs to
be gathered and read as well.
Other government agencies as
well as civilian firms have similar
risk management approaches.
NASA’s approach is a good
starting because of manned
space flight’s inherent risk. And
NASA’s emphasis on Safety and
Mission Assurance.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
29/186
The IRMA tool developed at
NASA Johnson Space Center is
the basis of risk management for
a NASA side.
Although this approach is
focused on the technical risks the
programmatic risks appear in the
database.
As well there are other risk
management systems and
paradigms.
Active Risk Manager (ARM) is a
popular one as well,
http://www.strategicthought.com/
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
30/186
The NASA Risk Management
Summary Card calls out
“schedule” impacts in three
places.
Connecting programmatic and
technical risk is a critical success
factor for a proposal as well as
an execution assessment.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
31/186
Adding probabilistic schedule and risk analysis to the IMS can be done through a structured process.
1. The initiating event of the risk is identified.
2. The result from this event is described
3. The consequence that flow from the scenario are developed
4. The connections, flows, interactions and correlations between the scenarios are modeled
5. The probability of occurrence for each of these scenarios is developed
6. The model of the probability of occurrence and consequences from the occurrence are combined
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
32/186
The Continuous Risk
Management paradigm found in
the technical risk world can be
applied to the programmatic risk
as well.
NASA has adopted Continuous
Risk Management (CRM)
through several guidelines listed
here.
The table summarizes how CRM
is managed in a structured
manner throughout the program
life,
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
33/186
There is a difference between the
design evaluation of the IMS and
the risk evaluation.
The design evaluation describes
how the technical activities
needed to develop and deploy
the product – in this case a
manned spacecraft – must come
together in the right sequence to
make the planned completion
date.
The risk evaluation defines the
probabilistic completion model for
each task, the correlations
between the tasks and the
resulting probabilistic model.
This model is a Bayesian
Network of all the tasks.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
34/186
To construct an IMS with integrity
and credibility both technical and
programmatic risk must be
connected.
This process starts with the
identification of the technical
risks in ARM and their mitigations
in the IMS. This is the explicit risk
approach.
Next comes the explicit
programmatic risk activities. This
can be the well known margin
needed in front of major
milestones, program events or
deliverables.
Finally comes the implicit risk
mitigation activities that will be
needed to differentiate this IMS
from any other IMS to start to
build confidence that we have a
“risk tolerant” IMS.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
35/186
A pedagogical literature survey
from the RAND Corporation
supports the notion that
probabilistic risk assessment is
not seen in a favorable light by
management.
• It is too complex.
• The underlying statistic are not
will understood.
• “It’s the customers that are
asking for this.”
• There is little historical data to
calibrate the underlying
probability distribution functions
for task completion times.
All of these gaps must be closed
in some way in order to call our
IMS Risk Tolerant
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 36/186
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
37/186
Managing in the presence of
uncertainty is the core behavior
for any modern program.
Trying to control this uncertainty
requires two basic
understandings:
1. The natural variations in the
schedule cannot be
sufficiently controlled to
remove risk. These are the
Deming variations and the
foreseen uncertainties
2. The unforeseen uncertainties
and the inherent chaos of the
program must be dealt with
through contingencies
Attempting to manage
uncertainty is limited to foreseen
risk. Managing in the presence of
uncertainty deals with unforeseen
and chaotic sources of risk
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
38/186
When estimating the completion times for tasks, there are three primary problems.
1. A number produced by a CAM or an IPT must be a statistical estimate, not a specific duration.
2. The meaning of “best” must be established prior to accepting the statistical estimate
3. The collecting of the “most likely” estimates cannot be added in the sense of adding scalar numbers, since they are probability distributions.
4. The “most likely” is NOT the average completion time, it is the completion time that occurs most often from a large sample of possible completion times.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
39/186
The first approach to “planning”
the program is to ask the CAMs
or IPT Leads for each task in
their WBS or IMP/IMS area: “how
long with this take to do?”
The numbers that come back are
then entered in the duration field
on the schedule.
These numbers are not only
wrong they are dangerously
wrong.
They are “point” estimates that
live inside a probability
distribution.
The built in bias from the
approach has clinically be shown
to be optimistic or pessimistic,
but rarely “most likely.”
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
40/186
The traditional approach is to roll
up the single point estimates into
a sum of the durations and
search for the longest path.
This is the Critical Path Method
(CPM) for assessing the finish
date of the plan.
The problem of course is these
“numbers” are not actual scalar
values. They are samples drawn
from probability distributions.
Addition is not mathematically
possible in the sense of addition,
defined over the set of natural
numbers (0, 1, 2, … ∞]
These probability distributions
can be “convolved” into a new
probability distribution, but a
better approach is Monte Carlo
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
41/186
When asked “what is the most
likely” or the “best guess”
duration, the variety of answers
removes any chance of getting a
reasonable answer.
The meaning of “best” is
undefined in almost any situation
that has not taken explicit steps
to bound the answers.
Without calibrating the meaning
of “best” the planner cannot
bound the underlying probability
distribution of all the value that
are not “best” but could possibly
occur in the project
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
42/186
When we use a term “best” or
“most likely” there is an implicit
assumption – often not
acknowledged – that other values
than “best” and “most likely” can
occur.
This is the probabilistic nature of
the duration estimate. A single
value cannot exist.
The actual shape of the
probability distributions is what is
needed for generating the “best”
estimate.
Without this knowledge, the
planner is guessing in the dark.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
43/186
Here are some steps to
producing “educated guesses.”
This is a model based approach
which depends on the maturity of
the data that is the basis of the
model.
While this is a high level
description, it needs raw data
underneath to make it valid.
Without this data the “guess” is of
little value.
What is missing in most cases is
any historical trends for the IMS
elements.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
44/186
Playing the 20 questions game is
on approach to calibrating the
“guess” for the duration.
This approach will get an answer
to without 10% to 20% in a few
questions.
This is a way to start the
“conversation” about duration
when the participants have
convinced themselves that they
can’t come up with the answer
because there is not enough
information.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
45/186
Another approach is to classify
the fidelity of the information.
This can be done with a 1, 2, 5
approach.
Gathering estimates by asking
for durations is the preferred
approach.
Instead, making a risk adjusted
estimate – duration and
confidence interval provides a
better approach.
This approach neutralizes the
guessing game by asking a risk
question first, then the duration.
The classification of risk provides
the lower and upper bounds of
the task duration. Along with the
underlying probability distribution,
this forms the basis of
probabilistic schedule analysis
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
46/186
In all cases, uncertainty is the
normal mode for information
gathering.
When we ask a CAM or IPT for
an estimate and do not ask for
the risk associated with that
estimate and the confidence
intervals for that number we are
simply increasing the risk to the
program by absorbing unreliable
numbers.
This unacknowledged risk is
always present . By not making it
visible, the program is
mortgaging the future without
budgeting for the cost of paying
off the mortgage.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
47/186
Starting with a good topology for
the IMS is important. Not only
because the programmatic
activities need to be well defined,
but the sensitivity of the risk
analysis depends on a “properly
formed” IMS.
If the logic of the IMS is ill–
formed than the results of the risk
analysis will also be ill–formed.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
48/186
There are several elements of
the probability model for duration.
Not only are the activities from
the IPTs and CAMs important,
but the subcontractors play on
important role.
The data from the subcontractors
includes:
• Durations and the
probabilities
• The internal connectivity of
the activities that produce the
external; “milestones”
conveyed to the prime
contractor.
• The other programmatic risk
factors for the performance of
subcontractor work
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
49/186
Although the formalities of the
probabilistic risk analysis are not
needed for this briefing. Here is
some background on
terminology.
If we are to learn to “speak” in
probabilistic programmatic risk,
these terms should become
familiar.
This is an almost endless topic,
but some understanding of
probability and statistics is
needed.
This of course requires some
effort and patience .
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
50/186
We should not be drawn into the
illusion that the Central Limit
Theorem is operable for the
program.
This is the core assumption of
PERT and CPM based planning.
This requires normally distributed
completion time and
independence between tasks.
Neither can be verified in
practice.
As such the impact of making
these assumptions is “whistling in
the dark.”
The result is that the program is
late before it starts.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
51/186
The Central Limit Theorem can
be useful in many cases. But it
needs to be understood where it
is not useful.
The assumptions of the CLT
applied to the PERT problem
mask even more problems when
naively applied to estimating the
duration of a program.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
52/186
The core of the Central Limit
Theorem of the production of a
Gaussian probability distribution
by assembling a collection of
arbitrary probability distributions.
The primary assumptions that
these distributions are
independent provides the basis
of the CLT.
If the activities represented by
the arbitrary distributions are not
statistically independent – which
is hardly ever the case on a real
project – then the assumptions of
the Central Limit Theorem are
false and the probability
distribution of the program
completion time is no long
Gaussian distributed
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
53/186
What happens is the rollup of the
most likely times of the critical
path activities is biased to an
optimistic location in the
probability distribution of the
project completion distribution.
This is the fundamental reason
PERT is not very useful.
This criticism is only partly true. If
a probabilistic PERT approach is
used or a Bayesian network
approach is used, then the
deterministic issues are
removed.
But it is easier to use a Monte
Carlo simulator since this avoids
gathering all the underlying
probabilistic distribution
information for an initial estimate
of the completion time of the
program
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
54/186
The probability distribution
function describes the frequency
of occurrence of the events in the
underlying statistical process –
say the duration of a task
completion, the roll of a die, or
the time it takes a light bulb to
burn out.
The ordinate of the graph (the y
axis) is normalize to a scale of [0,
1] which represents the
probability percentage 0.10 =
10%
The abscissa represents the
range of values that can be found
in the underlying sample
population. In this case [0.0, …,
5.0]
The mode is the “most likely”
value to occur when samples are
drawn from the distribution.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
55/186
The standard deviation is a
description of the “spread” of the
probability distribution function
around the mean.
Without understanding the
standard deviation ,a point
estimate or even a sampled
estimate is of little value.
The shape of the probability
distribution is also important in
understanding the confidence in
a single number. These “higher
order moments” will be discussed
later, but for now no estimated
number should be used without
the standard deviation value
being attached.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
56/186
Looking at the population
statistics of a random process is
not very useful. Humans have a
hard time making any sense from
the graphs.
The Histogram view can show
the frequency of occurrence of
the various values – how often a
specific value occurs in the total
population of value or the
sampled population of values, but
more insight is needed.
The Cumulative Probability
Density is a way to show this.
The CDF shows the probability
that a sampled number drawn
from the population of all possible
numbers
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
57/186
Various Probability Distribution Functions (pdf) have similar Cumulative Density Functions (CDF).
This is important for several reasons:
• The underlying probability distribution function has great influence on how the end point values are weighted. This has impact on the PERT formula
• The cumulative distribution is the source of random numbers in Monte Carlo. For a variety of pdf’s, similar CDF’s are generated, neutralizing the differences in the pdf’s. Monte Carlo isolates these underlying differences. This may be good or bad depending in the need.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 58/186
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
59/186
Any estimating process must
address the probabilistic
boundaries of the estimate.
Without this, planners and cost
estimators are hopelessly under
or over estimating duration and
associated cost.
The real issue is not over or
under estimate, but not knowing
which one it is or why.
This lack of knowledge about the
underlying statistical process
creates a greater risk.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
60/186
Making decisions in a risk neutral
manner is not advised.
We should always talk about risk
adjusted decisions, risk adjusted
values, and risk adjusted
outcomes.
The difference between
alternatives, uncertainties and
outcomes also needs to be
understood. They are not
interchangeable concepts
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
61/186
Decision making must address
the different types of uncertainty.
Understanding how these
uncertainties impact the decision
is critical to selecting alternatives
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
62/186
The idea that we can produce
“estimates” about the future in
the absence of models, historical
data, or a methodology for
discovering these models or
historical data is common in the
IMS planning realm.
Forecasting the future is sporty
business.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
63/186
The IMS contains “branches”
where the path of work makes a
change in direction.
These braches can be modeled
with a decision tree paradigm.
The risk management discipline
uses this approach. And it is
applicable to the construction of
the probabilistic branching found
in the network of tasks in the
IMS.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
64/186
In the “olde days” the line of
balance chart was used to
forecast the cost at completion.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
65/186
PERT and the Critical Path
Method are called out as explicit
methods to be used in the
planning process.
The formulas for PERT are
simplified models of the
underlying complexity of
probabilistic networks (Bayesian
Networks)
As such they have little or no
connection to the reality of the
IMS
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
66/186
In deterministic PERT the
durations are defined as a three
point estimate and the PERT
formula is used to compute the
mean and standard deviation for
the program duration as well as
the critical path.
This is the algorithm used in
Microsoft Project when the PERT
tool bar is turned on and the
three point estimates entered into
the appropriate columns.
It is billed as probabilistic but in
fact the 3–point estimates work
against a fixed probability
distribution function with no way
to adjust its shape, bounds or
moments.
As well, there is no way to insert
the correlations that naturally
occur in the IMS.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
67/186
The estimates produced by the
deterministic three point data can
be used to construct a
probabilistic PERT if the
underlying probability
distributions are defined for each
task completion time.
The development of the
probability distributions requires
historical data as well as an
understanding the behavior of
each node in the network
(coupling).
This is a difficult task without the
proper tools and data sets.
With the Risk+ tool, individual
distribution functions can be
assigned to each task. But the
“tuning” of each function is
difficult.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
68/186
When we speak of probabilistic
risk analysis, we also need to
speak of the statistical nature of
the activity network.
When we speak of a probabilistic
activity network (a Bayesian
network) we also need to speak
in terms of probability.
A question that can be asked of
the network is – “ what is the
probability of completing this task
by a certain date?”
A second question that can be
asked is – “what are the
underlying statistics of the
activities of the network?”
A final question that needs to be
asked is “what is the inherent
uncertainty in these estimates?”
In other words – how good is our
ability to guess in the presence of
a statistical process?
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
69/186
Once the activity durations are
treated as probability distributions
it can be seen that they can not
be “added” in the normal sense
to produce a program duration.
They must be “summed” in the
probabilistic sense. This can be
done with Monte Carlo or with
convolution of the Cumulative
Distribution Functions.
Again, add to this the correlation
issues (one task influencing the
outcome of another task), and
the simple approach of adding
the durations to come up with a
total duration falls apart.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
70/186
Here’s another look at distribution
functions.
This approach should be the
standard vocabulary for
discussing the IMS duration
estimates.
The topological integrity of the
IMS is important, but just as
important is our understanding of
how the activity durations have
been developed, their confidence
interval and the underlying
distribution of the values the
durations can take.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
71/186
A missing element is the
statistics of the “events” that
occur during the execution of the
program.
For example if a fixed date is
defined in the IMS (this is very
usual for things like IBR, PDR,
CDR), what is the underlying
probability distribution of the
confidence of that date.
The same is true for
subcontractor provided dates,
where the details of the
deliverables is not visible.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
72/186
With the input probability
distributions, the program
schedule can be treated as a
“system” with a response
function.
The “system” is a Bayesian
network where the elements of
the network are probabilistic and
the driving function is
probabilistic.
The “output” of the system is
therefore probabilistic as well.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
73/186
One probability distribution
commonly found in scheduling is
the Beta Functions.
This is a “tunable” probability
distribution function that has
been shown to closely match the
behavior of task completion
durations.
The term “closely” needs to be
used with care. The deviations
between actual completion times
and the “model” of completion
times needs to be assessed
before confidence in the
probabilistic results can be
useful.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
74/186
The Beta distribution is used for
PERT estimates. This use is
many times done with no
understanding of the shape or
the dynamics of the probability
distribution function. Beta is a
selection for Risk+ as well, with
no obvious way to change the
shape of the curve.
Some understanding of the
impact of the Beta function on
the outcome of the PERT formula
or the Monte Carlo simulation is
needed.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
75/186
There are many alternatives to
Beta. The Triangle distribution is
one. The triangle distribution has
an intuitive appeal due to its
simplicity useful for estimating
task durations.
But the triangle distribution still
has the problem that the most
likely value and the expected
(mean) value are not the same.
So when planning asks for the
“most likely” value many people
respond with the Mean, which
biases to result in the optimistic
direction.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
76/186
The triangle distribution can
better describe some statistical
processes, but it too needs
“tuning” for specific task duration
processes.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
77/186
BetaPERT is currently the vogue
in the probabilistic analysis world.
The BetaPERT distribution
provides a “tunable” curve where
the most likely “Mode” is near or
identical to the “mean” of the
distribution.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 78/186
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
79/186
The challenge to building a risk
tolerant IMS is the initial capture
of the task durations and the
sensitivity of the IMS to
correlations between tasks.
There is a optimism bias created
when a CAM is asked “what is
the duration?”
The answer is usually a “mean”
(average) duration rather than
the “Mode” (most likely).
If the Mean is used in place of
the Mode, then the three point
estimates are biased to start with
without the explicit knowledge of
the planning staff.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
80/186
The general flow of creating a
risk tolerant IMS looks like this.
The critical aspect is to get the
CAMs to identify the embedded
risks and the mitigation tasks for
those risks.
Once this is done, “planning” can
then assess if the mitigation
processes make sense in terms
of supporting the AC’s and SA’s
of the IMP/IMS.
Constant and continuous
feedback is needed for this to
work properly.
Without this feedback, the IMS is
assembled in the absence of the
knowledge base and the risk
tolerant aspects are lost or
become confused with the
mainline activities.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
81/186
The capturing of the risk
information is an interactive
process. A Kaizen is one way to
do this and probably the best.
Having the CAM fill out the “most
likely” durations and identify the
risk mitigations cannot be done
without direct contact.
Without this direct contact,
planners have not chance of
intervening in the process and
the IMS becomes a collection of
tasks rather than an “architected”
plan.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
82/186
The 3 point estimates required by
DiD–81650 have a variety of
uses.
They can be simple values used
for PERT calculations. These
calculations can be “made up” by
the IPT lead and entered into the
schedule.
A risk adjusted value can be
used from the Macro in Risk+.
The CAM or IPT Lead states the
relative risk in a number between
1 and 5. The macro defines the
percentage boundaries for the
classified risk.
Individual risk ranges can be
developed from historical
information. This is the best
approach, since it represents the
past.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
83/186
There are several classes of
programmatic risk. Although the
Pareto chart shows that scope
change is the most common,
delays are also common. These
come from the customer side
most often as well.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
84/186
The classification of risk results
in a percentile or quartile
classification scheme. This is a
better approach than asking
someone for the minimum and
maximum durations.
The challenge is to calibrate
these ranges in a meaningful
manner for the specific program.
There can be general
classification ranges, but having
them set for the specific program
is much better.
This of course requires that data
is kept from past programs,
normalized and then made
available in a form useful for
probabilistic risk analysis.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
85/186
During the data capturing
process where estimates are
extracted from the technical
experts, there is a natural
tendency to accept the numbers
at face value.
Without qualifying the numbers in
some statistical form, this
information is absorbed into the
IMS or Cost and becomes “fact.”
These “facts” then progress
through the program and are
never challenged for their lack of
statistical basis.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
86/186
The core problem with capturing
estimate from human beings is
they are biased.
Either negatively biased or
positively biased.
There is plenty of literature on
this effect and ways to overcome
it. For now we’ll just live with the
outcome of the bias
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
87/186
Let’s take another tour of the problems with PERT. These issues are well documented in the literature, but poorly understood in practice.
The poor understanding comes from the difficulty of the explanation – statistical conversations are usually not very interesting; and the natural tendency to look for easy answers to complex problems.
The core issue is that without a deep understanding of the errors produced by the PERT equation, the confidence in completion dates and the risk tolerance of the IMS is difficult to build.
When the actual numbers come in (ACWP and BCWP) and they don’t match the expectations – is it the original plan or the underlying performance?
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
88/186
There are several myths about
PERT. The first is that is was
scientifically thought out in detail.
This is not the case. The book
The Management of Projects,
Peter W. G. Morris provides the
background on this development
as well as other project
management histories.
The second historical myth is that
PERT is a general purpose
approach. In fact it is very
specialized and is applicable to a
narrow range of activity
networks. Those with normally
distributed completion times,
statistically independent
relationships, ones where the
critical path does not change and
with the “most likely” estimate
actually representing the “mode”
of the underlying probability
distribution function.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
89/186
When a manger asks “what is the critical path for this program?” there are several thoughts and actions:
• In a probabilistic activity network there are many critical paths, which change as a function of time, adjustments to the risk profiles, and the completion of work.
• Correlated activities are influenced by off–critical–path activities to place them on the critical path.
So the answer to the management question is “it depends on what you mean by critical and path.”
The real answer only comes by moving away from the static representations of the IMS to a probabilistic representation – and that requires much more effort.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
90/186
Once we recognize that the
activity network is probabilistic in
nature, the first choice (the naïve
choice) is to apply the PERT
method.
While this may be a useful “first”
choice it produces results that
are overly optimistic and
sometimes overly pessimistic.
Either way they are wrong from a
statistical point of view. They are
wrong because the assumptions
of PERT are wrong. These
assumptions are almost never
found to be true in practice.
Even if they were true, the
probability distribution function
used by PERT does not
represent any useful activity
completion time distribution.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
91/186
One of the “killer” assumptions in
PERT is the lack of
understanding of “merge bias.”
Merge Bias occurs when two or
more activities are joined at a
merge point. Usually a milestone
or a simple Finish to Start of
several tasks.
The result is the statistical
behavior of the activities prior to
this merge point influence the
statistics of the following
activities in undesirable ways.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
92/186
Since statistical distributions can
not be simply “added” the
duration of the downstream
activity is not the sum of the
duration of all the upstream
activities (or the longest activity).
Instead it is the statistical sum
(convolution) of the probability
distribution function (pdf)
Without understanding this, the
PERT estimate generates an
optimistic estimate of the
duration, since the PERT formula
simply adds the durations to
arrive at the total duration.
The PERT formula also adds the
individual activity variances to
arrive at a total project variance.
While this provided a simple
method to “guess” the total
duration it produces a poor model
for real analysis of risk.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
93/186
The PERT approach fails to
consider the “random variable”
nature of the dates in activity
network.
As well the correlation between
each of these random variables
is not considered.
The result is the potential for
large variances in the completion
time estimates – 15% is not
uncommon.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
94/186
The visual impacts of Merge
Point bias is show here. This is a
small and sample activity
network. A “real” network would
have different outcomes.
It is not important exactly how the
merge point bias impacts the
final completion date, but that the
merge point bias DOES impact
the final completion date.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
95/186
How the activity network is
arranged has significant impact
on the calculations for PERT.
Here are some examples.
Notice that the PERT mean (the
average) stays the same, while
the “real” mean and the variance
on that mean change
dramatically depending on the
arrangement.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
96/186
The reason for these changes
involves how the statistics are
“added” in the various
configurations.
The critical concept is that the
PERT calculations are unreliable
as a predictor of the completion
time in a probabilistic model of
the activities.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
97/186
The effect of the merge bias is
shown in the graph. It is unlikely
in any real plan that only three
parallel paths exists. This number
is usually much larger,
sometimes in the dozens.
All of this discussion is leading to
the suggestion that PERT is not
viable on any complex program.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
98/186
Use Monte Carlo, don’t use
PERT.
The problem of course is that
DID 81650 and even the
corporate guidelines either
require or strongly suggest the
use of PERT and CPM.
This can be done of course, but
don’t use the numbers for any
real planning processes.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
99/186
The use of Monte Carlo
simulation is a logical outcome of
the problems with PERT.
What is missing is the
understanding of how Monte
Carlo works, what it’s limitations
are, where it should not be used
and of course how to interpret
the outcomes when they don’t
meet our expectations.
Even though Monte Carlo is a
powerful tool it can produce
unexpected results. This section
is an attempt to give some
background on the mathematics
and stimulate further interest in
applying this tool to the problem
of schedule forecasting
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
100/186
Monte Carlo simulations provide
a useful approach to modeling
schedule risk. But their value is
more than that.
Unlike PERT or other
deterministic approaches – even
though the three point estimates
are billed as probabilistic – Monte
Carlo examines the schedule
network independent of a critical
path, topological constraints or
other “human induced” problems.
It looks at the network as a
collection of nodes and arcs,
independent of the “meaning” of
this information and produces a
model of the behavior of these
nodes and arcs
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
101/186
The concept behind Monte Carlo
is to sample the possible
durations for a task from the
population of all durations and
apply them to the schedule.
The population of possible
samples is defined by the
Cumulative Probability Density
(CDF) function for each task.
This in turn is defined by the 3–
point estimate for the task, which
selects the bounds in the CDF for
sampling.
Since there is no direct concept
of a Critical Path in Monte Carlo,
the near critical path tasks are
considered in the analysis of the
completion time.
As well the PERT biases
produced by the simple minded
PERT formula are avoided as
well.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
102/186
There are several “components”
to the Monte Carlo process. So
when we speak of Monte Carlo it
is both a process and a product –
in our current case Risk+
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
103/186
Samples drawn from the
underlying distribution function
can produce an “error estimate”
on a completion date.
These error estimates are
different than the fixed
boundaries for PERT, since they
represent the actual probabilities
distribution error bounds
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
104/186
The number of sample runs
needs to be sufficient to cover all
the possibilities in the pdf.
This is usually 500.
A production run for a Monte
Carlo simulation is around 2,000
to 3,000 iterations.
As the iteration count increases
the fidelity of the simulation
increases.
But there is a point where more
samples don’t add value. This
point can be determined by the
statistical performance of the
variance of these sample space.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
105/186
Since Monte Carlo does not need
to know about the Critical Path, it
is conceptually simpler to use.
A well formed network is needed
and the 3–point estimates need
to represent the proper risk
assessment.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
106/186
This is a view of how Risk+ sets
up the project file.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
107/186
The result is a cumulative
distribution and a probability
distribution function.
Interpreting this result is straight
forward.
The confidence of each date is
shown in the table on the right.
This is the probability of
completing the task by the date.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
108/186
A good IMS is needed.
The risk assessments should be
done with a ranking process
rather than specific 3–point
estimates. This disconnects the
personal opinions from the
assignment of risk.
A 5 level process is one
approach, but any odd numbered
level ranking is best.
The differences between the
levels should be geometric not
linear.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
109/186
Risk+ generates lots of
information useful for the
analysis of the program.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 110/186
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
111/186
Constructing a robust IMS means
building a “risk tolerant” plan.
The robustness of the plan
means that it (the plan) can deal
with disruptions that occur
naturally through the course of
execution or un–naturally through
external events.
In either case the “robustness” of
the plan must be visible to the
evaluator without any detailed
explanation, beyond the IMS
narrative. No hand waving
explanations of how the plan
works. The risk tolerant aspects
most be obvious.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
112/186
Thinking about schedule
contingency is different in a PRA
context. For a simple project,
15% contingency is assumed.
But placing the contingency is
the first problem. The process is:
• Run Risk+ and watch the final
date.
• Compare the 80% confidence
date against the deterministic
date. This difference is the
first cut at the needed margin.
• Assign this duration across
the project in front of the
critical (high risk) milestones.
• Rerun Risk+ and add or
subtract this margin until the
desired confidence date is
achieved.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
113/186
More detailed statistics and
interpretations of the results an
be produced with Risk+.
This information can them be
used to perform further analysis
of the IMS. The analysis is what
we’re after, not just the date
produced by Risk+.
Like the PERT numbers, the
Risk+ numbers must be
interpreted with the
understanding of how they were
arrived at.
This is one of the purposed of
this briefing – to provide
knowledge of how to use this
approach and what its strengths
and weaknesses are.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
114/186
Incorporating Technical
Performance Measures (TPM)
with Monte Carlo is a powerful
way of showing how risk is
reduced and maturity increased
in a program.
At each step in the program –
each Program Event – a target
confidence interval for a
completion date can be forecast.
Along with the technical
performance measure, this
programmatic performance
measure approach results in a
“risk tolerant” IMS.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
115/186
Risk tolerance in the IMS
requires more than just the
planning processes. It requires
the connections to technical and
cost.
This has been stated before, but
it needs to be made not only
visible but actionable.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
116/186
Using a simple process steps,
risk tolerance can be developed
from the same processes used
by the technical risk engineers.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
117/186
The goal here is to move the
integrated risk tolerance –
technical, schedule, cost –
forward from a dis–integrated
plan to an integrated plan
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
118/186
Read the chart as follows: The upper horizontal band on the plot is “Ready Early”. “Ready On–time” is the middle band that also spans the launch window. “Ready late” is the lower band, which means a 6–month slip to the next launch window and all associated costs that go with that slip. The upper line plotted is the deterministic
completion date (i.e. no risk) and the lower line plotted with the 20th and 80th percentile confidence bands on the risk–adjusted completion date. The project’s objective is to continue to invest in risk mitigation actions until the band and the area of highest likelihood is no longer in the “Missed Launch Period” area of the chart. Note the improving trend over time indicating the success of the risk mitigation actions as well some “Accepted” risks passing their exposure window without becoming problems.
Taken from [Risk Based Decision Support techniques for Programs and Projects] http://www.futron.com/pdf/RBDSsupporttech.pdf
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
119/186
As Program Events progress the
risk mitigation processes need to
progress as well.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
120/186
Here’s a 4 step progress for
installing risk in the IMS and
producing a “risk tolerant” plan
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
121/186
The use of branching
probabilities is important for the
assessment of the “risk
tolerance.”
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
122/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
123/186
The use of Risk+ and Monte
Carlo replaces the PERT
approach to schedule duration
probability analysis.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
124/186
The “goodness” of the IMS is
important to the quality of the
results
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
125/186
The distribution to use for a task
depends on the underlying risk
profile.
Triangle is common, but it over
biased the risk on the high end.
Beta can be used, but the simple
Beta distributions in Risk+ may
not represent the real risk profile.
BetaPERT is the better one, but
Risk+ does not support it.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
126/186
Which tasks drive the sensitivity
of a completion date needs to be
understood. Not all tasks have
the same impact on the outcome.
The “tornado” chart is one way of
showing this.
The Power Law’s behind Pareto’s
rule is worth understanding for
many reasons, not just schedule
and cost modeling. Power Laws
occur across a wide variety of
domains, from moon crater sizes
to the frequency of words in
English.
http://www.nslij–
genetics.org/wli/zipf/ is a good
place to look for the impacts of
Power Laws on everyday life.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
127/186
In order to build a model of the schedule we have to start with the schedule. But first we have to start with the model of the schedule.
This is the role of the IMP, but the connections between the ACs are needed, not just the list of the IMP elements.
From this model the schedule elements can be arranged to follow the strategy of the IMP rather than represent the passage of time and the consumption of resources.
From there a model of the risk areas, mitigations, parallel development paths, reevaluation points, and “hot spots” (sensitivity analysis) can be extracted. This information can them be used to assess the robustness of the IMS
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
128/186
The primary graphic for an IMS
evaluation is the cumulative
probability of a completion time.
This is technically referred to as
the Cumulative Density Function
(CDF)
This is the format most useful for
answering the question – how
long will this take?
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
129/186
The confidence intervals
produced by the CDF can be
assessed over time against
targets.
These targets can be Technical
Performance Measures or any
other style of metric that is
connected with cost, schedule
and technical performance
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
130/186
Another view is the confidence in
the schedule dates as a function
of time.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
131/186
It is important to understand the
sensitivity of a completion time to
the various “drivers” of this
sensitivity.
This makes visible the “hot spots”
in the IMS that require attention,
mitigation, or even re–planning to
reduce sensitivity
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
132/186
In Monte Carlo each task can
take on a wide variety of roles. It
can be the driver for the total
schedule duration at one time,
and at another time (in the
simulation) have little effect on
the outcome.
The Criticality of the task is how
“important” it is as a function of
the number of simulation runs.
The higher the criticality of the
task, the more important it is to
look at the details and determine
what mitigations should take
place to keep this task lower in
the criticality index.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
133/186
When sensitivity and criticality
are combined a sense of the
cruciality. Cruciality is defined as
“a state of critical urgency.”
Although this sounds like a
redundancy term, it can be used
to focus our attention on those
tasks that are both critical and
sensitive.
It is important to understand the
sensitivity aspects, since these
can change and drive the
schedule in non–obvious ways.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 134/186
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
135/186
Let’s look at some examples of
Monte Carlo
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
136/186
The Monte Carlo simulation
makes use of the three point
estimates generated during the
PERT analysis. This numbers
represent the upper, lower and
most likely durations.
This values are then used to
draw random numbers from the
probability distribution for
evaluating the activity network.
The branching probabilities can
then be added for the
alternatives paths and risk
mitigation activities.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
137/186
The use of “expert judgment”
itself needs to be calibrated.
The unanswered question on this
program and many others is
“what does a good risk tolerant
IMS actually look like?”
The “units of measure” for risk
tolerance and the confidence in
the probabilistic estimates needs
to be established before the
estimating and modeling process
can be “calibrated”
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
138/186
The ranking of risk or the ranking
of anything needs to be done in a
structured manner.
A geometric progression is a very
useful approach, since it forces
the focus on ranking.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
139/186
The “sense” of risk and real risk
need to be connected.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
140/186
Some type of risk ranking needs
to be developed for the IMS
tasks.
One approach is the TRL scale.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
141/186
When tasks are arranged in
series the cumulative probability
of completion is show in the table
on the right
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
142/186
When the tasks are arranged in
parallel a different completion
profile results.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
143/186
All of this is very interesting in a
Power Point presentation –
marketecture it's called.
Let’s look at a real schedule and
start to apply some of the things
we’ve learned.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
144/186
This is a very simple construction
plan. The tasks are networked in
a way to show how the Risk+ tool
works.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
145/186
The first picture of a completion
time is the PERT assessment.
The task Construction Schedule
Margin (the end of this task is the
end of the planned margin) has a
target date of 2/8/06 and a
forecast PERT date of 3/6/06.
This shows there is not enough
margin by one month for this task
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
146/186
The same task, evaluated with
Risk+ shows a different
completion date.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
147/186
Project Risk Analysis is part of
any good risk management
activity. This has been said
numerous times and needs to he
repeated daily.
Both the technical and the
programmatic risk aspects of the
program need to be shown in the
IMS.
Any questions, changes,
updates, suggestions – anything
that touches the IMS or the cost
model – needs to be assessed
from the point of view of
programmatic risk.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
148/186
The accuracy of the dates and
costs in the IMS is a “relative”
term.
±20% to start with is pretty good.
As the program proceeds
accuracy improves but it is
always a statistical estimate until
after the fact.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
149/186
If we take a deterministic
approach the planning then there
will be built in issues. The first is
that all estimates must include a
confidence interval or they are
wrong.
The “natural” approach to
estimating almost always results
in a bound that is too wide as
well as being optimistic or
pessimistic but hardly never
accurate.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 150/186
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
151/186
Now that we’ve reached a fairly
detailed level of discussion
regarding programmatic risk
assessment, it’s time to talk
about cost risk assessment.
The first concept to understand is
that cost and schedule are
connected. This is obvious. But
they are not connected in any
linear manner.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
152/186
The basic principles of cost
estimating start with the
understanding of the uncertainty
in the estimates of cost.
These uncertainties must be
connected to the technical
uncertainties as well as the
programmatic and simple cost
variances.
The arithmetic addition of costs
creates a false number of the not
only the cost but any variance in
this cost.
Monte Carlo simulation is one
starting point, but like the
programmatic simulations, the
underlying probability
distributions must be understood
before the numbers have any
real meaning
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
153/186
The connection between a
technical parameter is its cost is
not only potentially non–linear it
is probabilistic.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
154/186
A simple 9 step (not so simple
actually) process can be used to
build a cost estimate.
Starting with the “likely” program
in the form of an IMS, the tasks
for delivering that program are
defined.
The underlying probability
distributions for the cost of each
delivering activity are developed.
This is much like the
development of the baseline IMS,
but the next step is much
different.
The correlation between each
WBS element is developed.
These correlations are used to
build a model of sensitivity of the
cost to changes in the tasks.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
155/186
At this point the Risk+ tool fails to
deliver what is needed. Wither
Crystal Ball or @RISK is needed
to connect these correlations
together.
The technical uncertainty of the
program is used to drive the cost
uncertainty. This is where the
technical and programmatic risk
assessments joins.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
156/186
The production of the familiar
probability curves for the
likelihood of cost is the result.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
157/186
The risk margin in dollars is the
result needs to make this
connection.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
158/186
Resulting in the budget risk
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
159/186
Which produces the estimates for
the management reserve
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
160/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
161/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
162/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
163/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 164/186
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
165/186
We’re near the end now, so your
brain is certainly getting full.
This is quite a bit of information
to absorb, but it needs to be
done before we can say we are
building “risk tolerant” plans.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
166/186
When probabilistic schedule
analysis is used it does not
replace the need for a well
formed project network. It only
replaces the use of PERT for
estimating the completion dates.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
167/186
The quality of the probabilistic
estimates is the foundation of
confidence.
The next step is to clearly identify
where in the IMS risks are being
mitigated, the impacts of this
mitigation and the overall
confidence in the master plan
resulting from this mitigation
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
168/186
The correlations between cost,
schedule, and technical risk must
be made explicit.
A model of how these elements
interact is the basis for answering
the “what if” questions that occur
when the risk item becomes
active.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
169/186
Risk based schedule and cost
management is core to
programmatic integrity.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 170/186
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
171/186
Like any good idea it can be
improved on forever.
These opportunities are much
harder to address than the
process so far. They require care
and effort to build a correlation
matrix for the tasks. They require
detailed understanding of the
underlying statistical processes
and the historical data that was
used to develop these
distributions.
For most projects this is beyond
the scope of the effort and may
be beyond the business interests
as well – since the pay back is
not clearly defined.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
172/186
The dependencies between tasks
is the basis of the correlation
function. This is very important if
a true model of the network is to
be developed. In the absence of
the correlations it is assumed
tasks are independent, which of
course can not be the case.
Building a Program risk
assessment requires that cost
and schedule be connected as
well – correlated. Cost and
schedule are not linear, so any
simple model of changes in one
linearly effecting the other cannot
work.
Finally the idea of a causal model
– a cause and a set of effects
provides deeper insight into the
risk behaviors of the network.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
173/186
There is too much information
here for a single digestive
process. The only way to absorb
all this is to start practicing
probabilistic schedule and cost
analysis and make the
knowledge appear in the output
information.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
174/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
175/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
176/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
177/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
178/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
179/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
180/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
181/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
182/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
183/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
184/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
185/186
There are nearly unlimited
resources on the web. The
challenge of course is finding
them.
Here’s some know starting
points.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 8/10/2012
186/186
This has been a long journey
over hopefully many weeks of
discussion and hands on
experience with Risk+ and real
project schedules.
Building a risk tolerant IMS is a
“practice” and practices require
proficiency. Proficiency comes
from “doing the work,” looking at
the results and making changes
for improvement.
This is just the beginning.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005