section 9.2: parametric modeling of stationary … 9.2: parametric modeling of stationary processes...
TRANSCRIPT
Section 9.2: Parametric Modeling of Stationary
Processes
Discrete-Event Simulation: A First Course
c©2006 Pearson Ed., Inc. 0-13-142917-5
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 1/ 27
Parametric Modeling of Stationary Processes
We want to model an element given a data set collected onthe element of interest
e.g. arrival process or service times
One possibility: use trace driven methods
E.g., simulations in Chapter 1Described in detail in Chapter 9.1
The alternative: find a distribution that fits the data
Subject of today’s lecture, for stationary processesCalled parametric modeling
Obvious benefits for simulationDownside — additional error introduced
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 2/ 27
Step 1: Hypothesizing a distribution
Requires some experience — consider these as guidelines to help
What is the source of the data?
Discrete or continuous?Bounded? Non-negative?Sum or product of random variables?
Look at histogram shape.
Useful to eliminate distributionsSymmetric? Flat?
Compute some simple statistics
Coefficient of variation s/x
Skewness: measures symmetry
Find a statistician to help
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 3/ 27
Example 9.2.1
Data set of n = 23 service times x1, x2, . . . , x23
0 50 100 150 2000.0
0.1
0.2
0.3
0.4
f(x)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...................................................................................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.....................................................................................................................................................................................................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..................................................................................................................................................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
...................................................................................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
x = 72.22 s = 37.49 sx
= 0.52 1n
n∑
i=1
(
xi − x
s
)3
= 0.88
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 4/ 27
Guidelines to Select a Distribution
Plot skewness γ3 versus coefficient of variation γ2
As measured for sample dataPopulation values for parametric distributions
0.0 0.5 1.0 1.5 2.0 2.5−1
0
1
2
3
4
γ3
.
...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
.
..
.
..
..
.
..
.
..
.
..
.
..
..
.
..
.
..
.
..
.
..
.
..
.
..
.
..
..
.
..
.
..
.
..
.
..
.
.
..
.
..
.
..
.
..
.
..
.
..
.
.
..
.
..
.
..
.
.
..
.
..
.
.
..
.
..
.
.
..
.
.
..
.
..
.
.
..
.
.
..
.
.
..
.
.
..
.
.
.
..
.
.
..
.
.
.
..
.
.
.
..
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
.
.
.
..
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
..
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
•
•
γ2
GammaLognormal
Exponential
Weibull
Data
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 5/ 27
Weibull Random Variates
The continuous r.v. X is Weibull(a, b) if and only if
The real-valued shape parameter a satisfies a > 0The real-valued scale parameter b satisfies b > 0The possible values of X are X = {x |x > 0}The cdf of X is
F (x) = 1 − exp(−(bx)a) x > 0
Alternate, legitimite cdfs:
F (x) = 1 − exp(−(x/b)a)F (x) = 1 − exp(−bxa)
The mean and standard deviation of X are
µ =1
bΓ
(
1 +1
a
)
σ =1
b
√
Γ
(
1 +2
a
)
−[
Γ
(
1 +1
a
)]2
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 6/ 27
Step 2: Parameter Estimation
We have a hypothesis distribution
We must determine appropriate parameters for thedistribution
Parameters to match the sample data
Let q denote the number of parameters to determine
Exponential has one parameter so q = 1Uniform has two parameters so q = 2
Two basic approaches to estimate parameters
Method of momentsMaximum likelihood estimation
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 7/ 27
Method of Moments
Idea: select parameters so that
First q population moments
E [X k ] k = 1, 2, . . . , q
equal the first q sample moments
1
n
n∑
i=1
xki k = 1, 2, . . . , q
Gives q equations to solve q unknowns
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 8/ 27
Example 9.2.3
Estimate q = 1 parameters for Poisson(µ)
Population first moment is E [X ] = µ
Sample first moment is x
Set equal, solve for estimator µ
Gives method of moments estimator µ = x
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 9/ 27
Example 9.2.4
Estimate q = 1 parameters for Exponential(µ)
Population first moment is E [X ] = µSample first moment is x = 72.22 from sample dataGives estimator µ = 72.22
Exponential(72.22) cdf and empirical cdf:
0 20 40 60 80 100 120 140 160 1800.00
0.25
0.50
0.75
1.00
F (x)
x.......................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.......................................................
.
.
.
.
.
.
.
.
.
.
.
.
........................................................................................................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.............................................................
.
.
.
.
.
.
.
.
.
.
.
.
........................................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
....
.
.
.
.
.
.
.
.
.
.
.
.
............................................................................................................................................................................
.
.
.
..
.
..
.
..
.
..
.
..
..
.
..
.
..
.
..
..
.
..
..
.
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 10/ 27
Example 9.2.5
Estimate q = 2 parameters for Normal(µ, σ)
Denote first sample moment as m1 = x
Denote second sample moment as m2 = 1n
∑ni=1 x2
i
Set E [X ] = m1 and E [X 2] = m2
Since σ2 = E [X 2] − µ2, rewrite as
µ = m1
σ2 + µ2 = m2
Solve for µ and σ to get estimators
µ = m1 = x
σ =√
m2 − m21 = s
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 11/ 27
Example 9.2.6
Estimate q = 1 parameter for Geometric(p)
Population mean isp
1 − p
Sample mean is x
Set equal, solve for p
Get estimator
p =x
1 + x
Do we have 0 < p < 1 ?
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 12/ 27
Example 9.2.7
Estimate parameter for Gamma(a, b) for sample data
Use equations
µ = m1
σ2 + µ2 = m2
Gamma distribution (see Chapter 7.6) has
µ = ab and σ = b√
a
Get estimators
a =x2
s2
b =s2
x
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 13/ 27
Example 9.2.7 ctd
For our sample data, we estimate parameters as
a = x2
s2 = 72.222
37.492 ≃ 3.7
b = s2
x= 37.492
72.22 ≃ 19.46
Gamma(3.7, 19.46) cdf and empirical cdf:
0 20 40 60 80 100 120 140 160 1800.00
0.25
0.50
0.75
1.00
F (x)
x.......................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.......................................................
.
.
.
.
.
.
.
.
.
.
.
.
........................................................................................................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.............................................................
.
.
.
.
.
.
.
.
.
.
.
.
........................................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
....
.
.
.
.
.
.
.
.
.
.
.
.
............................................................................................................................................................................
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
..............................
..
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 14/ 27
Example 9.2.8
Estimate parameters for Uniform(a, b)
Equate population and sample moments
a + b
2= m1
(b − a)2
12+
(
a + b
2
)2
= m2
Solve for a and b to obtain estimates
a = x −√
3 s b = x +√
3 s
Truly Uniform(a, b) data must fall between a and b
Estimators a and b may not satisfy this
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 15/ 27
Maximum Likelihood Estimators
Idea: find parameters most likely to produce the sample data
Let θ = (θ1, θ2, . . . , θq) denote the vector of q unknownparameters
Define the likelihood function
L(θ) =n∏
i=1
f (xi , θ)
where f (·) is the distribution’s pdf (including parameters θ)
The Maximum Likelihood Estimators (MLE) θ maximize L(θ)
Sometimes easier to maximize the log likelihood function
log L(θ) =
n∑
i=1
log f (xi , θ)
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 16/ 27
Example 9.2.9
Let x1, x2, . . . , xn be a random sample from an Exponential(µ)population.
The likelihood function is:
L(µ) =
n∏
i=1
f (xi , µ) =
n∏
i=1
1
µexp(−xi/µ) = µ−n exp
(
−n∑
i=1
xi/µ
)
The log likelihood function is:
log L(µ) = −n log µ −n∑
i=1
xi/µ
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 17/ 27
Example 9.2.9 ctd
Maximize log likelihood function:
Differentiate (with respect to µ)Set derivitive to zeroSolve for µ
0 =d
dµlog L(µ) =
d
dµ
(
−n log µ −n∑
i=1
xi/µ
)
= −n
µ+
∑ni=1 xi
µ2
Obtain MLE
µ =1
n
n∑
i=1
xi = x
For this distribution, method of moments and MLE areidentical
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 18/ 27
Example 9.2.10
Given a sample of size n = 3 from an Exponential population:
x1 = 1 x2 = 2 x3 = 6
The MLE µ = x = 3 is the value of µ that maximizes L(µ)
0 1 2 3 4 5 6 7 8 9 100.0
0.1
0.2
0.3
0.4
f(x)
x
.
.
.
.
.
..
.
.
.
..
.
.
.
.
..
.
.
.
..
.
.
..
.
.
.
..
.
.
..
.
.
..
.
..
.
.
..
.
..
.
..
.
..
.
.
..
..
.
.........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Product of lengths of vertical dashed lines is maximized
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 19/ 27
Example 9.2.11
Let x1, . . . , xn be a random sample from a Weibull(a, b) population
Pdf is f (x) = baaxa−1 exp(−(bx)a)
Likelihood function:
L(a, b) =n∏
i=1
f (xi ) = banan
(
n∏
i=1
xi
)a−1
exp
(
−n∑
i=1
(bxi )a
)
Log likelihood function:
log L(a, b) = n log a + an log b + (a − 1)n∑
i=1
log xi − ba
n∑
i=1
xai
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 20/ 27
Example 9.2.11 ctd
Maximize log likelihood function: multi-variable calculus!
Compute partial derivatives, set to zeroSolve q simultaneous equations for q unknowns
0 =∂
∂aL(a, b) =
n
a+ n log b +
n∑
i=1
log xi −n∑
i=1
(bxi )a log bxi
0 =∂
∂bL(a, b) =
an
b− aba−1
n∑
i=1
xai
No closed-form solution for a and b
Can use numerical methods to estimate a and b
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 21/ 27
Fitted Weibull
For the 23 service times, obtain MLE a = 2.10 and b = 0.0122
Weibull(2.10, 0.0122) cdf and empirical cdf:
0 20 40 60 80 100 120 140 160 1800.00
0.25
0.50
0.75
1.00
F (x)
x.......................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.......................................................
.
.
.
.
.
.
.
.
.
.
.
.
........................................................................................................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
.............................................................
.
.
.
.
.
.
.
.
.
.
.
.
........................................................................................................................
.
.
.
.
.
.
.
.
.
.
.
.
....
.
.
.
.
.
.
.
.
.
.
.
.
............................................................................................................................................................................
............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
......................................
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 22/ 27
Accuracy of Point Estimators
So far, have used point estimates for parameters
Also makes sense to use interval estimates
For q = 1 parameter, will get a confidence interval
For q > 1 parameters, will get a confidence region
Details beyond this class
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 23/ 27
Example 9.2.12
95% confidence region for Weibull parameters:
0 1 2 3 40.000
0.005
0.010
0.015
0.020
b
a
.
.
.
.
.
.
.
.
.
......
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
..
.
.
..
.............................................................................................
...................................................................................................................
..
.
.
.
.
.
.
.
.
.
.
.
..
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
..
.
..
.
..
......................................................................................................................
.........................................................................
................................
•
2 (−113.691 − log L(a, b)) < 5.99
Point is (a, b) = (2.10, 0.0122)
Notice region does not include a = 1
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 24/ 27
Step 3: Goodness of Fit
Visual tests:
Compare fitted pdf f with empirical histogram
Compare fitted cdf F with empirical cdf
Generate a P-P plot
Sort the sample data, x(1), x(2), . . . , x(n)
Define adjusted empirical cdf F (x(i)) = (i − 0.5)/n
Plot F (·) versus F (·) for sample dataPerfect fit: points will fall on line y = x
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 25/ 27
Example 9.2.13
The P-P plot for the fitted Weibull distribution:
0.0 0.2 0.4 0.6 0.8 1.0
F (x(i))
0.0
0.2
0.4
0.6
0.8
1.0
F (x(i))
.
..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
•
•
•
• •
•
•
• •••
• • • •
•
•
•
• •
• •
•
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 26/ 27
Analytical Tests for Goodness of Fit
“Standard” statistical tests
Discrete data: chi-square test
Uses differences between discrete histogram and fitted pdfChi-square statistic is a function of the squared differences
Continuous data: Kolmogorov-Smirnov test
Largest vertical distance between empirical and fitted cdfUsually denoted Dn for sample of size n
Example 9.2.14: Kolmogorov-Smirnov tests for sample data
Distribution D23
Exponential(µ) 0.307Weibull(a, b) 0.151Gamma(a, b) 0.123
Lognormal(a, b) 0.090
Discrete-Event Simulation: A First Course Section 9.2: Parametric Modeling of Stationary Processes 27/ 27