A general first-order global sensitivity analysis method
Chonggang Xu, George Z. Gertner*
Department of Natural Resources and Environmental Sciences,University of Illinois at Urbana-ChampaignUSA
Uncertainty & Sensitivity analysis Techniques
Design of experiments method Sampling-based method Fourier Amplitude Sensitivity Test (FAST) Sobol’s method ANOVA method Moment independent approaches
FAST Search function:
Fourier transformation:
Variance decomposition:
-1 1 1( arcsin(sin( s))) s
2i i ix F
1 2 n( , ,..., ) ( )Y f x x x f s
01
{ cos( ) sin( )}k kk
Y A A ks B ks
0 -
-
-
1( )d
21
( )cos( )d
1( )sin( )d
k
k
A f s s
A f s ks s
B f s ks s
2 2k k k
1( )
2A B p
piiV
i characteristic frequency parameter i
Sum spectrum
FAST advantages & limitations
Computationally efficient global sensitivity analysis method;
Suitable for nonlinear and non-monotonic models;
Aliasing effects for small sample sizes(frequency interference) ;
Suitability for only models with independent parameters;
Real applications
Dependence among parameters;
Complex model with many parameters, which needs much computation times and large sample sizes in traditional FAST;
FAST improvement Reorder
independent parameters limitation;
Random Balance Design (Tarantola et al. 2006) commom frequency for all parameters
and permuting
Overcome aliasing effect limitation;
Tarantola, S, Gatelli, D, Mara, TA. Random balance designs for the estimation of first order global sensitivity indices. Reliability Engineering and System Safety, 2006; 91(6): 717-727.
1i
1 1arcsin(sin s s
2
i(x = F ( ( ))) )
1 2{ , ,..., ,..., } j NS s s s s
for each parameter.
↑
Details of synthesized FAST
Xu, C. and G.Z. Gertner 2007. A general first-order global sensitivity analysis method. Reliability Engineering and System Safety (In press).
Reorder for correlation: Model Y = x1+ x2, correlation=0.7, characteristic frequency is 5 and 23
respectively.
Independent sample order Correlated sample order of X1
0
0.5
1
1.5
2
0 100 200 300 400 500
y
00.10.20.30.40.50.60.70.80.9
1
1 101 201 301 401
x1
x2
00.10.20.30.40.50.60.70.80.9
1
1 81 161 241 321 401
x1
x2
Response Variable Y
Reorder for correlation: Model Y = x1+ x2,
correlation=0.7,common characteristic frequency is 5 for both x1 and x2
Independent sample order Correlated sample order of X1
00.10.20.30.40.50.60.70.80.9
1
1 101 201 301 401
x1
x2
00.10.20.30.40.50.60.70.80.9
1
1 81 161 241 321 401
x1
x2
0
0.5
1
1.5
2
0 100 200 300 400 500
yResponse Variable Y
00.10.20.30.40.50.60.70.80.9
1
1 81 161 241 321 401
x1
x2
correlation=0.0
00.10.20.30.40.50.60.70.80.9
1
1 81 161 241 321 401
x1
x2
correlation=0.2
00.10.20.30.40.50.60.70.80.9
1
1 81 161 241 321 401
x1
x2
correlation=0.5
00.10.20.30.40.50.60.70.80.9
1
1 81 161 241 321 401
x1
x2
correlation=0.9
FAST sample with common characteristic
frequency
Reordered sample
Model outputs
based on reordered
sample
Sample for FAST
analysis of x1
Sample for FAST
analysis of x2
Sample for the common variable s
1)
2)
3)
4)
5)
5)
6)
6)
x1 x2 order
0.72 0.72 1
0.83 0.83 2
0.39 0.39 3
0.06 0.06 4
0.50 0.50 5
0.94 0.94 6
0.61 0.61 7
0.17 0.17 8
0.28 0.28 9x1 x2 order1 order2
0.39 0.61 3 70.72 0.28 1 90.50 0.94 5 60.17 0.17 8 80.61 0.72 7 10.28 0.50 9 50.06 0.06 4 40.94 0.83 6 20.83 0.39 2 3
x1 x2 order1 order2 y0.39 0.61 3 7 1.000.72 0.28 1 9 1.000.50 0.94 5 6 1.440.17 0.17 8 8 0.330.61 0.72 7 1 1.330.28 0.50 9 5 0.780.06 0.06 4 4 0.110.94 0.83 6 2 1.780.83 0.39 2 3 1.22
x1 x2 order1 order2 y0.61 0.72 7 1 1.330.94 0.83 6 2 1.780.83 0.39 2 3 1.220.06 0.06 4 4 0.110.28 0.50 9 5 0.780.50 0.94 5 6 1.440.39 0.61 3 7 1.000.17 0.17 8 8 0.330.72 0.28 1 9 1.00
x1 x2 order1 order2 y0.72 0.28 1 9 1.000.83 0.39 2 3 1.220.39 0.61 3 7 1.000.06 0.06 4 4 0.110.50 0.94 5 6 1.440.94 0.83 6 2 1.780.61 0.72 7 1 1.330.17 0.17 8 8 0.330.28 0.50 9 5 0.78
s
-2.79
-2.09
-1.40
-0.70
0.00
0.70
1.40
2.09
2.79
Maximum Harmonic order selection
Frequency
0 50 100 150 200 250
Am
plit
ude
0.00
0.05
0.10
0.15
0.20
0.25
Frequency
0 50 100 150 200 250
Am
plit
ude
0.000
0.005
0.010
0.015
0.020
pp
iiV Scaled characteristic spectrum
1. Random balance design may introduce random error.2. Assume that the low characteristic amplitudes at high harmonic order are more susceptible to the random error than relatively high characteristic amplitudes at a low harmonic order .
Simulated annealing refinement for correlated samples
Sample size
0 200 400 600 800 1000 1200
Dev
ianc
e
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.14
BoxplotAverage0.02 reference
Temperature PAR Precipitation CO2
Temperature 1.00 0.70 0.24 0.41
PAR 0.70 1.00 0.27 0.07
Precipitation 0.24 0.27 1.00 0.05
CO2 0.41 0.07 0.05 1.00
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
1 11 21 31 41 51 61 71 81 91 101
Iterations
Dev
ianc
e
PAR is photosynthetic active radiation
TEST CASES
Synthesized FAST specification for test cases
Model Characteristic Frequency
Sample Size Maximum harmonic order
Test case one 23 921 14
Test case two 23 4614 for x1-x4
2 for others
Test case three 23 4614 for x1-x3
2 for others
Test case four 23 4614 for x2-x5
2 for others
Test case one:
Parameter
x1 x2
Sen
siti
vity
0.0
0.2
0.4
0.6
0.8
1.0
Analytical sensitivity index of x1
Analytical sensitivity index of x2
SFAST 10 replicationsSFAST 50 replicationsSFAST 100 replications
Y=2x1+3x2 , where x1 and x2 are standard normally distributed with a Pearson correlation coefficient of 0.7
SFAST is synthesized FASTCircles are analytical
Test case two: (Lu and Mohanty, 2001)
Parameter
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10
Sen
siti
vity
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
SFAST 10 replicationsSFAST 50 replicationsSFAST 100 replications
103 2
1
y ( ) 50( ) ,i i i 1 2 3i
a x x x x x
1 2 3 4 5
6 7 8 9 10
a = 100, a = 80, a = 60, a = 40, a = 20,
a = 0.1, a = 0.08, a = 0.06, a = 0.02, a = 0.01
1 0 0 0 0 0 0 0 0 0
0 1 0.3 0.7 0 0 0 0 0 0
0 0.3 1 0.4 0 0 0 0 0 0
0 0.7 0.4 1 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 1 0.4 0 0
0 0 0 0 0 0 0.4 1 0 0
0 0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 1
Circles are based on correlation ratio method by Saltelli(2001) based McKay’s one-way ANOVA. Nonparametric method suitable for nonlinear and monotonic models. 50,000 model runs (=100 replications x 500 samples)
Rank correlation
Test case three:
Parameter
x1 x2 x3 x4 x5 x6 x7 x8
Sen
siti
vity
0.0
0.2
0.4
0.6
0.8
SFAST 10 replicationsSFAST 50 replicationsSFAST 100 replications
1
( )n
i ii
f g x
4 2
( ) for 0 1 and 01i i
i i i ii
x ag x x a
a
1 0 0.6 0 0 0 0 0
0 1 0 0 0.7 0 0 0
0.6 0 1 0 0 0 0 0
0 0 0 1 0 0 0 0
0 0.7 0 0 1 0 0 0
0 0 0 0 0 1 0.4 0
0 0 0 0 0 0.4 1 0
0 0 0 0 0 0 0 1
(G-function of Sobol’. Non-monotonic test model)
Rank correlation
Circles are based on correlation ratio method . 50,000 model runs (=100 replications x 500 samples)
Test case four: World 3 Model(Meadows et al., 1992)
IndustrialCapital
fraction of industrial outputallocated to investment
industrial output
initial industrial capital
average life of industrial capital
industrial capitaldepreciation
industrial capitalinvestment
average life of industrial capital 1
average life of industrial capital 2
<POLICY YEAR>
<industrial capital output ratiomultiplier from resource
conservation technology>
<fraction of industrial outputallocated to agriculture>
fraction of industrial outputallocated to consumption
<fraction of industrial outputallocated to services>
<capacity utilization fraction><fraction of industrial
capital allocated toobtaining resources>
industrial capital output ratio
industrial capital output ratio 1
industrial capital output ratio 2
<Time>
<industrial capital outputratio multiplier from pollution
technology>
<industrial capital outputratio multiplier from land yield
technology>
fraction of industrialoutput allocated to
consumption constant
fraction of industrialoutput allocated to
consumption variable
industrialequilibrium time
<Time>fraction of industrial output allocated
to consumption constant 1
fraction of industrial output allocatedto consumption constant 2
industrialoutput per
capita desired
industrial outputper capita
fraction ofindustrial output
allocated toconsumptionvariable table
<population>
<POLICY YEAR>
<POLICY YEAR>
<Time>
Parameter uncertainty specification
Parameter Label Lower bound Upper bound
x1 industrial output per capita desired 315 385
x2 industrial capital output ratio before 1995 2.7 3.3
x3fraction of industrial output allocated to
consumption before 19950.387 0.473
X4fraction of industrial output allocated to
consumption after 19950.387 0.473
X5average life of industrial capital before
199512.6 15.4
X6average life of industrial capital after
199516.2 19.8
x7 initial industrial capital 1.89(10+11) 2.31(10+11)
Rank correlation of .6 between x3 and x4 and .4 between x5 and x6
Year
1900 1950 2000 2050 2100
Sen
siti
vity
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.14
0.16
0.18
SFASTCRM
Year
1900 1950 2000 2050 2100
Sen
siti
vity
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.14
0.16
0.18
0.20
0.22
SFASTCRM
Year
1900 1950 2000 2050 2100
Sen
siti
vity
0.1
0.2
0.3
0.4
0.5
0.6
SFASTCRM
Year
1900 1950 2000 2050 2100
Sen
siti
vity
0.0
0.1
0.2
0.3
0.4
0.5
SFASTCRM
x2 x3
x4 x5
Year
Sen
sitiv
ity
(Correlation ratio method (CRM). 50,000 model runs (=100 replications x 500 samples)
Year
1900 1950 2000 2050 2100
Sens
ivity
0.0
0.1
0.2
0.3
0.4
0.5
0.6
x2 x3 x4x5
Assume Independence
Year
1900 1950 2000 2050 2100
Sen
siti
vity
0.1
0.2
0.3
0.4
0.5
0.6
SFASTCRM
Year
1900 1950 2000 2050 2100
Sen
siti
vity
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.14
0.16
0.18
0.20
0.22
SFASTCRM
Assume correlation
3 x
4 x
Application: Uncertainty in forest landscape response to global warming
PnET-IITemperature
Precipitation
ANPP
SEP
LANDIS-IILandscape
composition
CO2
PnET-II is a forest ecosystem process model (LINKAGES) is a forest GAP modelLANDIS is a spatially dynamic forest landscape model)
Month
1 2 3 4 5 6 7 8 9 10 11 12
Tem
pera
ture
(o C
)
-20
-10
0
10
20
30
1984-1993 2090-2099
Year
1960 1980 2000 2020 2040 2060 2080 2100 2120
Pre
cipi
tati
on (
cm)
50
60
70
80
90
100
110
Year
1960 1980 2000 2020 2040 2060 2080 2100 2120
CO
2 C
once
ntra
tion
(pp
m)
300
400
500
600
700
800
Year
1960 1980 2000 2020 2040 2060 2080 2100 2120
Tem
pera
ture
(o C
)
0
2
4
6
8
10
(c) (d)
(b)(a)
Example of data after 1994 is based on prediction by Canadian Climate Center (CCC) in the Phase-II Vegetation-Ecosystem Modeling and Analysis Project (VEMAP). Data before 1994 is historical data. We assume the climate stabilizes after year 2099.
Parameter uncertainty Based on 27 predicted climate data structure from the
Intergovernmental Panel on Climate Change (IPCC) Third and Fourth Assessment Report , and the Phase-II Vegetation-Ecosystem Modeling and
Analysis Project (VEMAP) Variable N Mean Std Dev Median Minimum Maximum
Temperature 27 7.43576 2.36065 7.45714 3.97337 12.58936
PAR 27 571.26298 32.19750 566.47531 522.93955 633.86171
Precipitation 27 92.53673 6.93255 92.73039 79.19724 105.41716
CO2 27 689.34074 111.10015 699.40000 546.80000 923.25000
PAR is photosynthetic active radiation
Rank correlation structure
Spearman Correlation Coefficients, N = 27 Prob > |r| under H0: Rho=0
Temperature PAR Precipitation CO2
Temperature 1.00000 0.69780<.0001
0.238710.2305
0.409340.0340
PAR 0.69780<.0001
1.00000 0.268620.1755
0.069630.7300
Precipitation 0.238710.2305
0.268620.1755
1.00000 0.054950.7854
CO2 0.409340.0340
0.069630.7300
0.054950.7854
1.00000
PAR is photosynthetic active radiation
Uncertainty
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0 100 200 300 400
0
0.1
0.2
0.3
0.4
0.5
0.6
0 100 200 300 400
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 100 200 300 4000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 100 200 300 400
Spruce-firPine
Aspen-birch Maple-ash
Simulation year
Unc
erta
inty
Sensitivity
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 100 200 300 400
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 100 200 300 400
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 100 200 300 400
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 100 200 300 400
Spruce-firPine
Aspen-birch Maple-ash
Simulation year
Sen
sitiv
ity
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 200 400
Temperature
PAR
Precipitation
CO2
Conclusion Proposes a general first-order global sensitivity approach
for linear/nonlinear models with as many correlated or uncorrelated parameters as the user specifies;
FAST is computationally efficient and would be a good choice for uncertainty and sensitivity analysis for models with correlated parameters;
Conclusion Proposes a general first-order global sensitivity approach
for linear/nonlinear models with as many correlated or uncorrelated parameters as the user specifies;
FAST is computationally efficient and would be a good choice for uncertainty and sensitivity analysis for models with correlated parameters;
Thank You!