quantitative methods for decisionquantitative...
TRANSCRIPT
Quantitative Methods for Decision-Quantitative Methods for DecisionMaking Under Uncertainty
Sankaran MahadevanVanderbilt University, Nashville, TN
Email: [email protected]
Vanderbilt University reliability-studies.vanderbilt.edu
Email: [email protected]: www.reliability-studies.vanderbilt.edu
Reliability and Risk Engineering, Analysis, and Management
NSF-IGERT Graduate Program at VanderbiltNSF-IGERT Graduate Program at Vanderbilt
Educational and Research ThemesEducational and Research ThemesMultidisciplinary integrationLarge, complex systemsModeling and simulation Model
Devices, Components
Large Systems
Economic, legal, regulatory, and social perspectives
ModelIntegration
UncertaintyCross-cutting methodologies Uncertainty & Risk
Methods
g gUncertainty quantification, propagation Risk quantificationDecision-making under uncertainty
Participants• 42 graduate students (34 Ph.D., 8 M.S.) • 30 professors Engineering, Math, Economics, Business, Psychology,
Vanderbilt University reliability-studies.vanderbilt.edu
Medicine• Summer researchers (undergraduates, high school teachers)
Reliability and Risk Engineering, Analysis, and Management
Multiple Resources at VanderbiltpStructural/Mechanical
Systems Reliability(S M h d )CRESP -- Multi-university
Nuclear Waste RiskAssessment Consortium
(D. Kosson)
(S. Mahadevan)Institute for Space and
Defense Electronics(R. Schrimpf)
Institute for SoftwareIntegrated Systems
Multidisciplinary Doctoral ACCRE High Performance Integrated Systems(G. Karsai)Program in Reliability/Risk
Engineering & Management
ACCRE High PerformanceComputing Network
Vanderbilt Center forEnvironmental Management
Systems (VCEMS)(M Abkowitz)
Business systemsRisk Assessment and
Management(B Cooil)
VECTOR TransportationResearch Center
Vanderbilt University reliability-studies.vanderbilt.edu
(M. Abkowitz)(B. Cooil) Research Center(M. Abkowitz)
Reliability and Risk Engineering, Analysis, and Management
Industry & Government SupportINDUSTRY
BoeingGOVERNMENT
U. S. DOD (AFRL, Army, Navy, AFOSR)
y pp
Kellogg, Brown & RootFedexGeneral Motors
( , y, y, )U. S. DOE (EM)U. S. DOT (FHWA) NASA (LaRC MSFC GRC ARC JPL)
ChryslerGeneral ElectricPratt and Whitney
NASA (LaRC, MSFC, GRC, ARC, JPL)DOE Labs (SNL, LANL, INL, SRNL)Nuclear Regulatory Commission
Pratt and WhitneyBell HelicopterMedtronicUnion Pacific
Federal Aviation Administration
Nature of supportS i t hiUnion Pacific
XeroxSummer internshipsCollaborative research projectsAdvisory committee membershipSeminar speakersLaboratories
S th t R h I tit t
Vanderbilt University reliability-studies.vanderbilt.edu
pStudent recruitment• Southwest Research Institute
• Transportation Technology Center
Risk Analysis Issues
• System modeling• Physics-based behavior models finite elements, bond graphs• Surrogate models (GP, PC, RBF, NN)• Fault trees, event trees, petri-nets• Bayes networks
• Risk analysisRisk analysis• Multi-level -- Material component subsystem system • Risk variation over space and time• Multi-physics, multi-scale problems
• Data Uncertainty• Sparse data, interval data, measurement uncertainty• Expert opinion• Expert opinion• Heterogeneous information
• Model Uncertainty
Vanderbilt University reliability-studies.vanderbilt.edu
y• Model form, model parameters• Errors some deterministic, some stochastic
Reliability and Risk Engineering, Analysis, and Management
Materials durability, fatigue, fracture
y g g y g
Systems health diagnosis and prognosisDecision-making under uncertaintyModel uncertainty, calibration, validationy
Information Uncertainty
FEM and Dynamic
Usage monitoring
Model update
Physical Variability
Uncertainty
Uncertainty Modeling
Analysis
Probabilistic Fatigue
Prognosis
Bayesian Updating
Decision making /Risk management
Variability
Model Uncertainty
Damage Mechanism
Analysis
Prognosis
Inspection/Testing
Reliability update
Vanderbilt University reliability-studies.vanderbilt.edu
Analysis g p
Rotorcraft Damage Tolerance (FAA)
Rotorcraft mast• Two-diameter hollow cylinder
Elli ti l f k i fill t
Analyses• Model calibration
C S• Elliptical surface crack in fillet region
• Sub modeling technique Accuracy in stress intensity factor
– Calibrate EIFS, model parameters– Estimate model errors in different stages
of modeling
• Model validationAccuracy in stress intensity factor • Model validation
• Prediction uncertainty quantification
• Global sensitivity analysis
• Load monitoring and updating
Vanderbilt University reliability-studies.vanderbilt.edu
Sources of Uncertainty
• Physical variability• Loading• Material Properties
• Data uncertainty• Sparseness of data used to quantify material propertiesSparseness of data used to quantify material properties• Output measurement uncertainty (final crack size, detection probability)
• Model uncertainty/errors• Analysis assumptions LEFM planar crack• Analysis assumptions LEFM, planar crack• Finite element discretization errors• Combination of multiple crack modes• Approximation due to surrogate modelApproximation due to surrogate model• Crack growth law model form• Model parameters crack growth model
initial flaw size
Vanderbilt University reliability-studies.vanderbilt.edu
Dynamic Bayes Network
Cycle i Cycle i+1
ai
y
ai+1
yεexp
ΔK ΔKθ aN
C, m, n C, m, n
εcg
ΔKth, σf
εcg
ΔKth, σf
A
Vanderbilt University reliability-studies.vanderbilt.eduSlide 8 of 22
Model Validation Metrics
Null Hypothesis H0: y ∈ f(x)
Alternative Hypothesis: H : y ∉ f(x)
Model response xObserved values y Alternative Hypothesis: H1: y ∉ f(x)y
Classical hypothesis testing
H0: E(y) = E(x) Var(y) = Var(x)
H1: E(y) ≠ E(x) Var(y) ≠ Var(x)t testchi-square test
Bayesian hypothesis testing
B F t
( )( )
( )( )
( )( )⎥⎥⎦
⎤
⎢⎢⎣
⎡
⎥⎥⎦
⎤
⎢⎢⎣
⎡=
j
i
j
i
j
i
MPMP
MPMP
MPMP
nobservationobservatio
nobservationobservatioFrom Bayes
theorem “Confidence”
Vanderbilt University reliability-studies.vanderbilt.edu
Bayes Factor B
Confidence = B/B+1
Crack size prediction UQ
Std. Dev.
Bayes Factor Confidence
Vanderbilt University reliability-studies.vanderbilt.edu
Sensitivity Analysis
• Local Only one uncertainty is considered and allothers are ‘frozen’ at the mean valuesothers are frozen at the mean values
• Global Analyze sensitivity of output over the entiredomain of inputs rather than at mean valuesp
• First order effects (S) & Total effects (ST)
))|(( XYEV ))|(())|(( XYEVXYVE)(
))|(( ~
YVXYEV
S iiXiXi =
)())|((
1)(
))|(( ~~ ~~
YVXYEV
YVXYVE
S iXXiXXT
iiii
i−==
Initial flaw size Crack model error Crack model parameter C
Vanderbilt University reliability-studies.vanderbilt.edu
Cementitious barrier PA
Multi-physics analysis
Diffusion of IonsInputs
- Porosity Damage Chemical Reactions
- Porosity- Composition- Modulus- …..
Progression
Calibration of chemical reaction model parameters (equilibrium
t t )Damage Accumulation
constants)
Durability prediction
Vanderbilt University reliability-studies.vanderbilt.edu
Extrapolation from Validation to Application
Bayes Network Use for
Cm
• Calibration
• Validation
• Extrapolationab e
g Extrapolation
c
dg
f
MCMC techniques Gibbs sampling Extrapolation scenarios
• Nominal values to Extreme valuesNominal values to Extreme values
• Test conditions to Use conditions
• Validation variables to Decision variables
Vanderbilt University reliability-studies.vanderbilt.edu
• Components to System
UQ in system-level prediction
FoamSystem
level
Level 0 Level 1 Level 2
Material characterization
Component level Sub-system level
Joints
characterization level
Joints
Hardware data and photos courtesy of Sandia National Laboratories IncreasesSystem complexity
IncreaseSources of uncertainty
Vanderbilt University reliability-studies.vanderbilt.edu
DecreasesAmount of real data
Bayes Network Implementation
f1Yf
1XFoam Joints
f1Y j
1Y
1
f1ε j
1εf1X j
1X
fθf2ε
jθj
2ε
fX jX
f2Y
f2X
j2Y
j2X
Y = Experimental dataX = FEM prediction1 - Level 12 - Level 2
J = JointsF = Foamθ = Calibration parameters X
Vanderbilt University reliability-studies.vanderbilt.edu
2 - Level 2S - System
θ = Calibration parametersε = Error terms
sX
Likelihood Approach to Data Uncertainty
• Likelihood function⎞⎛⎟
⎞⎜⎛ mn bi P – distribution parameters
⎟⎟⎠
⎞⎜⎜⎝
⎛⎟⎟⎠
⎞⎜⎜⎝
⎛∝ ∏∏ ∫
==
m
iiX
n
i aX PxfdxPxfPL
i
i 11
)|( )|()(P – distribution parameters m – point data sizen – interval data size
interval data sparse point data
• Maximum Likelihood Estimate Maximize L(P)
• To account for uncertainty in P ∫
=)(
)()(PL
PLPf
• Two approaches– Family of distributions for X (for every sample of P probability
distribution for X)Si l di t ib ti f X– Single distribution of X
• Can use non-parametric distributions
∫= dPPfPxfxf )()|()(
Vanderbilt University reliability-studies.vanderbilt.edu
• Can use non-parametric distributions
Risk Management: System Health Monitoring
• System integration• Integrate reliability/risk methods with SHM• Integrate diagnosis with prognosis
• Rapid diagnosis and prognosis• Derive damage signatures• Derive damage signatures• Qualitative isolation, then damage quantification
• Uncertainty Quantification• Quantify variability, uncertainty, errors • Estimate Confidence in diagnosis/prognosis
Vanderbilt University reliability-studies.vanderbilt.edu
• Estimate Confidence in diagnosis/prognosis
Decision-Making Under Uncertainty
Optimization
A1 A2
MDA
A1 A2
Risk Analysis
• Various stages in life cycle design, operations, maintenance• Multiple objectives, MCDA, decision trees, utility-based formulations• Multi-disciplinary systems
Optimization for reliability and robustness• Optimization for reliability and robustness• Include both aleatory and epistemic uncertainties
• Dynamic, network systems y y• Critical facility protection – design of safeguards/detectors• Transportation networks, supply networks, emergency response systems
• System of systems
Vanderbilt University reliability-studies.vanderbilt.edu
• Multiple system linkages• Homeland security, military, commercial applications
Fire Satellite System
Target latitude,Target longitudeTarget size[Altit d ]
Analyses
Multi-disciplinary uncertainty propagation
Design optimi ation for reliabilit rob stness
Orbit
[Altitude]
Orbit period, eclipse period
Design optimization for reliability, robustness
Orbit Power
P_ACS
Orbit Period,Satellite velocity,Max slewing angle
_
I_min, I_max
Attitude
Vanderbilt University reliability-studies.vanderbilt.edu
[P_tot], [T_tot], [A_sat], [I_min], [I_max]
D i i
System of Systems Decision-Making Under Uncertainty
System Complexity
Decisionmaking
• Risk-informed• Design ComplexityOptimization
• Non-linear• Stochastic• Static/Dynamic
• Monolithic• Family of Systems• System of Systems
• Design• Operations • Utility theory
SOS UncertaintyModeling • Aleatory• Epistemic
• Coefficient based• System dynamics• Agent based
S t d l
• Analysis• Management
• Information bonded
• Fully rational • Bounded rational
• Surrogate model
Risk
Types of
Human in the loop
• Energy bonded• Hybrid
Vanderbilt University reliability-studies.vanderbilt.edu
ypSystems
Pandemic Influenza Risk ManagementCIPDSS (LANL)
Vanderbilt University reliability-studies.vanderbilt.edu
Conclusion
System risk assessmentContinuing opportunities for methods development
System risk assessmentRisk variation with time and spaceDynamic, multi‐physics, systems of systemsComputational effortComputational effort
Decision‐making under uncertaintyDesign, operations, maintenance, risk managementData collection Model de elopmentData collection, Model developmentEmbedding flexibility
Include data uncertaintyl d lSparse, noisy, qualitative, missing data, intervals, expert opinion
Multi‐scale fusion of heterogeneous information
Include model uncertainty
Vanderbilt University reliability-studies.vanderbilt.edu
yValidation, calibration, error estimation, extrapolation