quantification of the non- parametric continuous bbns with expert judgment iwona jagielska msc....

Quantification of the non- Quantification of the non- parametricparametric

continuous BBNs with expert continuous BBNs with expert judgmentjudgment

Iwona JagielskaIwona JagielskaMsc. Applied MathematicsMsc. Applied Mathematics

Outline of the presentationOutline of the presentation

I PARTI PART

1. Introduction1. Introduction II PARTII PART

2. Method of eliciting conditional rank correlations2. Method of eliciting conditional rank correlations3. Comparison of algorithms to calculate multivariate normal 3. Comparison of algorithms to calculate multivariate normal probabilities probabilities 4. Presentation of elicitation software UniExp4. Presentation of elicitation software UniExp

III PARTIII PART5. Building the Maintenance Performance Model5. Building the Maintenance Performance Model

Model variablesModel variablesDependence relationDependence relation

6. Results 6. Results

7. Conclusions and recommendations7. Conclusions and recommendations

CATS – casual model for Air Transport Safety – motivation and purpose - three sectors of human performance

ATC Model, Flight Crew Performance ModelMaintenance Performance Model

1. Introduction1. Introduction

2. Way of assessing dependence relations2. Way of assessing dependence relations

• Conditional Rank correlations Conditional probabilities of exceedence

• Why normal copula?

- Advantages• known relation between partial and rank correlation• equal conditional and partial correlations• possess zero independent property

- Disadvantages• no analytical form for multivariate cumulative distribution function

P1= P ( FX4(X4) > q | FX3(X3) > q )

“Suppose that the variable X3 was observed above its qth quantile. What is the probability that also X4 will be observed above its qth quantile? “

))(),...,((),...,( 11

11 NN uuuuC N


4,3 1 4,3( )P

To see the conditional probability as a function of rank correlation we integrate bivariate

normal density over the given region .21 ]),([ q

1 11 4,30.7 0.57e eP r

• we can calculate relationship between rank correlation and conditional probability

4,3 1 4,3( )r P r4,3 4,3r

P2= P ( FX4(X4) > q | FX3(X3) > q, FX2(X2) > q )

“ Suppose that not only variable X3 but also X2 was observed above their qth quantile. What is the probability that also X4 will be observed above its qth quantile? ”

To find the conditional probability we integrate trivariate normal density over the

given region with covariance matrix .

31 ]),([ q


1 12 4,2|30.8 0.37e eP r

4,2|3 2 4,2|3( )r P r

We assess the higher order conditional rank correlation in the similar way.

4,3 4,2

4,3 3,2

4,2 3,2

1

1

1

2 4,2 4,2 4,2|3 4,2|3( )P r

Proposed numerical integration methods:Proposed numerical integration methods:

• Algorithm I and II – by Genz

- first we apply transformation to simplify integration region- later randomized quasi Monte Carlo method is used - different choice of quasi points- in algorithm I we specify number of points; algorithm II assign number of points, s.t. the requested accuracy is provided

• Algorithm III and IV - based on successive subdivisions of integration region, where each subdivision is used to provide a better approximation of the integrand - polynomial rule is used to approximate integrand on each subregions- error estimate – difference between two polynomial rules of different order- algorithm IV may involve some simplification routines (change of variables)

n

n

b

a

xx

n

b

a

b

a

dxebaF1

2

2

1

1

'

)2(

1...),(

3.1. Algorithms to calculate multivariate normal 3.1. Algorithms to calculate multivariate normal probabilitiesprobabilities

TAi – time of calculation for algorithm i PAi – probability obtained by algorithm i EAi – estimated error of approximation provided by algorithm i700, 1500 – number of quasi random point in Alg I; 10 -5 requested accuracy for Alg II

3.2. Numerical3.2. Numerical ComparisonComparison

dimension = 4, determinant = 0.5271


• Algorithms III and IV are unpractical for large scale applications since they require long time for numerical calculations

- time for hypercube [0.5, inf]7 is more than 700seconds- when the procedure of subdivision of integration region is applied, algorithm do not provide the total error

• In Algorithm I user needs to specify number of quasi random points used to calculation; there is no control of provided error of estimation; time of calculation depends on the number of points, not of covariance matrix

• Time of calculation for Algorithm II is sometimes grater than for Algorithm I; time depends on covariance matrix; number of quasi random points depends on requested accuracy of solution

At this moment Algorithm II is used in the software UniExp as the most accurate one; Algorithm I also has future potencial for implementation.

3. Numerical3. Numerical Comparison – brief summaryComparison – brief summary

4. Software elicitation tool - UniExp4. Software elicitation tool - UniExp

1 Step – input of nodes and connections

4. Software elicitation tool - UniExp4. Software elicitation tool - UniExp2 Step – elicitation of conditional rank correlations

4. Software elicitation tool - UniExp4. Software elicitation tool - UniExp

Values of Rank Correlations can be found in RankCorrelationValues.txt file

5. Maintenance Performance Model 5. Maintenance Performance Model

Elicitation with single expert

we asked – 4 questions about marginal distributions – classical method of expert judgment

7 questions about conditional probabilities of exceedance

5. Maintenance Performance Model – dependence relation5. Maintenance Performance Model – dependence relation

All variables are negatively correlated with variable human error


At the bottom of each histogram the expectation and standard derivation are shown.

Unconditional expected value of human error is 0.266/10000

6. Maintenance Performance Model - conditioning6. Maintenance Performance Model - conditioning

Number of years of experience = 3expected value of human error increases 0.266/10000 -> 0.309/10000

6. Maintenance Performance Model - conditioning6. Maintenance Performance Model - conditioning

Requiring at least 6 hours of sleep provides decrease of expected human error from 0.266/10000 to 0.152/10000

Moreover E(HE|WorkCond=1,Alert=6) = 0.248/10000 while E(HE|WorkCond=1)=0.398/10000

7. Conclusions and recommendations7. Conclusions and recommendations

• Calculation of multivariate normal probabilities is not an easy task in case of high dimension; there is still need to develop more fast (and also accurate) algorithm for higher dimension

• Include Algorithm I in UniExp software; together with making UniExp to worked outside the Matlab environment

• Combining experts opinion to obtain better results

• Collect data describing to marginal distribution in Maintenance Performance Model

• Discover other possible influential factors in Maintenance Performance Model

Any other propositions?

Questions ???Questions ???

Additional Additional

SlidesSlides

A1. Covariance matrixes used in numerical testsA1. Covariance matrixes used in numerical tests

Dimension 4

determinant = 0.5271


A1. Covariance matrixes used in numerical testsA1. Covariance matrixes used in numerical tests

Dimension 7determinant = 0.4890


A2. Determinant of covariance matrix as the measureA2. Determinant of covariance matrix as the measure of spread from distributionof spread from distribution

Dimension 2

determinant = 1

A2. Determinant of covariance matrix as the measureA2. Determinant of covariance matrix as the measure of spread of distributionof spread of distribution

Dimension 2determinant = 0.51, =0.714

A2. Determinant of covariance matrix as the measureA2. Determinant of covariance matrix as the measure of spread of distributionof spread of distribution

Dimension 2determinant = 0.0199, =0.141

3.2. Numerical3.2. Numerical ComparisonComparison

Test 1 – identity covariance matrix dimension = 4

dimension = 7

TAi – time of calculation for algorithm i PAi – probability obtained by algorithm i EAi – estimated error of approximation provided by algorithm i700, 1500 – number of quasi random point in Alg I; 10 -5 requested accuracy for Alg II

Test 2 – covariance matrix with determinant 0.5



3. Numerical3. Numerical ComparisonComparison

Test 3 – covariance matrix with determinant 0.1



3. Numerical3. Numerical ComparisonComparison

-Motivation –- part of CATS model- build to describe the causal factors influencing the maintenance crew

Methodology- non-parametric BBN

QuantificationQuantification - Nodes – variables which can influence the human performance among the maintenance crew; marginal distribution – data or Classical Method (Expert Judgment)

- Conditional rank correlations – obtained from experts through the dependence probabilities of exceedance


5. Maintenance Performance Model – model variables5. Maintenance Performance Model – model variables

Variable Definition Source of marginal Source of marginal distributiondistribution

1. Job 1. Job Trainings average number of training per yearaverage number of training per year Expert judgmentExpert judgment

2. Alertness average number of hours an aircraft mechanic sleeps of per day

DataData

3. Communication current information transfer procedure in use, distinguishing: 1. only paper notes, 2. paper notes with oral feedback

Expert judgmentExpert judgment

4. Experience average number of years a person worked as aircraft mechanic

DataData

6. Aircraft Generation aircraft generation in scale from 1 to 4 where 4 is the most recent generation

Data

5. Working Conditions average number of maintenance operationsaverage number of maintenance operations

needed to be performed 1.out-side / needed to be performed 1.out-side /

2. inside the hangar per 10,000 2. inside the hangar per 10,000

maintenance operationsmaintenance operations

Expert JudgmentExpert Judgment

7. Human Error number of maintenance human errors that number of maintenance human errors that

might lead to hazardous situations per 10,000 might lead to hazardous situations per 10,000 maintenance tasksmaintenance tasks

Expert JudgmentExpert Judgment

quantification of the non- parametric continuous bbns with expert judgment iwona jagielska msc....

Documents