toward a common metric for risk assessment across diverse

2017-03-21 1

Toward a Common Metric for Risk Assessment across

Diverse Factors in Fatigue Risk Management Systems:

Quantifying Human Performance in Terms of Signal-to-Noise Ratio

Hans P.A. Van Dongen, Ph.D.Director, Sleep and Performance Research Center

Professor, Elson S. Floyd College of Medicine

Washington State University Spokane

2017-03-21 2

A Typical Problem in Fatigue Risk Management:

Assessing the System-Wide Risk from Fatigue in Coupled

Physical and Cyber Infrastructures in 24/7 Operations with

Humans in the Loop

Decision graph

2017-03-21 3

A Key Aspect of Fatigue Risk from Sleep-Deprived Humans in the Loop:

Instability in Sustained Attention

Baseline

Sleep deprivation

0 200 400 600 800 1000

response time (ms)

Lapse threshold

(500 ms)

fre

qu

en

cy

Psychomotor

Vigilance

Test

(PVT)

0

500

1000

1500

2000

0 60 120 180 240 300 360 420 480 540 600

time on task (s)

re

ac

tio

n t

ime

(m

s)

0

500

1000

1500

2000

0 60 120 180 240 300 360 420 480 540 600

time on task (s)

re

ac

tio

n t

ime

(m

s)

5 h awake (13:00)

29 h awake (13:00)

res

po

ns

e t

ime

(m

s)

res

po

ns

e t

ime

(m

s)

Psychomotor Vigilance Test

0 1 2 3 4 5 6 7 8 9 10

time on task (min)

Doran SM, Van Dongen HPA, Dinges DF. Arch Ital Biol 2001; 139: 253–267.

Satterfield BC, Van Dongen HPA. Fatigue Biomed Health Behav 2013; 1: 118–136.

2017-03-21 4

a

0

b

Diffusion Decision Model

ӿ ӿ ӿ ӿ ӿ ӿӿ ӿ ӿ ӿ ӿ

ӿ ӿ ӿ ӿ ӿ ӿ ӿӿ ӿ ӿ ӿ ӿ

ӿ ӿ ӿ ӿ ӿӿ ӿ ӿ ӿ ӿ ӿӿ ӿ ӿ

ӿ ӿ ӿ ӿӿ ӿ ӿ ӿ ӿ

ӿ ӿ ӿ ӿ ӿ ӿ ӿ ӿ

Numerosity DiscriminationNumerosity

Discrimination

Sleep Deprivation Degrades Cognitive Processing

in Neuronal Circuits Involved in Task Performance

Ratcliff R, Van Dongen HPA. Psychon Bull Rev, 2009; 16(4): 742–751.

Ratcliff R, Van Dongen HPA. PNAS, 2011; 108: 11285–11290.

reaction time (ms)

freq

uen

cy

diffusion

model

simulations

diffusion

model

simulations

for the PVT

C SD

freq

ue

nc

y

diffusion model

simulations for

PVT responses

BLSD

reaction time (ms)

2017-03-21 5







Numerosity Discrimination

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

BL SD REC

Dri

ft R

ate

BL SD/C REC

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

BL SD REC

Dri

ft R

ate

SD-easy

SD-medium

SD-harder

C-easy

C-medium

C-harder

Numerosity

Discrimination

a

0

b





2017-03-21 6








0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

BL SD REC

Dri

ft R

ate

BL SD/C REC

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

BL SD REC

Dri

ft R

ate

SD-easy

SD-medium

SD-harder

C-easy

C-medium

C-harder

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

BL SD REC

Dri

ft R

ate

SD-easy

SD-medium

SD-harder

C-easy

C-medium

C-harder

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

BL SD REC

Dri

ft R

ate

Numerosity

Discrimination

a

0

b





2017-03-21 7








0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

BL SD REC

Dri

ft R

ate

BL SD/C REC

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

BL SD REC

Dri

ft R

ate

SD-easy

SD-medium

SD-harder

C-easy

C-medium

C-harder

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

BL SD REC

Dri

ft R

ate

SD-easy

SD-medium

SD-harder

C-easy

C-medium

C-harder

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

BL SD REC

Dri

ft R

ate

Numerosity

Discrimination

a

0

b

Fatigue may make

any cognitive

processing harder

and less reliable





2017-03-21 8

A Diffusion Decision Model-Based Metric of the

Fidelity of Information Processing: PVT Signal to Noise Ratio (SNR)

1

1

2

1 1

2

1

N

i

N

i

N

i

iiiii

N

i

ii

SwwSw

SwN

SNR

Si = 1 / (RTi – C), wi = 1 / (r2 Si + 1), C = 100 ms, r2 = 196 ms,

RTi is the ith response time (in ms), and N is the number of PVT stimuli

Chavali VP, Riedy SM, Van Dongen HPA. Sleep, 2017, in press.

2017-03-21 9

8

12

16

20

24

28

32

36

40

0 6 12 18 24 30 36

Time Awake (h)

SN

R

Impact of Sleep Deprivation on PVT SNR in Laboratory Studies


Medium effect size

( f 2 = 0.34)

2017-03-21 10

8

12

16

20

24

28

32

36

40

0 6 12 18 24 30 36

Time Awake (h)

SN

R

Impact of Sleep Deprivation on PVT SNR in Laboratory Studies


PVT testing

with chin rest

2017-03-21 11

9

11

13

15

17

0 6 12 18 24 30 36

Time Awake (h)

LS

NR

(d

B)

A Baseline-Invariant Version of the PVT SNR Metric

- 0 dB

- –3 dB

- –6 dB


LSNR =

10 log10(SNR)

Large effect size

( f 2 = 0.40)

2017-03-21 12

Conclusions• The LSNR metric for the PVT quantifies the fidelity of information processing

• LSNR has high sensitivity to fatigue, high degree of statistical normality, and

absence of floor and ceiling effects

• A given change in LSNR always has the same meaning regardless of absolute

values – a reduction in LSNR of 3 units (i.e., a −3 dB change) represents a

50% drop in the fidelity of information regardless of the starting point

• The SNR baseline value may therefore be freely chosen to anchor the metric

(0 dB point), which is helpful for mathematical models of fatigue

• LSNR provides a basis for calculations of the overall reliability of partially

automated operational systems with sleep-deprived humans in the loop

• As such, LSNR may be a useful tool for systems-integrated fatigue risk

management


2017-03-21 13

Research supported by:

Office of Naval Research

Air Force Office of Scientific Research

Army Medical Research and Materiel Command

National Institutes of Health

NASA Headquarters

Department of Defense

US Naval Medical Logistics Command

Naval Postgraduate School

US Army Research Office

Department of Transportation

Federal Motor Carrier Safety Administration

Federal Aviation Administration

Transportation Research Board

Regional Airline Association

FedEx Express

Pulsar Informatics

Institutes for Behavior Resources

State of Washington

Acknowledgments

toward a common metric for risk assessment across diverse

Documents