volvo technology humans system integration volvo technology direct metrics of driver performance...

Volvo TechnologyHumans System Integration

Volvo Technology

Direct metrics of driver performance

Johan EngströmVolvo Technology Corporation

Driver Metrics WorkshopOttawa, October 2-3, 2006


Volvo Technology

Outline

• Background of research (HASTE and AIDE)

• Metrics• Lane keeping

• Steering

• Eye movements• Time sharing

• Gaze concentration

• Conclusions, lessons learned and topics for further research


Volvo Technology

Research in HASTE and AIDE on performance metrics

• Research conducted 2003-2006 in the HASTE and AIDE EU-funded projects

• General objective of the studies: • Investigate systematically the effects of visual and cognitive load on driving

performance -> define metrics for IVIS safety evaluation

• Data collected in simulators (of varying grade) and field (HASTE)

• Further analysed in AIDE

• Work reported here performed in collaboration with VTI (Swedish National Transport Research Institute)


Volvo Technology

The HASTE WP2 data set

• Collected in HASTE WP2 during 2003-2004

• 9 parallel studies at different sites in Europe and Canada

• Same general methodology and experimental design

• Varied mainly with respect to test set-up (desktop simulator, meduim-high-fidelity simulators and field trials)

• Secondary tasks:

Visual: Arrows task Auditory/cognitive: aCMT

Auditory Continuous Memory Task

(aCMT)

3 difficulty levels each


Volvo Technology

• Three general driving scenarios: Motorway, Rural, and Urban

• Present analyses based on data from three sub-studies• VTEC fixed-base simulator (rural and motorway)

• VTI moving-base simulator (rural and motorway)

• Volvo-VTI field study (instrumented Volvo S80 on motorway)

The HASTE WP2 data set (cont’d)

VTEC simulator VTI simulator Volvo S80


Volvo Technology

General result

Visual and cognitive load have qualitatively different effects on driving…


Volvo Technology

Summary of results: Visual task

Parameter Effect Interpretation

Lane keeping variation + Reduced lateral control due to visual time sharing

Steering wheel activity (e.g. reversals)

++ (mainly large reversals, 2-5 deg.)

Increased steering effort to correct lane keeping errors

Speed - Compensation for reduced lateral control to maintain safety margins

Headway and TTC + Compensation for reduced lateral control to maintain safety margins

Glance frequency ++ (increased with task difficulty)

Increased visual complexity -> more glances required

Glance duration ++ (increased with task difficulty)

Increased visual complexity -> longer glances required


Volvo Technology

Summary of results: Cognitive task

Parameter Effect Interpretation

Gaze concentration to road centre ++ Less cognitive resources available for visual monitoring to the periphery

Lane keeping variation - Indirect effect of increased gaze concentration

Steering wheel activity (e.g. reversals)

++ (mainly small reversals.)

More active and precise steering as a result of more visual input

Speed +- (inconsistent) Dependent on test scenario

Mean Headway and TTC No effect No compensation

Headway variation + Less consistent car following


Volvo Technology

General conclusion: Visual and cognitive load have different effects

• Visual

• Visual diversion

• Steering hold

• Lane keeping error

• Large corrective steering movements

• Slowing down & increasing headway to compensate

Cognitive

• Interference with attention selection mechanisms

• Gaze concentrates to road centre

• More visual control input than during normal driving

• More active and precise steering

• More accurate lane-keeping

Reduced visual

detection/

decision making


Volvo Technology

Outline



• Steering





Volvo Technology

Focus of this presentation

• Metrics intended for task-based IVIS evaluation

• Types of metrics covered:• Lane keeping

• Steering

• Eye movements


Volvo Technology

Outline



• Steering





Volvo Technology

Example data: Straight driving, rural road, VTEC simulator

120 130 140

1020304050

Baseline

Ra

dia

l ga

ze (

de

g)

Driver 7, straight rural road

120 130 140-5

0

5

St.

wh

ee

l an

gle

(d

eg

)

120 130 140

-0.5

0

0.5

La

ne

po

sitio

n (

m)

120 130 140

70

75

80

85

Sp

ee

d (

km/h

)

Time (s)

940 950 960

1020304050

Baseline

940 950 960-5

0

5

940 950 960

-0.5

0

0.5

940 950 960

70

75

80

85

Time (s)

1000 1010 1020

1020304050

Visual task, level 3

1000 1010 1020-5

0

5

1000 1010 1020

-0.5

0

0.5

1000 1010 1020

70

75

80

85

Time (s)

100 110 120

1020304050


100 110 120-5

0

5

100 110 120

-0.5

0

0.5

100 110 120

70

75

80

85

Time (s)

120 130 140

1020304050

Baseline

Ra

dia

l ga

ze (

de

g)


120 130 140-5

0

5

St.

wh

ee

l an

gle

(d

eg

)

120 130 140

-0.5

0

0.5

La

ne

po

sitio

n (

m)

120 130 140

70

75

80

85

Sp

ee

d (

km/h

)

Time (s)

940 950 960

1020304050

Baseline

940 950 960-5

0

5

940 950 960

-0.5

0

0.5

940 950 960

70

75

80

85

Time (s)

1000 1010 1020

1020304050


1000 1010 1020-5

0

5

1000 1010 1020

-0.5

0

0.5

1000 1010 1020

70

75

80

85

Time (s)

100 110 120

1020304050


100 110 120-5

0

5

100 110 120

-0.5

0

0.5

100 110 120

70

75

80

85

Time (s)

80 100

10

20

30

Baseline

Ra

dia

l ga

ze (

de

g)


80 100

-2

0

2

4

St.

wh

ee

l an

gle

(d

eg

)

80 100

-0.20

0.20.40.6

La

ne

po

sitio

n (

m)

80 100

90

100

110

120

Sp

ee

d (

km/h

)

Time (s)

760 780

10

20

30

Baseline

760 780

-2

0

2

4

760 780

-0.20

0.20.40.6

760 780

90

100

110

120

Time (s)

120 140

10

20

30

Cognitive task, level 3

120 140

-2

0

2

4

120 140

-0.20

0.20.40.6

120 140

90

100

110

120

Time (s)

960 980

10

20

30


960 980

-2

0

2

4

960 980

-0.20

0.20.40.6

960 980

90

100

110

120

Time (s)

80 100

10

20

30

Baseline

Ra

dia

l ga

ze (

de

g)


80 100

-2

0

2

4

St.

wh

ee

l an

gle

(d

eg

)

80 100

-0.20

0.20.40.6

La

ne

po

sitio

n (

m)

80 100

90

100

110

120

Sp

ee

d (

km/h

)

Time (s)

760 780

10

20

30

Baseline

760 780

-2

0

2

4

760 780

-0.20

0.20.40.6

760 780

90

100

110

120

Time (s)

120 140

10

20

30


120 140

-2

0

2

4

120 140

-0.20

0.20.40.6

120 140

90

100

110

120

Time (s)

960 980

10

20

30


960 980

-2

0

2

4

960 980

-0.20

0.20.40.6

960 980

90

100

110

120

Time (s)

80 100

10

20

30

Baseline

Ra

dia

l ga

ze (

de

g)


80 100

-2

0

2

4

St.

wh

ee

l an

gle

(d

eg

)

80 100

-0.20

0.20.40.6

La

ne

po

sitio

n (

m)

80 100

90

100

110

120

Sp

ee

d (

km/h

)

Time (s)

760 780

10

20

30

Baseline

760 780

-2

0

2

4

760 780

-0.20

0.20.40.6

760 780

90

100

110

120

Time (s)

120 140

10

20

30


120 140

-2

0

2

4

120 140

-0.20

0.20.40.6

120 140

90

100

110

120

Time (s)

960 980

10

20

30


960 980

-2

0

2

4

960 980

-0.20

0.20.40.6

960 980

90

100

110

120

Time (s)

80 100

10

20

30

Baseline

Ra

dia

l ga

ze (

de

g)


80 100

-2

0

2

4

St.

wh

ee

l an

gle

(d

eg

)

80 100

-0.20

0.20.40.6

La

ne

po

sitio

n (

m)

80 100

90

100

110

120

Sp

ee

d (

km/h

)

Time (s)

760 780

10

20

30

Baseline

760 780

-2

0

2

4

760 780

-0.20

0.20.40.6

760 780

90

100

110

120

Time (s)

120 140

10

20

30


120 140

-2

0

2

4

120 140

-0.20

0.20.40.6

120 140

90

100

110

120

Time (s)

960 980

10

20

30


960 980

-2

0

2

4

960 980

-0.20

0.20.40.6

960 980

90

100

110

120

Time (s)


Volvo Technology

Frequency analysis

10-2

10-1

100

101

10-2

10-1

100

101

Frequency (Hz)

Fre

qu

en

cy c

on

ten

t in

st.

wh

ee

l sig

na

l

Baseline, fixed base simulator, motorway, joint sequence, all driversBaseline, fixed base simulator, rural straight, joint sequence, all driversBaseline, fixed base simulator, rural curved, joint sequence, all driversBaseline, moving base simulator, rural straight, joint sequence, all driversBaseline, moving base simulator, rural curved, joint sequence, all driversBaseline, field, motorway, joint sequence, all driversTask length frequency

10-2

10-1

100

101

10-4

10-3

10-2

10-1

100

101

Frequency (Hz)

Fre

quen

cy c

onte

nt in

st.

whe

el s

igna

l

Driver 1, fixed-base simulator, straight rural road

BaselineVisual taskTask length frequency

10-2

10-1

100

101

10-4

10-3

10-2

10-1

100

101

Frequency (Hz)

Fre

quen

cy c

onte

nt in

st.

whe

el s

igna

l



10-2

10-1

100

101

10-4

10-3

10-2

10-1

100

101

Frequency (Hz)

Fre

quen

cy c

onte

nt in

st.

whe

el s

igna

l



All subjects

Individual subjects


Volvo Technology

Common types of lane keeping metrics

Continuous Event-based

TLC (Time-to-line-crosssing)-based

Position-based

e.g. standard deviation of

lane position (SDLP)

e.g. proportion of lane

exceedences (LANEX)

e.g. mean of TLC minima (MN_TLC)

e.g. proportion of TLC minima <

X s (PR_TLC)

Non-normal distribution & too few instances -> difficult to use for task-based evaluation

Less sensitive than position-based metrics and yield roughly similar results -> no obvious advantage for present purposes

Focushere


Volvo Technology

(Modified) Standard deviation of lane position (SDLP) (1)

• Operational definition (AIDE D2.2.5 – Östlund et al. 2006):

• ”Standard deviation of lateral position data, high-pass filtered with a cut-off frequency of 0.1 Hz, where lateral position is defined as the average distance between the right side of the front or rear right wheel and the inner (closest) edge of the right hand lane marking.”


Volvo Technology

High-pass filtering needed to overcome this problem (Östlund et al., 2006)

SDLP depenency on data duration


Volvo Technology

Representative results from HASTE on SDLP

Visual task Cognitive task

VTEC simulator, rural road

00,050,1

0,150,2

0,250,3

0,350,4

Straight Curve

st_l

p (

m)

BL

SLv1

SLv2

SLv3

0

0,1

0,2

0,3

0,4

0,5

0,6

Straight Curve

st_l

p (

m)

BL

SLv1

SLv2

SLv3


Volvo Technology

• Advantages• Easy to measure, at least in the simulator (feasible also in the field

using off-the-shelf lane-tracking systems)

• Straightforward general interpretation as performance metric

• Disadvantages• Only moderately sensitivite to secondary task task load

• Strongly sensitive to environment factors (e.g. curvature, lane width)

• Sensitive to discontinuities due to lane changes and exceedences

• Relation to crash data• Open issue – no strong direct evidence of causal relation between

increased SDLP and crash risk (however, indirect evidence via visual distraction)

(M)SDLP pros and cons


Volvo Technology

Outline



• Steering





Volvo Technology

Example data: Straight driving, rural road, VTEC simulator

120 130 140

1020304050

Baseline

Ra

dia

l ga

ze (

de

g)


120 130 140-5

0

5

St.

wh

ee

l an

gle

(d

eg

)

120 130 140

-0.5

0

0.5

La

ne

po

sitio

n (

m)

120 130 140

70

75

80

85

Sp

ee

d (

km/h

)

Time (s)

940 950 960

1020304050

Baseline

940 950 960-5

0

5

940 950 960

-0.5

0

0.5

940 950 960

70

75

80

85

Time (s)

1000 1010 1020

1020304050


1000 1010 1020-5

0

5

1000 1010 1020

-0.5

0

0.5

1000 1010 1020

70

75

80

85

Time (s)

100 110 120

1020304050


100 110 120-5

0

5

100 110 120

-0.5

0

0.5

100 110 120

70

75

80

85

Time (s)

120 130 140

1020304050

Baseline

Ra

dia

l ga

ze (

de

g)


120 130 140-5

0

5

St.

wh

ee

l an

gle

(d

eg

)

120 130 140

-0.5

0

0.5

La

ne

po

sitio

n (

m)

120 130 140

70

75

80

85

Sp

ee

d (

km/h

)

Time (s)

940 950 960

1020304050

Baseline

940 950 960-5

0

5

940 950 960

-0.5

0

0.5

940 950 960

70

75

80

85

Time (s)

1000 1010 1020

1020304050


1000 1010 1020-5

0

5

1000 1010 1020

-0.5

0

0.5

1000 1010 1020

70

75

80

85

Time (s)

100 110 120

1020304050


100 110 120-5

0

5

100 110 120

-0.5

0

0.5

100 110 120

70

75

80

85

Time (s)

80 100

10

20

30

Baseline

Rad

ial g

aze

(deg

)


80 100

-2

0

2

4

St.

whe

el a

ngle

(deg

)

80 100

-0.20

0.20.40.6

Lane

pos

ition

(m)

80 100

90

100

110

120

Spe

ed (k

m/h

)

Time (s)

760 780

10

20

30

Baseline

760 780

-2

0

2

4

760 780

-0.20

0.20.40.6

760 780

90

100

110

120

Time (s)

120 140

10

20

30


120 140

-2

0

2

4

120 140

-0.20

0.20.40.6

120 140

90

100

110

120

Time (s)

960 980

10

20

30


960 980

-2

0

2

4

960 980

-0.20

0.20.40.6

960 980

90

100

110

120

Time (s)

80 100

10

20

30

Baseline

Ra

dia

l ga

ze (

de

g)


80 100

-2

0

2

4

St.

wh

ee

l an

gle

(d

eg

)

80 100

-0.20

0.20.40.6

La

ne

po

sitio

n (

m)

80 100

90

100

110

120

Sp

ee

d (

km/h

)

Time (s)

760 780

10

20

30

Baseline

760 780

-2

0

2

4

760 780

-0.20

0.20.40.6

760 780

90

100

110

120

Time (s)

120 140

10

20

30


120 140

-2

0

2

4

120 140

-0.20

0.20.40.6

120 140

90

100

110

120

Time (s)

960 980

10

20

30


960 980

-2

0

2

4

960 980

-0.20

0.20.40.6

960 980

90

100

110

120

Time (s)

80 100

10

20

30

Baseline

Ra

dia

l ga

ze (

de

g)


80 100

-2

0

2

4

St.

wh

ee

l an

gle

(d

eg

)

80 100

-0.20

0.20.40.6

La

ne

po

sitio

n (

m)

80 100

90

100

110

120

Sp

ee

d (

km/h

)

Time (s)

760 780

10

20

30

Baseline

760 780

-2

0

2

4

760 780

-0.20

0.20.40.6

760 780

90

100

110

120

Time (s)

120 140

10

20

30


120 140

-2

0

2

4

120 140

-0.20

0.20.40.6

120 140

90

100

110

120

Time (s)

960 980

10

20

30


960 980

-2

0

2

4

960 980

-0.20

0.20.40.6

960 980

90

100

110

120

Time (s)

80 100

10

20

30

Baseline

Ra

dia

l ga

ze (

de

g)


80 100

-2

0

2

4

St.

wh

ee

l an

gle

(d

eg

)

80 100

-0.20

0.20.40.6

La

ne

po

sitio

n (

m)

80 100

90

100

110

120

Sp

ee

d (

km/h

)

Time (s)

760 780

10

20

30

Baseline

760 780

-2

0

2

4

760 780

-0.20

0.20.40.6

760 780

90

100

110

120

Time (s)

120 140

10

20

30


120 140

-2

0

2

4

120 140

-0.20

0.20.40.6

120 140

90

100

110

120

Time (s)

960 980

10

20

30


960 980

-2

0

2

4

960 980

-0.20

0.20.40.6

960 980

90

100

110

120

Time (s)


Volvo Technology

• Standard deviation of steering wheel angle

• High frequency steering – 3 versions

• Steering entropy – 2 versions (Boer, 2000;Boer, 2005)

• Steering wheel reversal rate – 2 versions (HASTE version; Modified version developed in AIDE, Markkula and Engström, 2006)

Metrics investigated in AIDE


Volvo Technology

Results in sensitivity (effect size) – visual load

Visual

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Stand

ard

devia

tion

Revers

al ra

te1

Revers

al ra

te2

HF stee

ring1

HF stee

ring2

HF stee

ring3

Steer

ing en

tropy

1

Steer

ing en

tropy

2

Sta

nd

ard

ised

eff

ect

size

Fixed, mw

Fixed, rural, straight

Fixed, rural, curve

Moving, rural, straight

Moving, rural, curve

Field

All metrics fairly sensitive in all conditions except Standard Deviation

Markkula and Engström (2006)


Volvo Technology

Results in sensitivity (effect size) – cognitive loadCognitive

0

0.2

0.4

0.6

0.8

1

1.2

1.4

Stand

ard

devia

tion

Revers

al ra

te1

Revers

al ra

te2

HF stee

ring1

HF stee

ring2

HF stee

ring3

Steer

ing en

tropy

1

Steer

ing en

tropy

2

Sta

nd

ard

ised

eff

ect

size

Fixed, mw

Fixed, rural, straight

Fixed, rural, curve

Moving, rural, straight

Moving, rural, curve

Field

Reversal Rate2 and Steering entropy most sensitive

Markkula and Engström (2006)


Volvo Technology

• Operational definition• Entropy of the prediction errors made

by a linear predictive filter applied on the steering wheel angle signal (see Boer 2005 for detailed mathematical definition)

• Interpretation• ”…increase in high frequency

steering corrections that result after periods of diverted or reduced attention (i.e., in response to a perceived vehicle drift outside the acceptable tolerance margins that mounted during these periods of degraded information)” (Boer, 2005)

Steering entropy (1)


Volvo Technology

• Advantages• Strongly sensitive to visual and cognitive load in a range of conditions

• SW data easy to measure, also in the field

• Relatively robust to differences in driving environment (road type, curvature, test set-up etc.)

• Disadvantages• Fairly complex to compute (though straightforward)

• Somewhat difficult to interpret, even in terms of performance (increased SE may indicate both increased and reduced lateral control)

• Interpretation of free parameters (alpha and re-sampling rate) not entirely straightforward

• Requires baseline data for computation of task condition data

• ”Normalisation” to baseline data makes BL and Task data somewhat dependent

• Relation to crash data• No established relation to crash data (only indirectly via visual distraction)

Steering Entropy pros and cons


Volvo Technology

Steering Wheel Reversal Rate (SRR)

• Operational definition• The number, per minute, of

steering wheel reversals larger than a certain angular value referred to as the gap size (see Markkula & Engström, 2006, for detailed mathematical definition)


Volvo Technology

Representative results from HASTE: SRR1, 1 degree gap size

Visual task Cognitive task


0

2

4

6

8

10

12

14

16

Straight Curve Event

rr_s

t1

BL

SLv1

SLv2

SLv3

0

2

4

6

8

10

12

14

Straight Curve Event

rr_s

t1

BL

SLv1

SLv2

SLv3


Volvo Technology

SRR Sensitivity (effect size) as a function of gap size

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

0.1 0.5 1 2 3 4 5 10

Gap size (degrees)

Sta

nd

ard

ise

d e

ffe

ct

siz

e (

σ)

Visual, fixed, mw

Cognitive fixed mw

Visual, fixed, rural, straight

Cognitive, fixed, rural,straightVisual, fixed, rural, curve

Cognitive, fixed, rural, curve

Visual, field

Cognitive, field

Visual, moving, rural,straightCognitive, moving, rural,straightVisual, moving, rural, curve

Cognitive, moving, rural,curve


Volvo Technology

• Advantages• Strongly sensitive to visual and cognitive load in a range of conditions

• SW data easy to measure, also in the field

• Easier to interpret than steering entropy

• Does not involve normalisation of task data to baseline data (like Steering Entropy)

• Disadvantages• Sensitive to environment factors

• Somewhat difficult to interpret in terms of performance - increased SRR may indicate both reduced and increased lane keeping performance (however, can be tuned by changing gap-size)

• Relation to crash data• Like other steering wheel metrics, no established relation to crash data (only

indirectly via visual distraction)

SRR pros and cons


Volvo Technology

Outline



• Steering





Volvo Technology

Example data

120 130 140

1020304050

Baseline

Ra

dia

l ga

ze (

de

g)


120 130 140-5

0

5

St.

wh

ee

l an

gle

(d

eg

)

120 130 140

-0.5

0

0.5

La

ne

po

sitio

n (

m)

120 130 140

70

75

80

85

Sp

ee

d (

km/h

)

Time (s)

940 950 960

1020304050

Baseline

940 950 960-5

0

5

940 950 960

-0.5

0

0.5

940 950 960

70

75

80

85

Time (s)

1000 1010 1020

1020304050


1000 1010 1020-5

0

5

1000 1010 1020

-0.5

0

0.5

1000 1010 1020

70

75

80

85

Time (s)

100 110 120

1020304050


100 110 120-5

0

5

100 110 120

-0.5

0

0.5

100 110 120

70

75

80

85

Time (s)

80 100

10

20

30

Baseline

Ra

dia

l ga

ze (

de

g)


80 100

-2

0

2

4

St.

wh

ee

l an

gle

(d

eg

)

80 100

-0.20

0.20.40.6

La

ne

po

sitio

n (

m)

80 100

90

100

110

120

Sp

ee

d (

km/h

)

Time (s)

760 780

10

20

30

Baseline

760 780

-2

0

2

4

760 780

-0.20

0.20.40.6

760 780

90

100

110

120

Time (s)

120 140

10

20

30


120 140

-2

0

2

4

120 140

-0.20

0.20.40.6

120 140

90

100

110

120

Time (s)

960 980

10

20

30


960 980

-2

0

2

4

960 980

-0.20

0.20.40.6

960 980

90

100

110

120

Time (s)

80 100

10

20

30

Baseline

Ra

dia

l ga

ze (

de

g)


80 100

-2

0

2

4

St.

wh

ee

l an

gle

(d

eg

)

80 100

-0.20

0.20.40.6

La

ne

po

sitio

n (

m)

80 100

90

100

110

120

Sp

ee

d (

km/h

)

Time (s)

760 780

10

20

30

Baseline

760 780

-2

0

2

4

760 780

-0.20

0.20.40.6

760 780

90

100

110

120

Time (s)

120 140

10

20

30


120 140

-2

0

2

4

120 140

-0.20

0.20.40.6

120 140

90

100

110

120

Time (s)

960 980

10

20

30


960 980

-2

0

2

4

960 980

-0.20

0.20.40.6

960 980

90

100

110

120

Time (s)


Volvo Technology

Outline



• Steering





Volvo Technology

Factors to account for

• Total time spent looking away from the road

• Intensity (”how much looking away per time untit”)

• Distribution of single glance durations

• Eccentricity

B

A

C

On-road

Off-road

On-road

Off-road

On-road

Off-road


Volvo Technology

Traditional (ISO 15007) glance-based metrics

Measure Definition

Glance frequency The number of glances to a target within a pre-defined time period, or during a pre-defined task, where each glance is separated by at least one glance to a different target (ISO 15007).

Single glance duration Time from the moment at which the direction of gaze moves towards a target to the moment it moves away from it (ISO 15007).

Mean single glance duration The average duration of the glances towards a target.

Number of glances > 2 seconds The number of glances towards the system with a duration longer than 2 seconds.

Total glance time (towards a target)

Total glance time (or percentage of time) associated with a target (e.g. in-vehicle device).


Volvo Technology

Automating the ISO 15007 metrics: The VDM Tool (Larsson, 2002; Johansson et al., 2006)


Volvo Technology

Difficulties with automating the ISO 15007 metrics

• Standard originally intended for manual transcription

• Glance-based metrics are very sensitive to noise

• Requires careful calibration and signal pre-processing

• Much data still needs to be discarded (~30% in HASTE)


Volvo Technology An alternative: Road centre-based metrics

Road Centre

On-road glances

Off-road glances


Volvo Technology

Percent Road Centre (Victor, 2005)

• Operational definition:• PRC-Task: The percent of fixations directed towards the road centre (RC)

during a task. Represents intensity only.

• PRC-Window: The percent of fixations directed towards the RC during a moving time window of 1 minute. If the task is shorter than 1 minute, the remaining time is completed with a constant PRC of 80%. The windowing adds a weighting for task duration.


Volvo Technology

Example data from HASTE (Victor, Harbluk and Engström, 2005)


Volvo Technology

Pros and cons of PRC

• Advantages

• Very sensitive to visual task difficulty

• Allows for baseline data (which glance-based metrics to do not)

• Should be more robust to measurement noise (focus measurement where eye tracking accuracy is normally best, data order does not matter)

• Disadvantages

• PRC-Task measures only intensity

• PRC-Window accounts for task duration, but somewhat arbitrarily

• Does not account for eccentricity

• Relation to crash data

• Strong empirical evidence on the relation between visual diversion from the forward road scene and accident risk (e.g. Wierwille and Tijerina, 1995; Klauer et al., 2006)


Volvo Technology

Further ideas

• RC-based versions of the ISO metrics (Kronberg et al., 2006)

• Other ways to account for both intensity and duration

• Weighting function for single glance duration

• Account for eccentricity

• For example:

),(2/3 EgVDi

i gi=single off-road glance durationE=eccentricity weighting function


Volvo Technology

Outline



• Steering





Volvo Technology

Measuring gaze concentration: Standard deviation of gaze angle

• Operational definition

• The standard deviation of the combined horizontal and vertical angles. The combined angle is the square root of the sum of squared vertical and squared horizontal angles (Pythagoras theorem) and thus is a one-dimensional angle between the origin and a gaze point


Volvo Technology

Effects of cognitive task on gaze concentration

Gaze angles (pich and yaw)

Baseline Cognitive task (levels 1-3 aggregated)



Volvo Technology

Example data from HASTE (Victor, Harbluk and Engström, 2005)


Volvo Technology

Pros and cons of SD gaze angle

• Advantages

• Sensitive to cognitive load (more than PRC) – good metric of gaze concentration

• Robust to noise since data order does not matter

• Disadvantages

• Only applicable to assessment of purely cognitive load

• Relation to crash data

• No empirical data on the relation between gaze concentration and crash risk


Volvo Technology

Outline



• Steering





Volvo Technology

Conclusions and lessons learned

• The metrics addressed here mainly relevant for evaluating visually demanding tasks• Lateral control performance metrics somewhat problematic as surrogate safety metrics –

no clear link to crash data

• Direct eye movement metrics seem to be the most promising (though still practical difficulties with data collection and analysis)

• For cognitive tasks, other metrics are needed to capture the main safety-relevant effects (e.g. detection task metrics such as PDT)

• Lack of agreed driver model – very little consensus on how to interpret even the most common driving performance metrics

• Little discussion and emprical work on the link between performance metrics and safety (especially in Europe)


Volvo Technology

Topics for further research

• Development of metrics representing sensory-motor coordination (e.g. correlation between steering and eye movements)

• More comprehensive visual demand metrics, taking into account both duration, intensity and eccentricity

• Establish relation between different performance metrics and crashes (using data from naturalistic field studies) -> valid criteria for IVIS safety evaluation and ADAS safety benefits analyses

• Investigate how to incorporate exposure data (frequency of use) into the IVIS evaluation methods (e.g. in a general formula for visual demand exposure)

• Establish stronger theoretical foundation for driving performance assessment• Multiple resource theory (Wickens, 2002) does not explain all variance in the data (e.g.

driver adaptation and effects of cognitive load in terms of gaze concentration, and ”improved” lane keeping)

• Incorporate modern perception and attention (”active vision”) theories into driving research (see Victor, 2005)


Volvo Technology

References• Engström, J., Johansson, E and Östlund, J. (2005). Effects of visual and cognitive load in real and

simulated motorway driving. Transportation Research Part F, pp. 97-120.• HASTE special issue (contains all papers from the WP2 studies): Transportation Research Part F: Traffic

Psychology and Behaviour, Volume 8, Issue 2• Johansson, E., Engström, J., Cherri, C., Nodari, E., Toffetti, A., Schindhelm, R., Gelau, C. (2004) Review

of existing techniques and metrics for IVIS and ADAS assessment. EU project AIDE, project IST-1-507674-IP, Deliverable 2.2.1

• Johansson, E., Kronberg, P., Victor, T., Martens, M., Chin, E. and Nathan, F. (2006). Visual demand measurement tool development. AIDE Deliverable 2.2.2. EC contract No. IST-1-507674-IP.

• Kronberg, P., Victor, T. and Engström, J. (2006). Road-centre-based measures of visual demand. Vision in Vehicles, Dublin.

• Larsson, P. (2002). Automatic Visual Behavior Analysis. Dissertation for a Master of Science Degree Applied Physics and Electrical Engineering Control and Communication Department of electrical engineering Linköping University, Sweden. LiTH-ISY-EX-3259.

• Markkula, G. and Engström, J. In press. A Steering Wheel Reversal Rate Metric for Assessing Effects of Visual and Cognitive Secondary Task Load. ITS World Congress, London 2006.

• Östlund, J., Peters, B., Thorslund, B., Engström, J., Markkula, G., Keinath, A., Horst, D., Mattes, S. Foehl, U. 2005. Driving performance assessment: Methods and metrics. AIDE Deliverable 2.2.5. European Commission, IST-1-507674-IP.

• Östlund, J. Carsten, O., Merat, N., Jamson, S., Janssen, W., & Brouwer, R., et al. (2004). Deliverable 2—HMI and safety-related driver performance. Human Machine Interface And the Safety of Traffic in Europe (HASTE) Project. Report No. GRD1/2000/25361 S12.319626.

• Victor TW (2005) Keeping eye and mind on the road. PhD Thesis, Uppsala University, Sweden.• Victor, T. W., Harbluk, J. L. & Engström, J. (2005). Sensitivity of eye-movement measures to in-vehicle

task difficulty. Transportation Research Part F 8:167-190

volvo technology humans system integration volvo technology direct metrics of driver performance...

Documents