measuring the allocation of control in a 6 degree-of-freedom docking experiment

Measuring the Allocation of Controlin a 6 Degree-of-Freedom Docking Experiment

Maurice R. Masliah and Paul Milgram

Ergonomics in Teleoperation and Control (ETC) LabDepartment of Mechanical and Industrial EngineeringUniversity of Toronto, Ontario, Canada, M5S 3G8http://etclab.rose.utoronto.ca{moman, milgram}@etclab.mie.utoronto.ca

Introducing the M-metric

(Images courtesy of Shumin Zhai and Ravin Balakrishnan)

Motivation

Overview

• Measures/definitions of performance metrics

• The M-metric

• Hypothesis for 6 DOF docking tasks

• 6 DOF docking experiment

• Results using the M-metric and discussion

• Shortcomings / Future Work of the M-metric

Start Position

Goal PositionA

Start Position

Goal Position

B

Start Position

Goal Position

E

Start Position

Goal Position

F

Start Position

Goal Position

G

Start Position

Goal Position

D

Start Position

Goal Position

C

Hypothetical Trajectories : 2 DOF

Measures/Definitions of Coordination

• time-on-target (“not very suitable” [Poulton ‘74])

• accuracy speed [Behbehani et al. ‘88]

• spatial or temporal invariance [Morrison & Newell ‘98]

• cross-correlations [Vereijken et al. ‘92, Zhai et al. ‘96]

• integrality [Jacob et al. ‘94]

• inefficiency [Zhai & Milgram ‘98]

Integrality vs. Inefficiency

• Integrality is a measure of simultaneity

(in the time domain)

• Inefficiency is a measure of distance traversed

(in the space domain)

Time

0

2

4

6

8

10

Am

ount

of M

ovem

ent

Amount of Movement Over Time

A

B

The M-metric

• Measures the allocation of control across DOFs

• “Control” = any movement which reduces error

• “Error” = the difference between the goal position and

the current position

• M-metric = (control simultaneity) × (control efficiency)

Err

or

Incr

eas

e

Err

or

Re

duct

ion

Time

DOF "X"DOF "Y"

Definition of Control Simultaneity

CHANGE IN

ERROR

Normalized Error

Reduction

Area under DOF curve = 1

Area of overlap, intersection between the DOFs.

Control Efficiency

Efficiency =the weighted average of the ratios of the length of the “optimal” trajectory for each DOF divided by the actual trajectory

StartPosition Goal

Position

a

bc

Efficiency =c

a + b

M-metric: Primary Features

• measures the allocation of control

• = simultaneity efficiency

• values between 0 and 1

• computed for any number of DOFs ( 2)• (also subsets of the total available DOFs)

• computed across DOFs encompassing different measurement units (cm, degrees)

Hypothesis for 6 DOF docking tasks

• Non-equal allocation of control across DOFs

• Novices • will allocate their control between translation and rotation

DOFs • will switch control back and forth

• As expertise develops:• will continue to allocate their control between translation

and rotation DOFs with improved control• will develop uniform allocation of control across all 6 DOFs

8 subjects total (between subjects design)

2 input devices :

216 docking trials per session

5 one hour sessions

Experimental Design

= 8640 total trials

Spaceball

Finger-ball

Results: Task Completion Times

Docking Performance over TimeIsotonic Position Device

Trial Number

0 200 400 600 800 1000

Tas

k C

ompl

etio

n T

ime

(sec

onds

)

0

5

10

15

20

25

30

35

40

Session Means

Docking Performance over TimeIsometric Rate Device

Trial Number

0 200 400 600 800 1000

Tas

k C

ompl

etio

n T

ime

(sec

onds

)

0

5

10

15

20

25

30

35

40

Session Means

Average M-metric Value for all Two-Way Comparisons

0.0

0.1

0.2

0.3

0.4

0.5

0.6

XY XZ YZXRX

XRYXRZ

YRXYRY

YRZZRX

ZRYZRZ

RXRY

RXRZ

RYRZ

M-m

etr

ic

Results: M-metric Scores 2-way Comparisons

within translation within rotationbetween translation & rotation

Results: M-metric Scores 3-way Comparisons

Average M-metric Value for all Three-Way Comparisons

0.0

0.1

0.2

0.3

0.4

0.5

0.6

XYZ

XYRX

XYRY

XYRZ

XZRX

XZRYXZRZ

YZRX

YZRYYZRZ

XRXRY

XRXRZ

XRYRZ

YRXRY

YRXRZ

YRYRZ

ZRXRY

ZRXRZ

ZRYRZ

RXRYRZ

M-m

etr

ic

within translation within rotationbetween translation & rotation

Results: M-metric Scores Over Time

within rotation

between translation & rotation

0

0.1

0.2

0.3

0.4

0.5

0.6

1 2 3 4 5

Session (Time)

M-m

etric

Isotonic Position Y-Z-RYIsometric Rate Y-Z-RYIsotonic Position RX-RY-RZIsometric Rate RX-RY-RZ

M-metric Summary

• new metric for measuring allocation of control

• tested in a longitudinal 6 DOF docking task

• subjects allocated allocated unequally control across all 6 dofs

• subjects controlled rotation & translation separately

• separation of control for the isometric device greater than for the isotonic device

Shortcomings / Future Work

• Need to define the “correct” path unequivocally• In docking, any trajectory which accomplishes the

docking goal is acceptable.

• Expand M-metric definition to include tracking, tracing, and target acquisition tasks.

• Next experiment : test M-metric on a dynamic 6 DOF tracking task

Conclusion:In a multi-degree of freedom continuous movement task:

• the M-metric provides a measure of how control is allocated across available DOFs

• it is possible to have two movements with equal performance scores, but with very different time-space trajectories

Acknowledgements

• Institute of Robotics and Intelligent Systems (IRIS)

• Natural Sciences and Engineering Research Council (NSERC)

• Shumin Zhai, IBM Almaden Research

• Ravin Balakrishnan, University of Toronto

Taxonomy of Manual Control TasksTime Domain

Externally Paced Self-Paced(Time Matching) (Time Minimizing)

Spac

e D

omai

n

D

istr

ibut

ed

Sin

gle

Loc

atio

n

L

ocat

ion

Target Acquisition ball catching instrument

playing at atempo

Target Docking menu

selection peg-in-hole

tasks

DynamicTracking target

gunnery driving at a

speed limit

Tracing drawing marking

menus

[Masliah ‘99]