1 portable haptic aids for training and rehabilitation li jiang april 4 th 2008

1

Portable Haptic Aids for Training and Rehabilitation

Li Jiang April 4th 2008

2

Background VR Training Conclusion Stroke and Multiple Sclerosis

Presentation Outline

• Background

• Portable haptic aids for emergency personnel virtual reality (VR) training

• Portable haptic aids for stroke and multiple sclerosis

• Conclusion

• Future work

Emergency personnel training

Stroke Rehabilitation

3


Portable vs Grounded

Grounded Device Portable Device

4


Force

Vibration

PortableGroundedDesign Tradeoffs:

Performance

Price

Portability

……..

Haptic Devices

“Rumble” joystick Pager motors

Cyberforce Cybergrasp

5


Event-Cue vs Amplitude-Based Feedback

Event-Cue Feedback (ECF)Amplitude-Based Feedback (ABF)

ABF: analog information ECF: symbolic information

6


Portable Haptic Aid for Emergency Personnel Virtual

Reality (VR) Training

7


Haptic Devices for Emergency Personnel VR Training

“Immersive” training systems withhaptic feedback have been expensive and encumbering

8


Portable Haptic Aids for VR Emergency Personnel Training

Unlike motor-skill related VR training, VR emergency personnel training focuses on high level abstract skill training such as team procedures and collaborations in dangerous and confusing environments.

9


Questions to Answer through First Set of Experiments

• Can low-cost, portable vibrational feedback improve trainees’ training performance in emergency procedures?

• How does vibrational “event-cue” feedback compare with amplitude-based force feedback?

10


VR for Experiment I

Virtual environment generated by Half-Life videogame engine

*Half-life: a science fiction first-person shooter computer game.

11


Experiment I ScenarioThe task given to users was to go through a dark, cluttered and potentially hazardous corridor in the aftermath of an explosion.

15 random-order obstacles in the corridor,

A dim red light indicates the direction of the exit.

12


Portable Haptic Feedback

13


Experiment Procedure and Metrics

8 Subjects (5 male, 3 female)

Experiment order was randomized and counterbalanced *Subjects were told that time is not a metric in the memory trials

Speed Trials Memory Trials

Requirement Go through the corridor as fast as possible, avoiding collisions

One minute for negotiating the corridor (ample time to pass through the corridor).

Metric Time Number of obstacles recorded (i.e., remembered)Number of obstacles recorded correctly.Time*

Two Feedback Modes: ECF (event cue) and no-haptic feedback

14


Experiment Results: Memory

ECF Moderecorded

No-Haptic Moderecorded

Paired T test:

F(1,7) = 14 P<0.01(significant)

15


Experiment Results: MemoryPaired T test:

F(1,7) =8.75 P<0.01(significant)

ECF Moderecordedcorrectly

No-Haptic ModerecordedCorrectly

16


Experiment Results: Speed

Speed Trials

Time with ECF vibration feedback

Tim

e w

ithou

t vib

ratio

n fe

edba

ck

Memory Trials

Time with ECF vibration feedback

Tim

e w

ithou

t vib

ratio

n fe

edba

ck

17


VR generated by Half-Life Game engine:a dark and confusing building with rooms to navigate and clear.

Commercial vibration, force joysticks

Modified by Immersion Studio.

Experiment II: Event-Cue Vibration Feedback (ECF) vs Amplitude-Based Force Feedback (ABF)

18


Experiment MetricsWhile soldiers are clearing the building layout, they should stop at each door before entering each rooms to make sure there is no potential danger in the rooms. [Holifield, Leonard., “Close-Quarter Combat: A Soldier's Guide to Hand-To-Hand Fighting,” Paladin Press, May 1997.]

Subject Hostage

Subjects were asked to stop at each door to mimic this event. A contact between the subject’s body and the wall beside the door was counted as the completion of the check. The number of times a subject failed to execute this check is counted as an error metric.

19


Building Layouts and Feedback Methods

Typical building layout (layouts were chosen randomly from a variations on the one shown above)

Feedback Methods:

Amplitude-Based Force Feedback (ABF)

Event-Cue Vibration Feedback (ECF)

No Haptic Feedback (NHF)

Error metric: Number of missed safety checks

20


Experiment II Results

Bonferroni Corrected Paired T test:

F(1,11) = 14.86 P<0.01 (significant)

F(1,11) = 19.85 P<0.01 (significant)

12 subjects, experiment order counterbalanced

21


VR Emergency Personnel Training Experiment Results

• Can portable event-cue vibrational feedback improve trainees’ training performance?

• How does vibrational event-cue feedback compare with amplitude-based force feedback in VR emergency personnel training scenarios?

Yes, Significantly !

Event-cue vibrational feedback can improve subjects’ performance as well as amplitude-based force feedback.

22


Portable Haptic Aids for Stroke and Multiple Sclerosis

23


Previous Haptic Feedback Devices

The ARCMIME system and MIME system

24


Haptic Feedback Strategy Framework

Force Sensor

Force information collected by sensors on the impaired hand are rendered to the healthy hand through small vibrational tactors

25


Three Feedback Modes Provided

NHF mode: No-Haptic Feedback

ABF mode: Amplitude-Based Feedback

ECF mode: Event-Cue Feedback

26


Stroke and MS Experiment Goals

• Can vibrational feedback improve patients’ ability in manual force control?

• How does the event-cue feedback (ECF) strategy compare with the amplitude-based feedback (ABF) strategy?vs.

27


Stroke Patients Multiple Sclerosis Patients

Poor Hand Motor Skill Better Hand Motor Skill

Not able to open impaired hand Able to open impaired hand

Comparison of Impairments

Conclusion: First, we need a device to help stroke patients to open their impaired hands

Sensory loss Sensory loss

28


Hand-Opening Device Design

Design Process:

Six design iterations and prototypes (details in the thesis)

Each prototype was evaluated by stroke patients and therapists.

Design Goal:

A passive mechanical system that can transfer power from healthy hand to the impaired hand to help the impaired hand open.

Impaired Side Healthy Side

29


Hand Opening Device DemoWithout the help

from the healthy side With the help

from the healthy side

30


Feedback Strategy for Stroke

Healthy Hand Impaired Hand

Microcontroller and circuits

Force Data

Box with Force SensorsVibration

Tactor

Impaired hand grasps an instrumented box, with help of vibration feedback applied to the back of the opposite hand.

31


Tactor: A Low-Cost, Small, Wearable Vibrational Feedback Device

Pulse Width

Period …. t

V 5V

Pager motor output (magnitude of vibratory force) is a nonlinear function of applied voltage or current.

Best results are obtained by creating a pulse train with varying frequency andduty cycle.

32


Event-Cue Feedback Mode

Event-Cue Feedback (ECF)

grasp force is below desired threshold grasp force exceeds desired threshold

33


Amplitude-Based Feedback Mode

Vibration pattern:

Period varies Reciprocal to measured grasp force.

Pulse width increases with the increase of grasp force.

34


ABF mode Perception Test

-1 0 1 2 3 4 5 6 70

10

20

30

40

50

60

70

80

90

Measured Grasping Force

Perc

eive

d m

agni

tude

R2 = 0.9909Slope = 13.7887

Based on data of 6 subjects

35


Force Sensing Experiment with Stroke Patients

Instrumented object with force sensors to measure the grasping forcesthat subjects provided with their impaired hand.

Force Sensor

ForceSensor

Grasp Force

36


Pilot Experiment with Stroke Patients • Three Subjects (2 male, 1 female).

• Task:Subjects were asked to grasp an object and try to use minimum force to maintain a stable grasp without dropping the object for 10 seconds and then replace it.

• Two metrics: Force Failure rate

• Three feedback modes:

NHF ABF ECF

37


ANOVA test: p=0.0005 Bonferroni corrected paired T test:

NHF vs ABF: p<0.005, NHF vs ECF: p<0.0002.

ABF vs ECF p<0.905

Experiment Data Analysis

No significant results were found in Failure

Rate

38


Masku Neurological Rehabilitation Center

39


Feedback Strategy for Multiple Sclerosis

Healthy Hand Impaired Hand

Microcontroller and circuits

Force Sensorattached to fingerpad

Vibrational tactorattached to fingernail

40


Force SensorsExperiment with Multiple Sclerosis patients

41


Experiment DesignGoal:Determine whether portable haptic feedback can improve patients’ finger force control ability.

Task:Grab an object and raise it up from the desk. Subjects were asked to balance the forces they applied on the index, middle and ring finger. When they feel the force is balanced, they should hold the object for 5 seconds and then put it back to the desk.

Feedback Modes:NHF Mode, ABF Mode, ECF Mode

ECF: users alerted whenever one finger apply too much or too less force.ABF: vibration pattern (intensity) is in proportion to the measured force

42


Experiment Procedure

• Every user completed the task under all three modes in one of the six possible orderings (NHF-ABF-ECF, NHF-ECF-ABF, ABF-ECF-NHF, etc…).

• Ample time was provided for pre-test practice to minimize learning effects.

• Practice trials were provided every time the mode was switched.

24 Subjects (8 male, 16 female, age range: 33 to 64 with a mean of 56.4). The recruited subjects all have reduced sensation in one hand and good sensation in the other hand.

43


Force Data Analysis

Average force : Fa = (Fi+Fm+Fr)/3;

Fi: Index finger force Fm: Middle finger force Fr: Ring finger force

The metric:sum of the force differences with respect to the average force:

Force_sum = abs(Fi – Fa)+abs(Fm – Fa)+abs(Fr – Fa)

Analysis uses the last 5 seconds of force data before subjects released the box. (Subjects were asked to maintain force balance for 5 seconds before replacing the box)

44


Sum of Force Differences for 24 Subjects

NHF ABF ECF0

0.5

1

1.5

2

2.5

Sum

of f

orce

diff

eren

ces

to th

e av

erag

e fo

rce Sum of Force Differences in 3 Different modes

Bonferroni corrected PairedT test:

NHF vs ECF, p < NHF vs ABF, p <

ABF vs ECF, p < 0.27

45


Nine-Hole-Peg Test

Task: placing 9 dowels in 9 holes.

Subjects are scored on the amount of time it takes to place and remove all 9 pegs.

It is widely used in MS clinical trials to quantitatively assess upper extremity function.

Impairment Level:

IL = (Subject’s completion time/Normal completion time*) - 1

*for subject’s age group

Looking for evidence of correlation between impairment level and most useful type of feedback

46


more impairedless impaired

10-1

100

101

-40

-30

-20

-10

0

10

20

30

Impairment Level (IL)

Diff

eren

ce in

Per

cent

Impr

ove

men

t for

AB

F vs

EC

FA

BF

bett

erE

CF

bett

er

Difference in Improvement for ABF vs ECF, with Respect to NHF Baseline

R2 = 0.53

47


ABF vs ECF: p < 0.00001 ABF vs ECF: p < 0.00001

Sum of Force Difference for More and Less Impaired Groups

More impaired group (14 subjects) Less impaired group (10 subjects )

48


Event-Cue Feedback vs Amplitude-Based Feedback

Mildly impaired patients performed better with event-cue feedback (ECF), while severely impaired patients performed better with amplitude-based feedback (ABF).

49


Failure Rates for the Different Modes

Significance found:

NHF vs ECF, p < 0.0006;

NHF vs ABF, p < 7·10-8;

ABF vs ECF, p < 0.002;

50


Summary of Stroke and MS Studies

• Can portable vibrational feedback (ECF or ABF) improve patients’ ability in grasping force control?

• How does the amplitude-based feedback strategy compare with the event-cue feedback strategy?

Yes! For stroke: 15% For multiple sclerosis: 60%

They both result in reduced failures.

For mildly impaired patients: ECF > ABFFor severely impaired patients: ABF > ECF

ABF results in less failures than ECF overall.

51


• Event-cue, portable vibration feedback can significantly improve trainees’ performance in VR training procedures. – It works as well as proportional force feedback for certain training procedures.

• Event-cue and amplitude-based vibrational feedback, applied to the opposite hand, can significantly improve the ability of stroke and multiple sclerosis patients’ to control their grasp forces.

• Mildly impaired MS patients performed better with event-cue vibration feedback while severely impaired patients performed better with amplitude-based feedback mode.– This suggests that where a feedback channel is mostly intact, event cues are

helpful to maintain performance (e.g. due to distraction of fatigue).Conversely, where a feedback channel is absent, it is best to provide one.

Conclusions

52


Other Contributions

• Designed a novel passive mechanical hand-opening device for stroke patients that can transfer power from the healthy hand to impaired hand.

• Developed a framework for feeding force information from the impaired hand to the healthy hand and proved its validity through experiments on stroke and MS patients. Part of the framework includes a method for mapping force magnitudes to the period and duty cycle of trains of vibration pulses.

53


Future Works

• Rehabilitation results have not been not shown in this research because of time limitations. Further studies could be done to determine whether any lasting rehabilitation results after long-term use of the devices.

• The concept of putting multiple, small tactors with patterns of pulses on the body could be extended to other training applications.

54


Acknowledgements

My great thanks to :

My advisor:

Professor Mark Cutkosky

Other mentors:

Professor Larry Leifer

Professor Machiel Van der Loos

Professor Joan Savall

My labmates:

Karlin Bark, Jason Wheeler, Yonglae Park, Sangbae Kim and all other BDML labmates

55


Thank you very much!Questions?

1 portable haptic aids for training and rehabilitation li jiang april 4 th 2008

Documents