Paper Review:

Xueliang Huo et. al.

A Dual-Mode Human Computer Interface Combining Speech and Tongue Motion for People

with Severe Disabilities

Presented By

Md Kafiul Islam

Outline

• Introduction Motivation

• Paper Summery Main Contribution: dTDS

Performance Evaluation

Results

• Conclusion

• Q & A

Motivation• Assistive Technology (AT) for Severely Disabled People

• Use computers

• Control wheelchairs

• Control other home/office electronic appliances

• Problem with other ATs

– Head movement based ATs

• Not suitable with high level SCI

• Head should always be in positions within the range of the device sensors

– Eye movement/EOG based ATs

• Affect the users’ eyesight by requiring extra eye movements

– EEG based BCI as AT

• Very slow and limited in bandwidth

• Implantable BCI requires brain surgery

– Costly

– Long-term reliability

– Possibility of infection or brain tissue damage

Tongue Drive System• Why Tongue

– Most flexible and capable parts of human body– Occupy an amount of sensory and motor cortex

that rivals that of fingers & hand– Escapes severe damage even in high level SCIs– Noninvasive access to the tongue is readily

available– Last to be affected in most neuromuscular

degenerative disorders– Many degrees of freedom, can move very fast

and accurately within the mouth cavity– Certain degree of privacy

Operational Principle:

Early Stage iTDS (intra-oral TDS)

Dual-mode Tongue Drive System, dTDSSpecial Features

Independently processed tongue motion and speech

Significantly higher performance than either unimodal forms

Particularly in completing tasks that require both pointing and text entry.

TDS (primary) modality is used to enable/disable SR (secondary) modality via GUI to reduce power

Increased Speed, Flexibility, Usability and Independence

Major Components of dTDS

Performance Evaluation• Subject

– 14 able-bodied subjects • age 21-30, 9 males, 5 females• 7 had prior experience of TDS,

others are new• 7 native, 7 non-native English

speakers

– 3 severe SCI subjects• all male, aged 37-53• TDS experienced before

• Experimental Design– Each subject repeating the same

task using 3 devices: TDS, Dragon (SR-based) and dTDS

– Tasks are• Text transcription task• Maze navigation task• Center-out tapping task

• Experimental SetupTyping Task

Center-out Taping Task

Maze Navigation Task

Results

Avg. Time taken for Maze Navigation Task by SCI Subjects

Avg. Time taken for Center-out Tapping Task by SCI Subjects

SCI patients’ subjective rating on different devices at the end of sixth session.

Results (Cont…) Effect of Different Factors on dTDS Maze Navigation Performance

Effect of Different Factors on dTDS Center-out Tapping Performance

Conclusion

• Successful demonstration of dTDS

• Significant improvement in terms of speed for SCI subjects

• Using dTDS doesn’t affect the user’s performance with either one of the

two modalities

• Prior TDS experience helps to perform better

• Significant effect of accent on EF typing in modified center-out typing task

• Future work includes to explore addition of other input modalities and

add proportional control ability to TDS modality.

Q & A

Q & A

Related Links

https://www.youtube.com/watch?v=nD7A39Bj0zs&feature=related

https://www.youtube.com/watch?v=KZHBNYd-eWs

http://users.ece.gatech.edu/mghovan/index_files/TongueDrive.htm

http://www.ece.gatech.edu/research/labs/gt-bionics/research_tds.shtml

Sip-and-Puff Wheelchair https://www.youtube.com/watch?v=bYKUxOdUAao