Mobile HCIMobile HCI

Presented by Bradley Barnes

Mobile vs. StationaryMobile vs. Stationary

Desktop – StationaryUsers can devote all of their attention to the

application.Very graphical, detailedUse the keyboard and mouse for input

Mobile and Wearable DevicesUsers in motion – can’t devote all of

attention to the applicationLimited screen real estateInput and output capabilities are restricted

for users on the move

Mobile Device InteractionMobile Device Interaction

The interface for mobile and wearable devices continues to mimic those of desktop computers.

New interaction techniques are needed to safely accommodate users on the move.

Interaction should be subtle, discreet, and unobtrusive.

Mobile Interaction MethodsMobile Interaction Methods

KeyboardTouch ScreenSpeech RecognitionHead motion, 3-D soundEyeglass displays with Gestural Interaction

CHI2005 PaperCHI2005 Paper

Toward Subtle Intimate Interfaces for Mobile Devices Using an EMG Controller

Using a mobile device in a social context should not cause embarrassment and disruption to the immediate environment.

Intimate InterfacesIntimate Interfaces

Discrete interfaces that allow control of mobile devices through subtle gestures in order to gain social acceptance.

Must take into account the social context where the interaction will occur.

Most interaction occurs around other people (bus, train, street, etc.)

Electromyographic SignalsElectromyographic Signals

EMG signals are generated by muscle contraction.

Signals are picked up by contact electrodes.Allows a definition of “subtle” or

“motionless gestures” that can be used to issue commands to mobile devices.

Can sense muscular activity not related to movement.

The SystemThe System

Armband controller recognizes gesturesSignals transmitted via BluetoothCompliant device receives signals and

performs appropriate action.Can be PDA, mobile phone, etc.The armband works on all users: no

calibration or training required.

When combined with eyeglass displays, the system becomes “hands free”.

Can be operated when users are carrying items.

Can be used in specific fields, such as maintenance, for assistance when the user’s hands are tied.

Design ProcessDesign Process

Iterative process centered on users. Three pilot studies and one formal Study.

Pilot Study 1: Bicep is chosen muscle, and the gesture is defined as a brief unnoticeable contraction of the bicep.

Pilot Study 2: Refine the gesture definition, and create an algorithm for its detection.

New subjects w/ variety of muscle volumesGesture not fully described to subjectsCompared EMG signals of gesture to those

of normal activity.Algorithm detects peaks in the EMG

signals.

Pilot Study 3: Fine tuning of the system, wanted to test for false positives and false negatives

Consisted of new and returning usersWorked with the algorithm until the number

of false positives approached zeroAlso, they decided to try a gesture alphabet

with two gestures. They are defined as two short contractions of different duration.

Formal Study: Validation of ResultsPilot studies set up the system parameters

by testing gestures on subjects who were not mobile.

Conducted to assess the usability of EMG as a subtle interaction technique for mobile devices.

Evaluated the system usability in a mobile context.

Formal Experiment DesignFormal Experiment Design

10 adult participants-Ages 23 to 34Perform 5 walking tasks – one with no

contraction to calculate misclassification rate, and other four with contractions of different durations.

Subjects make laps around obstacles while doing the contractions

Familiarization sessions preceded the walking tasks.

These involved standing and making contractions. The participants were prompted to contract by a MIDI piano tone delivered through the wireless headphones.

System recognized contractions between 0.3 and 0.8 seconds, but the subjects did not know the duration of their contraction: only that the system recognized it.

TasksTasks

1) Walking, No Contractions – 10 laps2) Standing, Familiarization, Generic

Contractions3) Walking, Stimulus-Response, Generic

Contractions4) Standing, Familiarization, Short

Contractions

5) Walking, Stimulus-Response, Short Contractions

6) Standing, Familiarization, Long Contractions

7) Walking, Stimulus-Response, Long Contractions

8) Walking, Stimulus-Response, Mixed Long and Short Contractions (low tone – long contractions, high tone – short)

ResultsResults

The online recognition rates for the four walking tasks were:

Generic: 96%Short: 97%Long: 94%Mixed: 87%

ConclusionConclusion

An EMG based wearable input device can be used for subtle and intimate interaction.

The system presented can recognize motionless gesture without training or calibration.

EMG gestures can be utilized as a socially acceptable alternative for mobile device interaction

Future WorkFuture Work

Expand gesture alphabetTest in more “real world” scenarios, like

when lifting something.

ReferencesReferences

Lumsden, J., Brewster, S. A paradign shift: alternative interaction techniques for use with mobile & wearable devices. Proc. Of the 13th Annual IBM Centers for Advanced Studies Conference CASCON’2003.

Costanza, E., Inverso, S., Allen, R. Toward Subtle Intimate Interfaces for Mobile Devices Using an EMG Controller.