navigating a 3d avatar using a single switch

Player-Game Interaction ResearchUniversity of Nevada, Reno

Navigating a 3D Avatar using a Single SwitchEelke Folmer, Fangzhou Liu, Barrie Ellis - FDG’11, Bordeaux

Thursday, June 30, 2011


Interaction

★2D★Text

★3D★Visual



Users with Disabilities

Screen reader

“ The eiffel tower was built in ...”

Assistive Technology



“Gamers” with Disabilities

“ ...........”


Player Game Interaction ResearchUniversity of Nevada, Reno

Severe Motor Impairments

Quadriplegics

one handed controller

eye tracker quad controller switch


Human Computer Interaction ResearchUniversity of Nevada, Reno

Adapted Controllers2-5 analog 13 buttons

1 binary

?

Ability

more

less

1 analog

mapping

1 analog5-7 binary



Switch Controller

Severe Motor/Cognitive impairmentsSturdy / CheapNo calibration / easy to useBinary input (on/off)

jellybean sip/puff pinch



Input Reduction

?



Input Reduction

Automate

?



Input Reduction

AutomateRemove

?



Input Reduction

AutomateRemoveScanning

?



Scanning 101

A B C

Selection Set Input generated

scanning rate



Scanning 101

A B C

ActivateSelection Set

90

Input generated

scanning rate

C



Scanning Pattern

A B C

D

G

E

H

F

I

Cartesian Polar

AB

C

D

E

F

G

E



Scanning Control

A B C

hold to scan

Not always possible Unable to hold switch (Arthritis)Sip and puff



Scanning Control

A B C

release to select

C

Not always possible Unable to hold switch (Arthritis)Sip and puff



Discrete Selections

H E L L OText Entry

Web Navigation

Menu Navigation



Avatar Navigation

Selection set [FORWARD,LEFT,RIGHT,BACK]Types of input:Continuous [FORWARD] for 300msMixed [FORWARD + TURN RIGHT]

400ms 600ms 600ms 800ms



Continuous input approximation

[Forward] 300ms

[Forward, Le1, Right, Back]

input to generate

selection set




[Forward] 300ms


input to generate

[Forward] [Forward] [Forward] 100 ms

3 switch activations

selection set




[Forward] 300ms


input to generate

selection set




[Forward] 300ms


input to generate

[Forward] 200 ms

1 switch activations

selection set



Mixed Inputs Approximation[Forward+Le1] 200 ms




[Forward] [Le1] 200 ms

2 SA




[Forward] [Le1] 200 ms

2 SA

[Forward] [Le1] [Forward] [Le1] 100 ms

4 SA



Navigating an avatar




[Forward] [Le1] [Forward] [Le1] [Forward] [Right] [Forward] [Forward] [Forward] [Right] [Right]




[Forward] [Le1] [Forward] [Le1] [Forward] [Right] [Forward] [Forward] [Forward] [Right] [Right]

SlowCumbersomeUnnatural



Research QuestionCan we develop a more efficient scanning system that can generate continuous and mixed inputs?



Improved Scanning Control

[Forward]

[Forward]Discrete

[Forward]hold and release

[Forward] [Forward] [Forward]

hold release

Continuous input

less efficient more efficient



Generating Mixed Inputs

[Forward+Le1]

[Forward, Le%, Right, Back, Forward+Right, Forward+Le%, Back+Le%, Back+Right]

★Extending selection set

[Forward, Le%, Right, Back]

[Cancel, Le%, Right][Forward]dynamically generated

★Multistep selection

[Le1]



Selection set order[Forward, Le%, Right, Back][Forward] cost

[Back, Le%, Right, Forward]

wait wait wait 3SR+RT

RT

More frequently used inputs should be at the front of the selection set



Parameters to Compare

Discrete Hold and Releasecontrol

mixing Extension Multistep



selection sets

dia 100,200....1,000 ms

vs

vs

10

24

40,320

[Forward, Le%, Right, Back] [Cancel, Input, Input] 144

2



Parameters to Compare





selection sets

dia 100,200....1,000 ms

vs

vs

10

24

40,320


2

~400k



Characterizing Avatar Navigation

8 Able bodied users Focus on grounded navigationNavigated in SL Environment for 8 minute

Forward 873 Forward+Left 220Left 293 Left+Forward 167 Forward 675

Second Life Key Logger



Keystroke frequencyTable 1: Characteristics of Avatar Navigation.

Input Single Multi Total TimeFORWARD 74.9 - 46.8 79.4RIGHT 10.6 - 6.6 6.1LEFT 12.7 - 8.0 7.0BACK 1.9 - 1.2 1.8FORWARD+RIGHT - 46.7 17.5 10.3FORWARD+LEFT - 52.4 19.6 8.9BACK+RIGHT - 0.5 0.19 0.12BACK+LEFT - 0.5 0.19 0.21

total 100% 100% 100% 100%

key that is held the longest (86.92% of single & multi) of thetotal time participants held keys. RIGHT and LEFT are usedless frequently in single strokes but comprise 46.21% and52.71% of multistrokes. This confirms our assumption thatmultistrokes are primarily used to adjust the course of anavatar. BACK is rarely used. We consider the collected datato be representative of 3D navigation behavior in generaland we use it to inform the design of our scanning system.

4.2 Optimization Criteria

Approximating continuous and mixed inputs with discreteinputs is inefficient and leads to approximation errors (seeSection 2). Consequently to improve upon existing scanningsystems we seek to minimize the following three parameters:

1. Average time (T ) to generate an input.

2. Average number of switch activations (SA) required togenerate an input.

3. Approximation errors (error).

Where T depends on SA it also depends on the length ofthe set of input symbols that we need to scan over (to mixinputs we may expand the input set) and the scanning rate(SR). User error, e.g., the user failing to activate the switchwhen the desired input option is highlighted or accidentallyactivating a wrong input option, is considered to be propor-tionally related to the number of SA’s required to generatethe input. User error is primarily affected by SR, whichhas been extensively studied in previous work [11, 13, 21].For example, studies have shown that an optimal SR canbe achieved by dividing a user’s response time by 0.65 [21]and therefore we do not incorporate it in the design of ourscanning method.

4.3 Continuous input

Rather than making a discrete selection when the switch isactivated, we modify the scanning control method to holdthe input until the user releases it. After selecting an inputthe scanner will keep highlighting the activated input whichwhen the user presses their switch will release it. This inher-ently increases the cost of providing an input with one extraswitch activation, but for inputs that are held for longer thantwo times the time a discrete input would be activated, hold-ing and releasing an input is anticipated to be more efficient.This is further motivated by the observation that the small-est key press we found was 26 ms and the average keystrokelength 804 ms. The set of inputs to be scanned over can alsobe optimized. For example, our study found FORWARD to

be the most frequently used input. The input set should berearranged as such to allow inputs with a higher frequencyof occurrence to be scanned to faster.

4.4 Mixing inputs

Adding an input to an input already activated input can befacilitated using one of the following two strategies:

1. Extending the set of inputs with symbols that repre-sent mixed inputs, e.g., FORWARD+RIGHT (F+R).

2. Multistep selection where after selecting an input, e.g.,FORWARD, we scan over a subset of inputs that can bemeaningfully added to the currently active input orcancel the active input, e.g., [RIGHT, LEFT, CANCEL].

Each strategy involves different tradeoffs that affect effi-ciency and error of generating an input. The average timeto scan to an input will increase for a larger selection set.If the user wants to mix inputs, they must cancel the cur-rently held input and then wait for the mixed input optionto become highlighted in the set. For an SR of 1 second,and a scanning set of [F, L, R, B, F+L, F+R, B+L, B+R] itwould already take 5 seconds to activate F+R. Expandingthe selection set stops the avatar when a different input isselected, which allows for precise but slower navigation.

The selection set can also be rearranged based on the like-lihood of this input being selected, where the results of theuser study can be used to inform how the selection set is opti-mally composed. For example, an analysis of the keystrokescollected from our user study (see Table 1) reveals thatusers used the input FORWARD+RIGHT (17.5%) significantlymore than FORWARD (7%). By rearranging the selection setwe should allow for such inputs to be scanned to faster byputting more frequently used inputs at the beginning of theselection set. The length of the selection set significantlyaffects the time to scan to an input. Rather than includingall available inputs in the selection set, we could scan over asubset. For example we could cull single and mixed inputsinvolving BACK as these are rarely used. This will increaseerror as we must approximate these inputs using discreteinputs, e.g., similar to how currently mixed inputs must beapproximated using existing scanning systems.

Multistep selection is potentially more efficient as users canadd an input while a key is held and adjust the course ofan avatar while it is moving rather than having to stop theavatar when selecting a mixed input option. This may allowfor more efficient navigation, but this comes at the cost ofan approximation error, e.g., when an input is held the usercan either add a new input or cancel the current input orwhen both inputs are held the user could also cancel eitherof them. The problem is however that these additional inputoptions must be made available to the user using scanning,which introduces a small delay, as the user must wait untilthe desired input option is highlighted. For example, if theuser is navigating their avatar using FORWARD and all of asudden encounters and obstacle and wants to add RIGHT, theuser must wait until the scanner is highlighting RIGHT in theavailable selection set [RIGHT, LEFT, CANCEL]. Dependingon the specific SR used the average time it takes to selectan input from the subset is on average 1.5∗SR, which may

4,300 keystrokes totalSingle: 62.6%Multi: 37.4%



keystroke length

min: 26ms max: 30,606msaverage: 804 ms (δ=1,404)



Parameters to Compare update





selection sets

dia 25, 50,....1,000 ms

vs

vs

2

40

24

40,320

[Forward, Le%, Right, Back, Forward+Right, Forward + Le%]

120


7 sets total



Parameters to Compare update





selection sets

dia 25, 50,....1,000 ms

vs

vs

2

40

24

40,320

~3.3 M [Forward, Le%, Right, Back, Forward+Right, Forward + Le%]

120


7 sets total



Simulator

Simulator



Simulator

Simulator


keystrokes



Simulator

Simulator


keystrokes


selection sets



Simulator

Simulator


keystrokes


selection sets

[Forward, Back, Right, Le1]

permutations

[Back, Le1, Forward, Right][......]



Simulator

Simulator


keystrokes


selection sets

25<i<1,000, i+=25dia


permutations




Simulator

Simulatoraverage per stroke:★time (ms)★switch activations★error (ms)


keystrokes


selection sets

25<i<1,000, i+=25dia


permutations




Simulator

Simulatoraverage per stroke:★time (ms)★switch activations★error (ms)


keystrokes


selection sets

25<i<1,000, i+=25dia


permutations


SR = 1,000msRT = 650ms



Results



Results

Discrete



Results

H&R [Multistep] »time: 1,539 ms per stroke »error: 122 ms (~35 cm)

Discrete [Extended] »time: 3,442 ms per stroke »error: 114 ms (~32 cm)

H&R [Extended] »time: 2,444 ms per stroke »error: 0 ms

Hold and release approximates keystrokes the best.

Significantly faster p<.01

Significant fewer errors p<.01

H&R 2.4x slower than keyboard


Human Computer Interaction ResearchUniversity of Nevada, Reno

Demo / Implementation



Limitations

User error modeling is limitedassume user has perfect timing skills»selection accuracy >90% but not 100%

correction may be difficult --> extended setmodel error using Bayesian noise & error

correction strategies. switch users may develop different behavior -->remove [Back]



Future Work

Other VW functions»Object interaction

User studies with switch users

Real time response environments (FPS/ Racing game)

Hold and Release integrated in console OS as HUD



Questions



Questions

?Thursday, June 30, 2011

navigating a 3d avatar using a single switch

Technology

compare control discrete

mixed inputs approximation

cartesian polar player

continuous input approximation

forward le

selection set

scanning pattern

000 ms selection