textsl: a screen reader accessible virtual world client for second life

Human Computer Interaction ResearchUniversity of Nevada, Reno

TextSL: A Command-Based Virtual World Interface for the Visually ImpairedEelke Folmer, Bei Yuan, Manjari Sapre, Dave Carr - ASSETS 2009, Pittsburgh


Overview

Virtual WorldsBarriers to AccessTextSLUser StudyFuture Research

Virtual Worlds


Virtual Worlds

Second Life, World of Warcra&, Habbo HotelControl Avatar through 3rd person interfaceCommercial success80% of internet users will have VW account [gartner]Potential to replace the web

Second Life Habbo Hotel Home


VW Taxonomy

StoryGoalsScoreCombatDeath

Game VW

No game elementsUser generated contentSocial communitiesPlethora of different

usages & experiences

Social VW


Virtual World Viewer

“browser” like functionality:» exploration/navigation» communication» interaction» content creation

Education Museums Communities

Barriers to Access


So&ware Interaction

natural interactionmore immersive experience


Barriers to Access

Switch / Scanning

Screen reader“ ........................”

No textual representation

high degree of interaction

Provided Audio inadequate

Audio

location? objects?

avatars?


Research / Motivations

?

Education (Section 508)Socialization opportunitiesDisabled Communities (Virtual Ability)Include users with VI in our Information Society

Second Life no functional vision

TextSL


Requirements

ExplorationCommunication InteractionContent creation

UsableAccessible


Requirements

ExplorationCommunication InteractionContent creation

UsableAccessible

most important!


Solution StrategiesVI Accessible GamesSecond Life

Terraformers

AudioQuake

Shades of Doom3rd person interface

OutputSynthetic Speech

Audio Cues

Inputarrow keys

shortcuts

Earcons

Sonar

Powerup


VW LimitationsSecond Life?

objects have name field

Output

Synthetic Speech

Audio Cues

Earcons

Inputarrow keys

Sound radar

shortcuts

don’t control the content

no combat

Generic “use” too limited

difficult to navigateSit ontake

Drive Car

buy

Open Door

...............

lots of different objects

maybeno yes

miaw .......


Multi user dungeon games

Text OnlyCommand BasedIterativeNatural LanguageScreen reader accessible!

Zork

Extracts text from SLScreen reader /

customizationVW Agnostic low end machine

Text SL


Interpreter / Commands

Exploration (move, teleport, describe, where)Communication (say, whisper, mute)Interaction (sit on, touch)Content creation (not yet)Support (help / Tutorial)

walk

go

move

move

natural language prepositions

move to chairthe

adjectives

give my flower to jane


Text SL output

chair

firedog

jill

Text SL

“you see a chair, a dog, a fire and jill.”

“the chair is to your right”

screen reader

spatial queries

Direction agnostic360 degree rangespatial queries interact & communicate

interaction “sit on chair.”

“say hi to jill”communication


Second Life Content

Densily populated with objects»Overwhelm users with feedback»Difficult to navigate collision free

Lack of meta data (40%)» underwhelm users with feedback

tree

?fire

bike

dogtree

moe

jack

this object has a really long name

wall

chair

tree

table

jill

curly

carchest

larrybike bike

bike

tree

tree

?

?

??

?

?

bike


Summarizer & Path Finding

Summarizer: Cull non-descriptive objectsRank objects on distance and name lengthPathfindingmental mapping?normalization [nirje]

tree

fire bike

dog

tree

moe jack

cat

wall chair

tree

table

jill curly

car

chest

larry

“you see 5 people and 12 objects.”>describe objects“you see a tree, a dog, a cat......”>describe tree“this is a green spruce tree.”

?

??

>move north 20


Demo

User Study


How to evaluate VW?

Usage defined by content Focus on eval browser functionality» exploration» interaction» communication

Education Business Games

how much fun?supports learning? how profitable?


Hypotheses

Accessibility

usability

H0: TextSL allows exploration, communication, interaction with the same successrate as the SL viewer

textSL command based interaction screen reader output

H1: TextSL allows exploration, communication, interaction with the same learnability, efficiency, memorability, errors & satisfaction [Nielsen] as the SL viewer


User Study Design

1.Tutorial (pass)2.(explore, interact, talk)+

3. teleport to new location4.repeat 2.5.play with client (5-20 minutes)6.questionnaire

SL Viewer TextSL8 sighted 8 screen

reader users

videologs

islandtutorial agent


Task Completion Rates

0

10

20

30

40

Exploration communication interaction

Second Life TextSL

Accept H0: (Fishers’ exact probability test P=1.0, α=0.01)Accessibility


Task Performance Times


Questionnaire

0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Command Screen reader Efficiency feedback

2.8

1.81.6

2.3

Good

Bad


UsabilityAttribute Metric TextSL SL Viewer

Learnability successful completion

1.0 (SD=0.0) 1.0 (SD=0.0)

Efficiency Task completion times

explorationcommunicationinteraction

explorationcommunicationinteraction

Memorability Help / Help menu

1.0 (SD=0.7) 0.0 (SD=0.0)

Errors unrecognized commands

0.75 (SD=0.7) 0.38 (SD=0.2)

Satisfaction Questionnaire < 2.5 -

Reject H1


Conclusions

TextSL is: Accessible ✔Usability: Slower ✖»but acceptable (questionnaire) ✔»Communication is efficient ✔

Command based approach feasible»content creation »object interaction

Future Work


Meta Data / Feedback

Lack of meta data»Raise Awareness among VW Developers»Enforce names for objects»post hoc automatic labeling

Feedback»Audio for objects»voice over IP»Taxonomy

miaw

you see 3 animals


Content Creation / Interaction

Interaction with scripted objects?

Command based 3d object creation?

“the billboard started playing a video.”

“the object turned red”

>create a green cube

>create a dog


AcknowledgementsIIS-0917362: HCC-Small: TextSL: A Virtual World Interface for Visually Impaired. (Eelke Folmer / George Bebis)

IIS-0738921: HCC-SGER: Developing an Accessible Client for Secondlife (Eelke Folmer)

Bei YuanManjari SapreDave Carr

HTTP://www.textsl.orgmore info?

feedback?

contribute? collaborate?

download?

http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0917362

http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0917362

HTTP://www.textsl.org

HTTP://www.textsl.org


Questions?

textsl: a screen reader accessible virtual world client for second life

Technology