2/11/2013

1

A Smartphone App to Assist the Visually Impaired at Intersection Crossings

Chen-Fu LiaoMinnesota Traffic Observatory

Department of Civil EngineeringUniversity of Minnesota

TRB 92nd Annual MeetingSession #459

January 14, 2013

Acknowledgements

RITA, USDOT; UMN ITS Institute; CTS; and MTO

Vision Loss Resources (VLR)

Steve Misgen & Tim Bangsund – MnDOT

Linda Spaulding & Nancy Prussing – Certified O&M Specialists

Avanish Rayankula, Sowmya Ramesh, Niveditha Baskar –Graduate Research Students

Kale Fischer, Jean Mullins – CTS Students

Gordon Legge – Department of Psychology

Michael Manser – HumanFIRST Program

Herbert Pick – Institute of Child Development

Henry Liu & Jianfeng Zheng – Department of Civil Engineering

Visual Impairment Services Team (VIST) coordinated by Broderick Burks at VA Medical Center, MN

2/11/2013

2

Outline

Background Accessible Pedestrian Signals (APS)Motivation Our Approach Recommendations from UI Survey Field Experiment and Results Lesson Learned Concluding Remarks

Low Vision Statistics

Every 7 minutes someone in U.S. becomes blind or visually impaired

More than 3.4 million (3%) Americans 40 years and older are legally blind or visually impaired

By 2030, the number of blind and visually impaired people is predicted to double

Alliance for Aging Research. (2010). The Silver Book: Vision Loss. http://www.silverbook.org/VisionLossSilverbook.pdfImproving the Nation’s Vision Health. http://minorityhealth.hhs.gov/Assets/pdf/Checked/FINAL_Improving_the_Nations_Vision_Health.pdf

2/11/2013

3

Accessible GPS for the VI

Sendero GPS for BrailleNotes (TTS & Braille)

Trekker Breeze talking GPS (TTS output) -Humanware

StreetTalk VIP (TTS & Braille output) –paired w/ PACMate

Mobile Geo (TTS & Braille support)

Loadstone GPS (Symbian phone)

Wayfinder Access (Symbian phone)

Others

Smartphone Apps for the BlindNavigation and Wayfinding

WalkyTalky (Android)

Announce a location as a person walks by it

Announce “turn-by-turn” walking directions and names of streets and landmarks along the way

Intersection Explorer (Android)

Learn directions before leaving home

Announce direction in which the finger walker is headed and distance traveled

Ariadne GPS, Sendero LookAround GPS, BlindSquare (iPhone / iPad)

2/11/2013

4

Potential Challenges at Crossings

No intersection geometry information

Locate the edge of street

Locate crosswalk

Interpret signal & traffic patterns

Align toward crosswalk

Locate pushbutton, if exists

Determine when to cross

Maintain alignment

while crossing

Intersection Crossing

48.6% of crossings started during walk interval and 26.9% of all crossings (w/o assistance) ended after the onset of the perpendicular traffic w/o APS – Barlow et al. (2005)

72% started with appropriate alignment, 42% ended outside the crosswalk w/o APS – Barlow et al. (2005)

With APS installation, 13% of crossings ended after perpendicular traffic began and APS reduced starting delay time by nearly 2 sec. – Scott et al. (2008)

Train pedestrians to use new cues or enhance their judgments with technology may be a more effective method of reducing crash events – Guth et al. (2005)

2/11/2013

5

Accessible Pedestrian Signals (APS)Brief History

Audible pedestrian signals starting in 1920 in U.S.

Not included in U.S. standards and regulations until MUTCD 2000

Cuckoo/cheep signals mounted on the pedestrian signal heads (pedhead-mounted APS) – mid 70’s

Integrated APS in pushbutton – mid 90’s

Latest Generation of APS

Audible and/or vibrotactile WALK signal indications

Pushbutton locator tone - repeats constantly at 1Hz to provide information about the presence and location of a pushbutton

Tactile arrow that points in the direction of travel on the crosswalk

Automatic volume adjustment

Walk time count down

2/11/2013

6

APS Installation

Costs - $6,000 per intersection plus labor

Noisy. Add 5 decibels of noise within 6 to 12 feet of pushbutton

Require additional stub, pushbutton station poles

Pushbutton pole location

Braille verification

Maintenance

APS at SE University & Berry St.LRT Westgate Station

2/11/2013

7

Motivation

Controversy between ACB and NFB

Provide personal pedestrian signal information with properly design user interface

Intersection geometry information

Blind pedestrian needs to locate and push the walk button by listening to separate audible cue

Audible messages are often confused with noises in the vicinity and heavy rush hour traffic

Recommendations from the VISummary of survey results

Present additional information about an intersection Use short auditory phrases Auditory and tactile information should not interfere

with the pedestrian’s ability to use their cane or listen to traffic cues

Use tactile cues for dangerous warnings A method to repeat warnings / output is necessary Allow for automatic activation of walk signal request

at an intersection; or Allow the user to activate a walk signal request

though a mobile APS device

2/11/2013

8

User Interface for a Mobile Device

User Feedback

Auditory or verbal (Text to speech, TTS)

Vibration

Tactile

User Input

Tactile (Tap on mobile phone or press a button)

Speech to text (STT, or voice input)

Braille overlay on smartphone screen

DrawBraille, a smartphone braille

SPaT Data Acquisition

TS2 Traffic Signal Controller Cabinet

Synchronous Data Link Control (SDLC)

MAPS DB Server

USB I/O

Wires toPushbutton

Smartphone

3G orWi-Fi

MnDOT Network

System Diagram of MAPS Data Communications

Firewall

3G orWi-Fi

UMN VPN

2/11/2013

9

Mobile APS User InterfaceGeometry Information

Single tap for orientation and geometry information

Text-to-Speech message

Heading North to 

Washington Ave  4‐Lane

Heading West to 

Oak Street  4‐Lane

Mobile APS User InterfaceSignal Information

Double tap to confirm crossing

Send walk phase request to traffic signal controller

Obtain signal timing information

Harvard. Walk sign is on to cross Harvard, 20 sec to cross

Wait for walk signal

2/11/2013

10

Field Experiment – Test Sites

Golden Valley, MN Winnetka Ave and

Golden Valley Rd, Crossing Winnetka 5

lanes Cross from NW corner to

NE corner Crossing time Approx 22

seconds APS equipped

APS Golden Valley, MN Minnesota Highway 55 &

Rhode Island Ave N, Crossing Rhode Island

Ave. 5 lanes, 111 ft Crossing from NE corner

to NW corner Crossing time 23 sec. Walk – 7 sec Don’t Walk – 16 sec

No APS installed

Non‐APS

Experiment Tasks

Crossing

Walk 100~200 ft toward

intersection

Destination

Origin

2/11/2013

11

System Evaluations

Subjective Measures

Sufficient info needed to cross intersection Sufficient time to cross Feel safe during crossing Usefulness & satisfaction

(Van der Laan et al, 1997) Trust (Lee & Moray, 1992)

Objective Measures

Walk speed at sidewalk Walk speed at crosswalk Initial alignment Time to step into

crosswalk Veering and # of veering # of assistance needed

from O&M specialist

Visually Impaired Participants

11 Male, 7 FemaleAge – average 44.2 years (SD 15.2 years) 12 cane users 5 guide dog usersCane users – average 14.2 years (SD 14.1,

Max 49)Guide dog users – average 7.1 years (SD

5.7, Max 15)

2/11/2013

12

Crossing Freq. & Phone Ownership

9

5

2 20

5

10

More than 4intersections a day

1 to 2 intersections aday

1 to 4 intersections aweek

Less than 1intersection a week

Intersection Crossing Frequency

1

98

0

5

10

No phone Smartphone Non‐smartphone

Mobile Phone Ownership

Self‐Assessment Navigation & Wayfinding

0

2

4

6

8

10

12

Cane Guide dog Askingothers

No outsideasstance

Other

Preferred Method of Navigation Assistance

0

2

4

6

8

10

12

Well below Below Average Above Well above

Travel Skills ‐ General Sense of Direction

0

2

4

6

8

10

Well below Below Average Above Well above

Travel Skills ‐ Independent Travel

0

2

4

6

8

10

Well below Below Average Above Well above

Travel Skills ‐ Signalized Street Crossing

2/11/2013

13

Intersection #1 (APS)

Intersection #1 Responses No Yes Don't Know

Use GPS Navigation Previously 16 2 0

Have Prior APS Experience 2 16 0

Prefer Pushbutton (PB) over No PB 1 17 0

Have Difficulty in Locating PB 14 3 1

APS Provides Sufficient Info 2 16 0

Have Sufficient Time to Cross 7 11 0

Feel Aligned to Crosswalk 2 14 2

Intersection #2 (Non‐APS)

Intersection #2 Responses No Yes Note

Prefer Pushbutton (PB) over MAPS 11 41 ‐ Both 

1 ‐ Neither

Have Difficulty in Locating PB* 9 8 0

Have Sufficient Time to Cross 2 15 0

Feel Aligned to Crosswalk 2 15 0

MAPS Provides Sufficient Info 6 11 0

MAPS Provides Helpful Geometry Info 3 14 0

MAPS Provides Helpful Signal Timing Info 6 10 1 ‐ Unknown

2/11/2013

14

Objective Measures

Intersection ID 1 2

Intersection Type APS Non APS

Crosswalk Length, meter (ft) 28.7 (94) 33.8 (111)

Crossing Task # 1 2 3

Sample Size (N) 18 17 17

Ped Call APS PB Regular PB MAPS

Crosswalk Speed, Average (mph) 2.9 3.4 3.3

Crosswalk Speed, SD (mph) 0.4 0.8 0.6

Sidewalk Speed, Average (mph) 2.5 3.1 3.3

Sidewalk Speed, SD (mph) 0.4 0.5 0.6

Sidewalk vs. Crosswalk Speed (p‐value) 0.01 0.06 0.77

Pushbutton Search Time, Average (sec) 7.8 26.6 NA

Pushbutton Search Time, SD (sec) 6.1 56.5 NA

In‐Position Time, Average (sec) 14.5 34.8 9.8

In‐Position Time, SD (sec) 7.5 57.2 6.7

Time to Step Into Crosswalk, Average (sec) 3.1 7.1 5.5

Time to Step Into Crosswalk, SD (sec) 1.5 4.7 3.2

Number of Veers Outside Crosswalk 4 2 4

Usefulness & Satisfaction

UsefulPleasantGoodNiceEffectiveLikeableAssistingDesirableRaising Alertness

UselessUnpleasant

BadAnnoying

SuperfluousIrritating

WorthlessUndesirable

Sleep-inducing

Cronbach's α = 0.96

Van der Laan et al. (1997)

‐2

‐1

0

1

2

‐2 ‐1 0 1 2

Satisfaction

Usefulness

Usefulness & Satisfaction

17 Participants Mean

2/11/2013

15

System Trust and Confidence

65%

71%

79%

57%

65%

0% 25% 50% 75% 100%

Performance (1,4)

Process (2,5)

Purpose (7,8)

Overall (3,9)

Confidence (6)

Lee and Moray (1992)

Decision Support System

Our goal is to provide decision support for the visually impaired, elderly, or disabled at signalized intersections

Not to replace the orientation and mobility skills they already learned

Complement existing APS system

Provide an integrated solution for navigation, wayfinding and guidance

2/11/2013

16

Lesson Learned

Privilege of observing 18 VI people, cross the same intersection all day

Every VI person is a different traveler- advocate for one on one mobility instruction

Guide dogs are very distracted by COMS following even 10-30 feet away

Teaching the blind technology requires a lot of training time. Build in additional time for GPS training and mobility lessons

Lesson Learned (Continued)

VI individuals need their O&M skills, MAPS App shall be used as an assistant only, never walk across the street just listen to the phone

Urban canyon and other factors will make GPS unreliable

Great blind traveler could get lost in unfamiliar environments – have client travel on their own –home work assignments, travel days after completion to keep skills up

2/11/2013

17

Future Directions & Opportunities

Work zone bypass and routing

Orientation and direction support at bus stops (transfers, route # and arrival time)

Veering warning

Wayfinding and navigation

Indoor (e.g. Minneapolis skyway)

Outdoor

Route planning and guidance

Application for elderly or disabled

Intelligent Mobility Applications

Inform driver when blind pedestrian is waiting at an intersection

Warn blind pedestrian not to enter crosswalk where there is potential red light runner

Extend pedestrian phase for registered blind, elderly, or disabled people

Connected Vehicles Initiative (V2I, V2P, P2I)

2/11/2013

18

Thank You !

Chen-Fu LiaoMinnesota Traffic Observatory

Department of Civil EngineeringUniversity of Minnesota

(612) 626-1697cliao@umn.edu