Helping Hand

Pedro Rivera TorresAmmar AlmaniSrrah AlgheithyThad GurielAndy DuannHao Chen

CDR Overview

PDR Revision Functional decomposition (hardware)

Hardware subcomponents/implementation

Functional decomposition (software) Software

subcomponents/implementation Current project status Goals (milestone 1, 2, expo) Technicalities (schedule, budget)

Thad Guriel

Technical Specialist

Sam Siewart Google IO interface

Jim Pilkington Assistive Technology Specialist Colorado Department of Human Services Division of Vocational Rehabilitation

Catherine Kilcommons Assistive technology lab coordinator Disability services - University of Colorado

Boulder

Thad Guriel

PDR Revision

AndroidAndroid

KeypadKeypad

CPUCPU

RF Receiver

RF Receiver

IR Receiver

IR Receiver

RF

Comm

IR

Navigation

RF

Comm

IR

Navigation

User Info

Maps Info

Audio Output

User

Thad Guriel

Not Our Project…

Thad Guriel

Our Project…

Thad Guriel

Functional Decomposition Hardware

Hardware

Device

Navigation

IR Receiver

Communication

Transmitter Receiver

Microprocessor Audio

TTS 256 Phone analog

Door

Navigation

IR Transmitter

Communication

TransmitterReceiver

Thad Guriel

Entrance Wake Sequence

Is there a signal?

Query signal sent from

device Xbee

Xbee checks for signal

Xbee establishes

communication with device

Xbee sends interrupt signal to

processor

No

Yes

Andy Duann

After interrupt has been sent to processor

Xbees establish

communication

Processor wakes up

Processor kicks out sign data

Processor turns

on IR

Xbee cyclic sleep

Send data

Talk to device XBee

Turn off IR

Start clock

Processor goes to sleep

Andy Duann

Door Transmitter

Andy Duann

XBee Device

Cyclic sleep

Cycle wake

up

Xbee establishes communicat

ion

Interrupt from

processor

Send signal to

wake door XBee

Transmit data

Was there

a signal

?

YesNo

Andy Duann

Audio Circuit

Andy Duann

Device Audio: No signal

No audio signal

uP outputs logic low to

switches

Turns on Android switch

Turns off synthesizer

switch

Andy Duann

Audio Circuit

Andy Duann

Phone Audio

Phone audio

Comparator outputs logic

high

uP turns off synthesizer interrupts

Outputs logic low to

switches

Comparator goes low

when phone audio gone

Andy Duann

Audio Circuit

Andy Duann

Synthesizer Audio

uP sends interrupt enable to

synthesizer

uP sets logic high to

switches

Turns android off

Turns synthesizer

on

Synthesizer audio

Synthesizer sends interrupt request when data present

uP checks android

interrupt

Is it low?

No Yes

Andy Duann

Audio Circuit

Andy Duann

Overall Device Schematic

Andy Duann

Audio Circuit

Andy Duann

Keypad

Andy Duann

GPS Transition

Locate building entrance after hand-off from GPS

Read signs from a distance and determine its relative direction

Hao Chen

Example

GPS tells user they are in the vicinity of

Engineering entrance, and “Engineering Building; to your

northeast”, for example

Device tells user “Engineering lobby; to

your north” and “elevators; to your

West”.

Hao Chen

Sign Transmitter

3 infrared LED’s are mounted on the entrance, to send directional information at a wide angle

The information of the building is transmitted with XBee

Hao Chen

Sign Receiver

5 IR receivers are mounted on the device

IR receivers are sensitive to 950 nm wavelengths and 38 kHz carrier only

The direction of the entrance can be determined based on which IR receiver(s) received the signal

Hao Chen

Device Receiver Example

In the above picture, receiver 4 picks up the signal while the rest don’t, therefore the microprocessor determines the sign is at northeast direction Hao Chen

Reader

12

3

54

Power Saving

Motivation: battery runs out quickly if IR LED’s are on all the time

Solution: only turn transmitter on when user close

Device uses XBee to trigger transmitter The transmitter turns off after set

amount of time

Hao Chen

Building Information Reading

User approaches entrance,

transmitter turns on

XBee transmits

data to device

Device determines direction of

entrance

Device outputs

information and

direction through speaker

Sign automatically turns off after set

time

Hao Chen

Software Engineering

Srrah Algheithy

Functional Decomposition Software

Level 3

Level 2

Level 1

Level 0 Software

Android

Android app

GPS/location information

Directional information

Android ADK

PIC24FGB110

Keypad USB initialization

Xbee information

Android to ADK USB interface

Srrah Algheithy

Software Components

Srrah Algheithy

Android app to access

GPS and Compass

Android phone 2.3.4 gingerbread

USB interface

Android ADK

Android App

Continuously given

information about

surroundings

• Building names• Street names

Current location

continuously updated

Navigate to points of interest

• Bus stops

Srrah Algheithy

Android App

Srrah Algheithy

GPS App

Android Location Manager

Location Listener

Proximity Alert

Geocoding

Android App Code

requestLocationUpdates() Location provider Time between updates Distance between updates LocationListener()

ItemizedOverlay<Overlay Item> OverlayItem() ▪ Specify and name points of interest

Use built in compass for directionSrrah Algheithy

Data Handling System Overview Communicate effectively with the keypad

system. Coordinate Keypad output with Android

apps. Control internal functionalities of the

Android phone Manage operation and information coming

form Xbee Coordinate Xbee info to turn on IR sensors. Manage the audio system.

Ammar Almani

Development

Using PIC24F j256 GB110 Features:

16-bit: similar to DSP Still low power (3.3V) Programming Memory: 256 KB Full Speed USB operation in Device USB on-The-Go 3 ADCs: 500K conversion / second.

Ammar Almani

Microcontroller Functionality

Main()

Audio

Android/GPS

Sensors

Xbee info

Keypad

Voice Recognition (Android)

Bluetooth Connection

Primary ----Secondary ----

Ammar Almani

Keypad

Coordination Module Using the keypad encoder to generate

interrupts when a button is pressed.Module Keypad_IR()

Inputs -An interrupt signal from chip - 4 binary digits

Outputs - The appropriate command to both Android phone and Xbee

Functionality To coordinate between user and peripherals.

Test Procedure Test to control android app through PIC24F android chip Ammar Almani

Android/GPS

Input/output module

Ammar Almani

Module gps_Info()

Inputs - Destination specified by user through keypad

Outputs - Control audio system to issue guidance commands

Functionality Takes destination as input uses GPS in Android to output guidance voice commands

Test Procedure More than one

Sensors

Composite module

Ammar Almani

Module handle_sensors()

Inputs -Receive signal from GPS to turn on sensors-If analog needed use ADCs

Outputs - Output a specific direction for audio system

Functionality Takes IR information from five sensors and use transmitted data to perceive the sign’s location .

Test Procedure - Once HW ready, PIC24 kit is used to test.

Xbee communication

Input Module

Pedro Rivera

Module receive_xbee()

Inputs -Wake up xbee signal from GPS

Outputs -information in buffer to the audio function

Functionality Takes data from xbee, store it in a buffer, make sure all data is transmitted; and then send it to the audio function.

Test Procedure - Use UART utility to test function in 2 levels

Audio

A system that outputs audio signals from both Android device and text-to-speech chip

Pedro Rivera

Module output_audio()

Inputs -Synthesizer interrupt request-Android Interrupt

Outputs -Logical output to analog audio switches

Functionality Takes both data from xbee and main and an audio interrupts form android; then decides the priority and behave accordingly.

Test Procedure - More than one test

Temporal Diagram

Kyepad_IR

gps_info Audio

Bluetooth

Voice_recognition

Xbee_receive Sensors Extra sensors

Pedro Rivera

Primary ----Secondary ----

PIC USB – Android interface

Requirements for Open Accessory Hardware Making PIC the host▪ Being host entails:

-Initiating communication-Knowing what all devices are doing

Pedro Rivera

PIC USB – Android interface Open accessory Handshake

(When accessory is connected, the accessory will perform these steps): Send “Get Protocol” command Send device info to identify the

accessory to the Android Device Send “Start” Command The Android device should re-

enumerate in accessory mode and launch an app

Pedro Rivera

PIC USB – Android interface USB Accessory Handshake, part 2• Accessory mode if:• Vendor ID is 0x18D1 (Google)• Product ID is 0x2D00 or 0x2D01

• Read configuration descriptors• Look for first bulk IN and first bulk OUT

endpoints• Set configuration to 1• Endpoints are now ready for

communicationPedro Rivera

Software

Pedro Rivera

Current Status

Thad Guriel

Hardware Software

Audio circuit built & testing in progress

Android app for location written (testing in progress)

Keypad circuit built & tested ADC driver working for up to 80KHz

IR built & testing in progress Interrupt is pending

XBee testing in progress UART driver pending

Milestone I

Thad Guriel

Hardware Software

Successfully implements the “wake up” of signs when user is

near

App provides current location, destination, arrival, direction

Successfully generates 38 kHz modulated infrared signal and

receives it at a 5 meter distance

USB driver working

Successfully limits the range of XBEE to 10 meters and able to transmit information from one

XBEE to the other.

Keypad, press a button on keypad and it controls something on the

android

PCB is designed and ordered Xbee receives signal, gets information and saves it (doesn’t

have to be correct)

UART driver working

Milestone II

Thad Guriel

Hardware Software

3 IR LEDs are used to complete the signs

Interfacing between keypad and GPS

All 5 IR receivers are mounted on the device & successfully

determines the direction of the sign

App gives directions to destination

Received XBEE sign information is outputted through speaker

Xbee functioning

PCB revision built Audio under testing

Expo

All chips are mounted on PCB, device is fully functioning.

The project is demoed through video recording.

Thad Guriel

Updated Division of Labor

Thad Guriel

Task Thad Hao Srrah Andy Pedro Ammar

GPS App Primary Secondary Secondary

USB interface Secondary Primary Secondary

Microprocessor

Secondary Secondary Primary

Communication

Primary Secondary Secondary Primary

IR Secondary Primary Secondary Primary

Power Secondary Secondary Primary

Keypad Secondary Secondary Secondary Primary Secondary Primary

Audio Primary Secondary Secondary Primary

PCB Primary Primary Secondary Primary Secondary Secondary

Hardware -----Software -----

Updated Budget

Thad Guriel

Part Cost

PIC test board $60

XBee $50

Speakjet $24.95

TTS 256 $21.95

Audio Amp $3

Keypad encoder

$10

Voltage regulator

$2

Flip flop $2

IR $10

Arduino $30

Headphone jack

$5

Total -$218.90

UROP funding $1000

Remaining budget

$781.1

References

Wall e (slide 5) - http://www.robotbooks.com/robot_toys_II.htm User (slide ) - http://platinumjellybean.tumblr.com/post/1005345702/this-is-a-stick-man Device for visually impaired (slide 6) - http://www.humanware.ca/web/en/newsletter/240.htm Microchip (slide 36) - http://www.edn.com/article/520757-

Microchip_targets_active_current_reduction_in_PIC24F_expansion.php Android logo (slide 33 & 34) –

http://blogs.vertigo.com/personal/ericl/Blog/Lists/Photos/Forms/DispForm.aspx?ID=12 Android logo (slide 32) – http://www.digitaltrends.com/mobile/do-developers-prefer-apples-or-

androids/ Gingerbread Android logo (slide 31) – http://www.vpsems.com/android-gingerbread.html Visually impaired navigation (slide 22) - http://www.teu.ac.jp/hashino/research/index.html Clip art (slide 27, 26, 24 ,23 & 56) – Microsoft PowerPoint clipart Android/USB – http://www.everythingusb.com/android-open-accessory-kit-21147.html

Thad Guriel

Updated Schedule

Andy Duan

Questions

Thad Guriel