surroundsense : mobile phone localization via ambience fingerprinting
DESCRIPTION
SurroundSense : Mobile Phone Localization via Ambience Fingerprinting. Written by Martin Azizyan , Ionut Constandache , & Romit Choudhury Presented by Craig McIlwee. Motivation. Provide logical localization Using GPS only isn’t good enough Doesn’t work well indoors - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/1.jpg)
SurroundSense: Mobile Phone Localization via Ambience Fingerprinting
Written by Martin Azizyan, Ionut Constandache, & Romit Choudhury
Presented by Craig McIlwee
![Page 2: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/2.jpg)
Motivation
• Provide logical localization• Using GPS only isn’t good enough– Doesn’t work well indoors– Doesn’t account for dividing walls
• Dedicated hardware is not scalable
![Page 3: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/3.jpg)
Approach
• Create an ambience fingerprint using sound, light, color, and user movement– Noise signatures specific to type of location/store– Chain stores have color themes– User movement indicative of store type
![Page 4: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/4.jpg)
Architecture/Algorithm
![Page 5: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/5.jpg)
Architecture/Algorithm
• Data is recorded on the phone, preprocessed, and sent to a server
• Filter module– Subsets the candidates– Wifi, movement, sound
• Match module– Selects the best candidate– Color/sound, Wifi
![Page 6: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/6.jpg)
Architecture/Algorithm
• No single module needs to be perfect– If each module is ‘good enough’ then all modules
combined are sufficient– Being simple reasonably accurate instead of
sophisticated and perfect reduces resources required for processing
![Page 7: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/7.jpg)
Sound Module
• Filter– Sound varies over time
• Fingerprints captured from various times of day• Similarity of fingerprints is used to create a
threshold for a potential match• Match if within the threshold, discard otherwise– Threshold is generous– More false positives is better than false negatives
![Page 8: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/8.jpg)
Motion Module
• Filter– Variations in user behavior
• Record 4 samples/second, use moving average over last 10 samples
• Minor variations suppressed
![Page 9: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/9.jpg)
Motion Module
• User movement is classified as stationary or mobile
• 3 profiles defined– Long stationary – restaurant– Frequent movement with longer stationary –
browsing– Frequent movement with shorter stationary –
shopping• Some logical locations fit multiple profiles
![Page 10: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/10.jpg)
Motion Module
![Page 11: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/11.jpg)
Color/Light Module
• Match• Images captured from camera while facing
downward– Floor themes are consistent– Other orientations introduce noise– Common orientation when checking email, text
messages, etc
![Page 12: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/12.jpg)
Color/Light Module
• Analyze patterns in the image• First attempt was to convert pixels to RGB
values– Failed due to shadow and reflection influences
• Second attempt was to convert to HSL values– Isolates light on its own axis
![Page 13: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/13.jpg)
Color/Light Module
![Page 14: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/14.jpg)
Color/Light Module
• Same/similar colors result in clusters when graphed
• Dominant colors generate larger clusters• Similarity calculated as distance between
cluster centroids and size of the clusters• Most similar candidate is the match
![Page 15: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/15.jpg)
Wifi Module
• Normally a filter, match if camera is not available
• Capture MAC address of available access points every 5 seconds
• Compare occurrence ratio of currently available access points to known access points
![Page 16: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/16.jpg)
Known Issues
• Sound varies over time– Split day into 2 hour windows, capture fingerprints during
each window– No mention of day of week, time of year
• Camera in pocket– All testing done with phone in hand– Expected rise in wearable devices
• Mimicking user behavior– Initial data showed artificial behavior– Subsequent attempts shadowed real customers
![Page 17: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/17.jpg)
Known Issues
• Resource (energy) intensive• Accelerometer fingerprint takes time to
capture• Non-business locations may not exhibit
enough diversity– Offices, airports, libraries
![Page 18: SurroundSense : Mobile Phone Localization via Ambience Fingerprinting](https://reader036.vdocuments.site/reader036/viewer/2022062501/568167d4550346895ddd2a02/html5/thumbnails/18.jpg)
Evaluation
• Recorded fingerprints of 51 locations– “War-sensed” by students– 2 different groups during different times of day
• Group A’s fingerprints used as database while Group B was at the location collecting their own fingerprints
• Accuracy analysis was done on various combinations of sensors types
• All sensor types combined yielded 87% accuracy