Context awareness & Contextual Computing

Occupancy Monitoring of Mary Idema Pew Library

Advisor Professor EngelsmaSam Serpoosh

https://github.com/Sam-Serpooshhttps://linkedin.com/in/samserpoosh

http://masihjesus.wordpress.com@masihjesus

Device sensors more information

We collect big data• What do we do with it?

• Recommendation Engines

• More relevant and useful content for each individual

• Better user experience and more value

What is missing?

Human-human VShuman-computer

Current Context & Situational Information

Context can be different

Hang out suggestion

WHERE???

PREVIOUS ACTS???

WHO???

What you’re looking at

Context depends on domain

• Based on the application• Problem it is solving• The Domain“Any information that can be used in order

to describe the situation of an entity relevant to interaction between user and the application is considered Context.”

What can be improved on campus?

• Looking for a book?

• Campus Tour Guide?

Campus tour app kills social Interaction

Meeting with library staff

• new books notification

while they’re passing by different sections

• Traffic and

Occupancy Monitoring In certain areas

Occupancy monitoring wins

• Library staff are more interested in occupancy monitoring

• That’s an actual problem

• Students can get help on where is less crowded to study/work

How to solve it• Install Beacon devices in 3rd Floor Innovation

Zone & 4th Floor Study Room

• Add a traffic/occupancy monitoring feature to a popular application which is used by majority of students

• Collect and process traffic data and get real-time information for those areas

Overall architecture

Client application• Laker Mobile• Used by Majority of Students• Has a Library Section

• Need to make it more useful• Add a context-aware background service for monitoring traffic activities (e.g. Enter, Exit)

ibeacon• Wireless devices transmitting radio signals to

smart devices

• Devices can hear these signals (via BLE scanning)

• Communicate with beacons

• Estimate their proximity from them

Ibeacon (cont.)• Introduced by Apple in iOS 7

• Core Location Library

• Works with Bluetooth Low Energy (BLE)

• Proximity Calculation and Context Awareness

Main functionalities• Region Monitoring

• Can detect Enter/Exit for a monitored region/area

• Region Change detection

• Beacon Ranging• Can calculate the proximity to a certain beacon

device in a given region• Proximity levels Far, Near, Immediate,

Unknown

• Identifiable through UUID, Major and Minor numbers

We used estimote beacons

Ruby on rails web server

• Receiving traffic data• Returns traffic information in a given period in

conventional format: JSON, CSV• Gives real-time information on population of

monitored areas• Shows a heat map associated with monitored

areas

• All the other BORING parts of a web service

Sample dataSubmitted data from a client device looks like

this

Sample data (cont.)Areas population

Heat map

Overall workflowPretty Straightforward and Simple

Quick peak on Implementation

• CLBeaconManager From Core Location Library• When a device enters a monitored area• Certain callback gets invoked by beacon

manager• (void)beaconManager:(ESTBeaconManager*)didEnterRegion:

(ESTBeaconRegion)

• Context-Aware logic of the application is here or triggered here

Did enter region

Region monitoring &beacon ranging

• Region Monitoring can happen both in background and foreground

• Beacon Ranging can only happen when the application is in foreground• Proximity calculation• Signal Strength calculation• Solving the problem of overlapping areas

Frequency of ble scan

• Ranging happens quite frequently when application is in foreground • Almost every few seconds

• Region Monitoring depends on few thingsCondition Max time to detect Range

ChangenotifyEntryStateOnDisplay =

YES & Phone Awakened1 second

notifyEntryStateOnDisplay = NO

& Phone Awakened

NEVER

Application is just running in background

From few seconds up to 15 minutes

Subtleties on server• Device is On border of a region

• Two subsequent traffic data can be submitted with EXACT same time

• Which one do you consider in Area Population calculation?

What we can find out?

• How long a users occupy a given area• Enter at 9:00 AM• Exit at 13 PM

• Taking a break or temporarily leaving the area• Exit at 18:35• Enter at 18:44• Taking a break, Finding a book, meeting

someone, etc.

• Just walking by a monitored area closely• Enter at 17:35:07• Exit at 17:35:40

Hourly population report

Closer look

Devices per day

Lessons learnedApple users upgrade FAST

Lessons learned (CONT.)

• Dealing with new and unfamiliar technology• Technical Side = Advertisement and Promoting

• ??A lot of unknowns, unfamiliar obstacles??

• Building a infrastructure, platform, framework• Core logic is interchangeable

Lessons learned (cont.)

• Dealing with other organizations

• IT was worried about WiFi interference

• Bluetooth & WiFi 2.4 GHz frequency

• Students will face Internet connection problems

Wifi disconnects• Could not connect to WiFi through my

laptop???!!!• Everything else was connected (iPhone, iPad,

etc.)

• The conversation with IT hit me• we have an Estimote beacon in our lab• Stackoverflow to the rescue

Context matters• Human-Human interaction is very effective at

this• We want the human-computer interaction to

be efficient and valuable• recommending appropriate things, showing

relevant content• Why not better? We know one more

missing piece of the puzzle• Context is a Caved in Stone

Special thanks to• Professor Jonathan Engelsma• Carlos Rodriguez• Kyle Felker• Kristin Meyer• Ehsan Valizadeh• Davoud Qorashi• All the students who used the

application and collaborated in this work

References• Dey AK, Abowd GD. Towards a better understanding of

context and context-awareness. CHI’2000 Workshop on the What, Who, Where, When, and How of Context-Awareness.

• Yin, Chuantao, Bertrand David, and Rene Chalon. "Use your mobile computing devices to learn-Contextual mobile learning system design and case studies."Computer Science and Information Technology, 2009. ICCSIT 2009. 2nd IEEE International Conference on. IEEE, 2009.

• Prophet Kitchen application for Rubens House in Antwerp Belgium. http://vimeo.com/84760383

• Estimote official website. http://estimote.com/ • API documentation of Estimote.

http://estimote.com/api/index.html

References (cont.)• Estimote iOS SDK.

https://github.com/Estimote/iOS-SDK • Apple documentation on CLBeaconManager.

https://developer.apple.com/library/ios/documentation/CoreLocation/Reference/CLLocationManagerDelegate_Protocol/CLLocationManagerDelegate/CLLocationManagerDelegate.html - //apple_ref/occ/intfm/CLLocationManagerDelegate/locationManager:didEnterRegion:

References (cont.)• Frequency of region monitoring and beacon

ranging. http://developer.radiusnetworks.com/2013/11/13/ibeacon-monitoring-in-the-background-and-foreground.html

• Estimote on coexistence of WiFi and Beacon devices. https://community.estimote.com/hc/en-us/articles/201186046-Do-Wi-Fi-Networks-Interfere-with-Beacons-