1 desiging a virtual information telescope using mobile phones and social participation romit roy...

1

Desiging a Virtual Information Telescopeusing Mobile Phones and Social Participation

Romit Roy ChoudhuryAsst. Prof. (Duke University)

2

Virtual Information Telescope

3

Context

Next generation mobile phones will havelarge number of sensors

Cameras, microphones, accelerometers, GPS, compasses, health monitors, …

4

Context

Each phone may be viewed as a micro lens

Exposing a micro view of the physical worldto the Internet

5

Context

With 3 billion active phones in the world today

(the fastest growing comuting platform …)

Our Vision is …

6

Internet Internet

A Virtual Information Telescope

7

One instantiation of this vision through a system called Micro-Blog

- Content sharing- Content querying- Content floating

8

Content Sharing

Virtual TelescopeVirtual Telescope

Cellular,WiFi

Cellular,WiFi Visualization ServiceVisualization Service

Web ServiceWeb Service

People

Physical SpacePhysical Space

Phones

9

Content Querying

Virtual TelescopeVirtual Telescope

Cellular,WiFi

Cellular,WiFi Visualization ServiceVisualization Service

Web ServiceWeb Service

PhonesPeople

Physical SpacePhysical Space

Some queries participatoryIs beach parking available?

Some queries participatoryIs beach parking available?

Others are notIs there WiFi at the beach café?

Others are notIs there WiFi at the beach café?

10

Content Floating [on physical space]

superb

sushi

Safe@

Nite?

Safe@

Nite?

11

If designed carefully, a variety of applications may emerge on Micro-Blog

12

Applications

Tourism View multimedia blogs … query for specifics

Micro Reporters News service with feeds from individuals

On-the-fly Ride Sharing Ride givers advertize intension w/ space-time sticky notes Respond to sticky notes once you arrive there

Virtual order on physical disorder Land in a new place, and get step by step information

RSS Feeds on Location Inform me when a live band is playing at the mall

13

MiroBlog Prototype

Nokia N95 phones Symbian platform Carbide C++ code

14

Micro-Blog Beta live athttp://synrg.ee.duke.edu/microblog.html

15

Prototype

16

Case Studies

Micro-Blog phones distributed to volunteers

12 volunteers• 4 phones in 3 rounds

• 3 weeks

Not great UI• Basic training for users

Exit interview revealed useful observations

17

From Exit Interview

1. “Fun activity” for free time Needs much “cooler GUI”

2. Privacy control vital, don’t care about incentives “more interesting to reply to questions … interested in

knowing who is asking …”

3. Voice is personal, text is impersonal “Easier to correct text … audio blogs easier but …”

4. Logs show most blogs between 5:00 to 9:00pm Probably better for battery usage as well

18

Thoughts

Micro-Blog:Rich space for applications and services

But where exactly is the research here ???!!**

19

Problem I

Energy Efficient Localization(EnLoc)

20

To GPS or not to GPS

GPS is popular localization scheme Good error characteristics ~ 10m

Apps naturally assume GPS Shockingly, first Micro-Blog demo lasted < 10 hours

21

Cost of Localization

Performed extensive measurements GPS consumes 400 mW, AGPS marginally better Idle power consumption 55 mW

22

Alternate Localization

WiFi fingerprinting, GSM triangulation Place Lab, SkyHook …

Improved energy savings WiFi 20 hours GSM 40 hours

At the cost of accuracy 40m + 200m +

23

40

Tradeoff Summary:

20

Research Question:Can we achieve the best of both worlds

200

24

Given energy budget, E, Trace T, andlocation reading costs, egps , ewifi , egsm :

Schedule location readings to minimize avg. error

Given energy budget, E, Trace T, andlocation reading costs, egps , ewifi , egsm :

Schedule location readings to minimize avg. error

Formulation

L(t0) L(t1) L(t2) L(t3) L(t4)

L(t6)L(t7)

Error

t0 t1 t2 t3 t4 t5 t6

L(t5)

t7

Accuracy gain from GPS

Accuracy gain from WiFi

GPS WiFi

25

Dynamic Program

Minimize the area under the curve By cutting the curve at appropriate points Number of (GPS + WiFi + GSM) cuts must cost < budget

26

Offline optimal offers lower bound on error

Online algorithm necessaryOnline optimal difficult

Need to design heuristics

27

Our Approach

Do not invest energy if you canpredict (even partially)

28

Predictive Heuristics

Prediction opportunities exist Human users are not in brownian motion (exploit inertia) Exploit habitual mobility patterns Population distribution can be leveraged

Prediction also incorporated into Dynamic Program Optimal computed on a given predictor

Error

t0 t1 t2 t3 t4 t5 t6 t6

Predictiongenerates different errorcurve

29

Mobility Profiling

Build logical mobility tree per-user Each link an uncertainty point (UP) Sample location only when uncertain Location predictable between UPs

Exploit acclerometers Predict traffic turns Periodically localize to reset errors

HomeHome

GymGym

Roadcrossing

Roadcrossing

LibraryLibrary

GroceryGrocery

OfficeOffice

8:008:15

8:3012:00

8:0512:05

3:305:30

6:006:00

30

Humans may deviate from mobility profile

Predict based on population statistics

Population Statistics

Goodwin & Green

U-Turn Straight Right Left

E on Green 0 0.881 0.039 0.078

W on Green 0 0 0.596 0.403

N on Goodwin

0 0.640 0.359 0

S on Goodwin

0 0.513 0 0.486

31

Buy Accuracy with Energy

Comparison of optimal with simple interpolation GPS clearly not the right choice

32

Thoughts

Localization cannot be taken for granted Critical tradeoff between energy and accuracy

Substantial room for saving energy While sustaining reasonably good accuracy

However, physical localization May not be the way to go … Several motivations to pursue symbolic localization

33

Questions?

34

Problem 2

Symbolic localization(SurroundSense)

35

Symbolic Localization

Services may not care about physical location Symbolic location often sufficient E.g., coffee shop, movie, park, in-car …

Physical to Symbolic conversion Lookup location name based on GPS coordinate However, risky

RadioShackStarbucks

GPS Errorrange

36

Its possible to localize phones by sensing the ambience

such as sound, light, color, movement, orientation…

Hypothesis

37

Develop multi-modal fingerprint Using ambient sound/light/color/movement etc.

Starbucks

SurroundSense Server

Wall

RadioShack

SurroundSense

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

38

Each individual sensor not discriminating enough Together, they are quite unique

Use Support Vector Machines to identify uniqueness

FingerprintDatabase

ClassificationAlgorithm (SVM)

Location

SurroundSense

39

BBAA

CC DD

EE

GSM provides macro location (mall)GSM provides macro location (mall)SurroundSenseSurroundSense refines to Starbucks refines to Starbucks

Should Ambiences be Unique Worldwide?

40

Economics forces nearby businesses to be different

Not profitable to have 5 chinese restaurantswith same lighting, music, color, layout, etc.

SurroundSense exploits this ambience diversity

Why will it work?

The Intuition:

41

Fingerprints

Sound:

Color:

42

Fingerprints

Light:

Movement:

43

++

Ambience Fingerprinting

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Test Fingerprint

Sound

Compass

Color/Light

RF/Acc.

LogicalLocation

Fingerprint Filtering &Matching

FingerprintDatabase

==

Candidate Fingerprints

Macro Location

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

44

Full System on Nokia N95

Experimented on 58 stores 10 different clusters Different parts of Duke campus

and in Durham city

45

Full System on Nokia N95

Some classifications were incorrect But we wanted to know how much incorrect? We plotted Top-K accuracy

Top-3 accuracy proved to be 100% for all stores

46

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Issues and Opportunity

Cameras may be inside pockets Now, we detect when its taken out Activate cameras, and take pictures Future phones will be flexible (wrist watch) - see Nokia

Morph

Electroic compasses can fingerprint layout Tables and shelves laid out in different orientations Users forced to orient in those ways

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

47

Ambience can be a great clue about locationAmbient Sound, light, color, movement …

None of the individual sensors good enoughCombined they may be unique

Uniqueness facilitated by economic incentive

Businesses benefit if they are mutually diverse in ambience

Ambience diversity helps SurroundSense

Summary

48

Conclusion

The Virtual Information Telescope A generalization of mobile, location

based, social computing

Just developing apps Not enough

Many challenges Energy Localization Privacy Incentives, data distillation …

Internet Internet

49

Conclusion

Project Micro-Blog Addressing the challenges systematically Building a fully functional system with applications

The project snapshot as of today, includes:

Micro-Blog: Overall system and application

EnLoc: Energy Efficient Localization

SurroundSense: Context aware localization

CacheCloak: Location privacy via mobility prediction

Micro-Blog: Overall system and application

EnLoc: Energy Efficient Localization

SurroundSense: Context aware localization

CacheCloak: Location privacy via mobility prediction

50

PhonePoint Pens

Using phone accelerometers To write short messages in the air

51

Please stay tuned for more athttp://synrg.ee.duke.edu

Thank You

52

Several research challenges and opportunities

1. Energy-efficient localization2. Symbolic localization through ambience sensing3. Location privacy

4. Incentives5. Spam6. Information distillation7. User Inerfacing …

Our Research

53

Disclaimer

All of our projects are ongoing, hence not fully mature

Today’s talk more about the problemsthan about solutions

54

Today’s Talk

InformationTelescope

InformationTelescope

VisionVision System andApplicationsSystem andApplications

Challenges/OpporunitiesChallenges/Opporunities Ongoing, Future Work

Ongoing, Future Work

1. EnLoc2. SurroundSense3. CacheCloak