1 “did you see bob?”: human localization using mobile phones ionut constandache co-authors: xuan...
TRANSCRIPT
1
“Did you see Bob?”: Human Localization using Mobile Phones
Ionut Constandache
Co-authors: Xuan Bao, Martin Azizyan, and Romit Roy Choudhury
Modified by Chulhong
2
Localization Technologies
OutdoorDriving directions GPS, Skyhook
IndoorLocalization in office Cricket, Radar, BAT
Energy-Efficient Continuous localization EnLoc, RAPS
Logical Context-aware ads SurroundSense
3
Localization Technologies
OutdoorDriving directions GPS, Skyhook
IndoorLocalization in office Cricket, Radar, BAT
Energy-Efficient Continuous localization EnLoc, RAPS
Logical Context-aware ads SurroundSense
Human Localization:Guiding a user to finding another person
4
Usage Scenario
Bob
Alice
5
Usage Scenario
Where is Bob? Please escort me to Bob.
Bob
Alice
6
Usage Scenario
Where is Bob?
Bob
Alice
7
Usage Scenario
Where is Bob?
Bob
Alice
General approach today:1. Stroll around in the hotel until Alice can visually spot Bob2. Ask “Have you seen Bob around?”3. Phone call
However, what if Alice does not know Bob?
8
Usage Scenario
Where is Bob? Please escort me to Bob.
Bob
Provide an electronic Escort system.
Alice
9
Usage Scenario
Alice’s Phone
20 steps North5 steps East
N
Bob
10
Usage Scenario
Alice’s Phone
11
Usage Scenario
Alice’s Phone
Bob
12
Human Localization
Finding Bob in unfamiliar place(E.g. library, mall, engineering building)
13
Human Localization
Finding Bob in unfamiliar place(E.g. library, mall, engineering building)
14
Human Localization
Finding Bob in unfamiliar place(E.g. library, mall, engineering building)
Better for Alice to be escorted to Bob
15
Human Localization
Finding Bob in unfamiliar place(E.g. library, mall, engineering building)
Better for Alice to be escorted to Bob
Challenges:
Bob’s location unknown
16
Human Localization
Finding Bob in unfamiliar place(E.g. library, mall, engineering building)
Better for Alice to be escorted to Bob
Challenges:
Bob’s location unknown
Even if known still require …
WALK-able routes to Bob
17
Human Localization
Finding Bob in unfamiliar place(E.g. library, mall, engineering building)
Better for Alice to be escorted to Bob
Challenges:
Bob’s location unknown
Even if known still require …
WALK-able routes to Bob Once in his vicinity, identify Bob
Can current localization schemes help?
too heavy on requirements …
Infrastructure: specialized hardware (e.g. Cricket, BAT, etc.)
or War-driving: build fingerprint DB (e.g. Radar, Skyhook,
etc.)
Can current localization schemes help?
too heavy on requirements …
Infrastructure: specialized hardware (e.g. Cricket, BAT, etc.)
or War-driving: build fingerprint DB (e.g. Radar, Skyhook,
etc.)
… need lightweight localization solution
Can current localization schemes help?
21
Contents
Escort
Evaluation
Limitations and Future Work
Conclusion
22
Contents
Escort
Evaluation
Limitations and Future Work
Conclusion
23
Our Solution
Accelerometers/compasses track human movements Standard sensors in mobile phones Each user has a trailtrai
l
24
Our Solution
Accelerometers/compasses track human movements Standard sensors in mobile phones Each user has a trailtrai
l
stepi , directioni > = TRAIL <
25
Our Solution
Accelerometers/compasses track human movements Standard sensors in mobile phones Each user has a trailtrai
l
26
Our Solution
Deploy coordinate system to localize users Any (fixed) location can be the origin N, E directions are the Y, X axises
N
E
Origin
27
Our Solution
Users join the coordinate system When passing the origin At encounters with users already in the system
(0,0)
N
E
Origin
28
Our Solution
Users join the coordinate system When passing the origin At encounters with users already in the system
N
E
Origin
(x,y)
29
Our Solution
Users join the coordinate system When passing the origin At encounters with users already in the system
N
E
Origin
(x,y)
(x,y)
How does Escorting work?
How does Escorting work?
C
A
B
D
C
A
B
D
Escort ServiceCloud
A’s Trail
C
A
B
D
Escort ServiceCloud
A’s Trail
C
A
B
D
Escort ServiceCloud
C
A
B
D
Escort ServiceCloud
C
A
B
DIBD
IBC
IAC
Escort ServiceCloud
C
A
B
DIBD
IBC
IAC
Escort ServiceCloud
C
A
B
DIBD
IBC
IAC
IAC
IBC
IBD
ACB
D
Trail Graph
Escort ServiceCloud
39
Trail Graph
IAC
IBC
CB
D
IBD
A
40
Escort along the Trail Graph
IAC
C
D
IBD
B
IBC
Alice
Bob
A
41
Escort along the Trail Graph
IAC
C
D
IBD
B
IBC
Alice
Bob
A
42
Escort along the Trail Graph
IAC
C
D
IBD
B
IBC
Bob
A Alice
43
Escort along the Trail Graph
IAC
C
D
IBD
B
IBC
Bob
A
Alice
WALK-able routes
Alice guided along user trails:Trails need to be accurate
44
Challenges
Trails drift: acc. missed steps, compass biases
t1
t2
45
Challenges
Trails drift: acc. missed steps, compass biases
t1
t2
46
Challenges
Trails drift: acc. missed steps, compass biases
t1
t2
47
Challenges
Trails drift: acc. missed steps, compass biases
t1
t2
48
Challenges
Trails drift: acc. missed steps, compass biases
t1Compass bias
t2
θ
49
Challenges
Trails drift: acc. missed steps, compass biases
t1
t2
50
Challenges
Trails drift: acc. missed steps, compass biases
t1
t2
51
Challenges
Trails drift: acc. missed steps, compass biases
t1
t2
52
Challenges
Trails drift: acc. missed steps, compass biases
t1
t2
53
Challenges
Trails drift: acc. missed steps, compass biases
t1
ActualDrifted
t2
54
Challenges
Trails drift: acc. missed steps, compass biases
t1
ActualDrifted
t2
55
Challenges
Trails drift: acc. missed steps, compass biases
t1
ActualDrifted
t2
Error
56
Challenges
Trails drift: acc. missed steps, compass biases
t1
ActualDrifted
t2
Error
Need to correct:• User Location• User Trail
57
Correct User Location
Opportunities When passing the origin, the user is at (0,0) Close-encounters with users who passed the origin
recently Take this user’s position (it’s accurate)N
E
Origin
58
Correct User Location
Opportunities When passing the origin, the user is at (0,0) Close-encounters with users who passed the origin
recently Take this user’s position (it’s accurate)N
E
Origin
59
Correct User Location
Opportunities When passing the origin, the user is at (0,0) Close-encounters with users who passed the origin
recently Take this user’s position (it’s accurate)N
E
Origin
60
Correct User Location
Opportunities When passing the origin, the user is at (0,0) Close-encounters with users who passed the origin
recently Take this user’s position (it’s accurate)N
E
Origin
Encounter with origin (0,0)
61
Correct User Location
Opportunities When passing the origin, the user is at (0,0) Close-encounters with users who passed the origin
recently Take this user’s position (it’s accurate)N
E
Origin
Encounter with origin (0,0)
62
Correct User Location
Opportunities When passing the origin, the user is at (0,0) Close-encounters with users who passed the origin
recently Take this user’s position (it’s accurate)N
E
Origin
Encounter with origin (0,0)
63
Correct User Location
Opportunities When passing the origin, the user is at (0,0) Close-encounters with users who passed the origin
recently Take this user’s position (it’s accurate)N
E
Origin
Encounter withphone with good location estimate
(x,y)
64
Correct User Location
Opportunities When passing the origin, the user is at (0,0) Close-encounters with users who passed the origin
recently Take this user’s position (it’s accurate)N
E
Origin
Encounter withphone with good location estimate
(x,y)
65
Correct User Location
Opportunities When passing the origin, the user is at (0,0) Close-encounters with users who passed the origin
recently Take this user’s position (it’s accurate)N
E
Origin
Encounter withphone with good location estimate
(x,y)
How to detect encounters with origin/users?
66
Detecting Encounters using Sound
Phones periodically beacon their presence Beacons = unique audio tones Phones also listen for neighboring beacons
67
Detecting Encounters using Sound
Phones periodically beacon their presence Beacons = unique audio tones Phones also listen for neighboring beacons
Tone amplitude above threshold encounter
68
Correct User Trail
ActualDrifted
t1
t2
Error
69
Origin
Correct User Trail
ActualDrifted
t1
t2
Error
70
Origint1
V
Correct User Trail
ActualDrifted
t2
71
Origint1
Correct User Trail
L(t)
L’(t)
V12
1)()('tt
ttVtLtL
t2
ActualDrifted
72
Origint1
Correct User Trail
ActualDriftedCorrected
L(t)
L’(t)
V12
1)()('tt
ttVtLtL
t2
73
Escort
After drift correction Escort users along the corrected trails Place user in vicinity of the searched-for person
74
Escort
After drift correction Escort users along the corrected trails Place user in vicinity of the searched-for person
IBCIBD
A
CB
DIAC
75
Escort
After drift correction Escort users along the corrected trails Place user in vicinity of the searched-for person
EF
IBCIBD
A
C
DIAC
B
76
Escort
After drift correction Escort users along the corrected trails Place user in vicinity of the searched-for person
EF
IBCIBD
A
C
DIAC
How to visually identify Bob?
B
77
Solve human localization end-to-end Create visual fingerprint for each user
Alice’s Phone
Bob
Visual Identification
78
Visual Fingerprint
User picture
79
Visual Fingerprint
Upper Region
Lower Region
Fingerprint
User picture
Recognizing Bob
Recognizing Bob
Users advertise <name, visual fingerprint>
< Bob , >
< Tom , >
< Dan , >
Recognizing Bob< Bob , >
< Tom , >
< Dan , >
Alice’s Phone
< Bob , >< Tom , >< Dan , >
Recognizing Bob< Bob , >
< Tom , >
< Dan , >
Alice’s Phone
< Bob , >
Recognizing Bob< Bob , >
< Tom , >
< Dan , >
Alice’s Phone
< Bob , >
Recognizing Bob< Bob , >
< Tom , >
< Dan , >
Alice’s Phone
Bob
86
Contents
Escort
Evaluation
Limitations and Future Work
Conclusion
87
Evaluation
Escort target accuracy: several meters Require high ground-truth accuracy ~ 1m GPS not accurate enough
Our approach Run experiments in a testbed with dense markers Markers have known position
88
A
C
D
36 m
48 m
Testbed
Markers
Origin
B
89
A
C
D
36 m
48 m
Testbed
Markers
Origin
User Paths
B
90
A
C
D
36 m
48 m
Testbed
Markers
Origin
User Paths
B
4 Test Users
13 minutes experiment
User locations known at markers
40 escorting tests
91
Average Localization Error
Average Localization Error across all users: ~ 6m
92
Final Distance from Destination
Average Error at end of escorting: ~ 8m
93
Visual Identification Accuracy
94
Contents
Escort
Evaluation
Limitations and Future Work
Conclusion
95
Limitations and Future Work
Employees only access Trails may have restrictions
Phone placement Assumed in hand, investigate placement as future work
Imprecise navigation Humans can make educated guesses
Testing under heavy user load
96
Conclusions
We asked ourselves: Can mobile phones help in “routing” a person A to a person
B
Challenging because: Require walkable routes Needs to be free of infrastructure, war-driving
Possible because: Rich sensing capabilities on mobile phones High density of such devices
97
Conclusions
Our approach: “Stitching” human walking traces to compose a graph. Route humans on this walkable graph
Solution is analogous to routing in DTNs … Only packets are now humans
Alice Bob
98
Questions?
Thank You!
Visit the SyNRG research group @http://synrg.ee.duke.edu/