sensor-based positioning system that exploits spatio-temporal regularity … · 2019-12-18 · woot...
TRANSCRIPT
![Page 1: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/1.jpg)
1 WOOT ’15 Networked Systems Laboratory
Sensor-based Positioning System that Exploits Spatio-Temporal Regularity of Human Mobility
Takuya Watanabe†, Mitsuaki Akiyama‡ and Tatsuya Mori† † Waseda University ‡ NTT Secure Platform Labs
![Page 2: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/2.jpg)
2 WOOT ’15 Networked Systems Laboratory
Your location A stalker A boss
Unsolicited marketing
![Page 3: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/3.jpg)
3 WOOT ’15 Networked Systems Laboratory
Android devices
Camera
Contact list
SMS
GPS
Protected Not protected
Magnetometer
Gyroscope
Any sensors
Accelerometer
The privacy protection mechanism
![Page 4: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/4.jpg)
4 WOOT ’15 Networked Systems Laboratory
data processing
The ultimate goal
Identify absolute position
It is difficult
Accelerometer Magnetometer Gyroscope
Sensor data
![Page 5: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/5.jpg)
5 WOOT ’15 Networked Systems Laboratory
Then, we focus…
![Page 6: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/6.jpg)
6 WOOT ’15 Networked Systems Laboratory
• Static route • Regularly • Associated with
– place of residence
– workplace
– favorite bar
Features of train trip
departure
station
station
arrival station
arrival
![Page 7: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/7.jpg)
7 WOOT ’15 Networked Systems Laboratory
Our goal in this work
Detect the route of a train trip
Route Detector
Accelerometer Magnetometer Gyroscope
Sensor data
![Page 8: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/8.jpg)
8 WOOT ’15 Networked Systems Laboratory
• A malicious software on a smart device – Only internet permission is required
• This software secretly keeps collecting sensor values – It estimates the owners activity (walk, on a vehicle and still)
• An adversary knows the list of public transportation systems that are used by the victim.
Threat model
![Page 9: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/9.jpg)
9 WOOT ’15 Networked Systems Laboratory
Overview of RouteDetector
Detection of User Activities
Detection of Departure/Arrival Time Sequences
Extracting Candidate Routes
1 2 3
![Page 10: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/10.jpg)
10 WOOT ’15 Networked Systems Laboratory
13:00, 13:05, 13:09
Tokyo ↓
Osaka
Smart device
Sensor data
User activity
Time sequence
Candidate
A malicious app
Machine learning
Real time information Timetable
![Page 11: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/11.jpg)
11 WOOT ’15 Networked Systems Laboratory
Detection of User Activities
Machine Learning ( Random Forest )
Accelerometer Magnetometer Gyroscope
Sensor data User activity
Class 0 Still
Class 2 Walk
Class 1 On a vehicle
![Page 12: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/12.jpg)
12 WOOT ’15 Networked Systems Laboratory
cf: Android API to recognize activity
RouteDetector User activities detection
Android official API ActivityRecognitionApi
Permission required No permission required
![Page 13: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/13.jpg)
13 WOOT ’15 Networked Systems Laboratory
Summary of sensors
Unit Type Permission Description
Accelerometer 𝑚/𝑠↑2 Physical None Acceleration applied to a device
Linear accelerometer 𝑚/𝑠↑2 Virtual None Acceleration applied to a device
excluding the gravity
Magnetometer 𝜇𝑇 Physical None Strength of geomagnetic field
Gyroscope 𝑟𝑎𝑑/𝑠 Physcal None A device’s rate of rotation
10 Hz : read 10 values per second
![Page 14: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/14.jpg)
14 WOOT ’15 Networked Systems Laboratory
1. Compute norm =√𝑎↓𝑥↑2 + 𝑎↓𝑦↑2 + 𝑎↓𝑧↑2
– To eliminate the effect of differences in the directions
Data preprocessing for each sensor
![Page 15: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/15.jpg)
15 WOOT ’15 Networked Systems Laboratory
2. Divide time series data into a set of blocks
Data preprocessing for each sensor
block1 block2
𝑎↓1 , 𝑎↓2 , 𝑎↓3 ,…, 𝑎↓𝑛
{𝑎↓1 , 𝑎↓2 , …, 𝑎↓20 }, { 𝑎↓21 , 𝑎↓22 , …, 𝑎↓40 },...
![Page 16: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/16.jpg)
16 WOOT ’15 Networked Systems Laboratory
3. Calculate typical values: mean, max, min and σ
Data preprocessing for each sensor
Block1 = {𝑎↓1 , 𝑎↓2 , …, 𝑎↓20 }
m𝑒𝑎𝑛( 𝑎↓1 , 𝑎↓2 , …, 𝑎↓20 ), max( 𝑎↓1 , 𝑎↓2 , …, 𝑎↓20 ), m𝑖𝑛( 𝑎↓1 , 𝑎↓2 , …, 𝑎↓20 ),
σ( 𝑎↓1 , 𝑎↓2 , …, 𝑎↓20 )
![Page 17: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/17.jpg)
17 WOOT ’15 Networked Systems Laboratory
Feature vectors to detect activity
Accelerometer Linear acceleration Magnetometer Gyroscope
mean max min σ
* = 16 dimensions
=> A random forest classifier (supervised ML)
![Page 18: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/18.jpg)
18 WOOT ’15 Networked Systems Laboratory
Labels creation
HTC / Smartphone
Nexus 7 / Tablet
![Page 19: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/19.jpg)
19 WOOT ’15 Networked Systems Laboratory
Labels stats (# of blocks)
Data vehicle walk still total total time
HTC_hold 1,327 510 609 2,446 4,892 sec
HTC_bag 1,360 510 691 2,561 5,122 sec
Nexus_hold 1,352 505 686 2,543 5,086 sec
Nexus_bag 1,304 505 602 2,411 4,822 sec
HTC/Nexus … devices name hold/bag … situation
![Page 20: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/20.jpg)
20 WOOT ’15 Networked Systems Laboratory
Definition of FP/FN
Class 0 or 2 Still or Walk
Class 1 Vehicle
Class 1 Vehicle
Prediction Ground truth
False Positive
False Negative Class 0 or 2 Still or Walk
![Page 21: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/21.jpg)
21 WOOT ’15 Networked Systems Laboratory
Evaluation of Activities Detection
Data ACC FNR FPR
HTC_hold 0.941 0.042 0.078
HTC_bag 0.965 0.024 0.047
Nexus_hold 0.943 0.041 0.074
Nexus_bag 0.969 0.023 0.041
Performance of detecting vehicle activity
![Page 22: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/22.jpg)
22 WOOT ’15 Networked Systems Laboratory
Example of the classification
Ground truth: 22211111111111111111111000000000011111111111111111 Prediction: 22211001111111011111111110010000011111011111111111
0: still , 1: On a moving vehicle , 2:Walk
![Page 23: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/23.jpg)
23 WOOT ’15 Networked Systems Laboratory
Example of the classification
Ground truth: 22211111111111111111111000000000011111111111111111
0: still , 1: On a moving vehicle , 2:Walk
On the still train
On the moving train
![Page 24: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/24.jpg)
24 WOOT ’15 Networked Systems Laboratory
Ground truth: 22211111111111111111111000000000011111111111111111 Prediction: 22211001111111011111111110010000011111011111111111
0: still , 1: On a moving vehicle , 2:Walk
Detection of Departure/Arrival Time Sequences
Departure Departure Arrival
Some noises
![Page 25: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/25.jpg)
25 WOOT ’15 Networked Systems Laboratory
Noise reduction
EWMA
EWMA: Exponentially Weighted Moving Average 𝑆↓𝑛 =λ𝑆+(1−λ)𝑆↓𝑛−1
Predicted user activities Smoothened user activities
1
0
1
0
![Page 26: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/26.jpg)
26 WOOT ’15 Networked Systems Laboratory
Smoothened user activities
0.5 > S → 1 ( on vehicle ) 0.5 ≦ S → 0 ( still )
Corrected user activities
Departure Arrival
extract these times
11:18 11:15 11:21
11:21 11:18 11:24
Noise reduction (cont.)
![Page 27: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/27.jpg)
27 WOOT ’15 Networked Systems Laboratory
Accuracy of detected DPT/ARR times
Ground truth
Estimated times
DPT. 11:15:40
DPT. 11:15:46
Error Estimation error
6 seconds
![Page 28: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/28.jpg)
28 WOOT ’15 Networked Systems Laboratory
Data Departure Arrival
min max mean min max mean
HTC_hold 1.97 3.54 2.79 2.52 6.75 4.13
HTC_bag 2.04 3.06 2.53 1.71 4.63 3.21
Nexus_hold 2.33 7.94 4.60 3.07 10.78 6.03
Nexus_bag 1.55 2.76 2.17 2.22 5.16 3.43
Stats of absolute estimation errors (secs)
![Page 29: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/29.jpg)
29 WOOT ’15 Networked Systems Laboratory
Candidate Routes Extraction
Seq. of (DPT, ARR)
(11:30, 11:32)
(11:32, 11:35)
(11:35, 11:38) Timetables
● Tokyo →Kanda →Akihabara →Okachimachi
Candidate 1
● Shinjuku →Shin-Okubo →Takadanobaba →Mejiro
Candidate 2
![Page 30: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/30.jpg)
30 WOOT ’15 Networked Systems Laboratory
Timetables
• Collected timetables of passenger train companies operating in Japan
• The DB covers all the prefectures in Japan
– 9,090 railway stations
– 597 lines
– 172 railway companies
– 2,277,397 “links” http://ekikara.jp/
![Page 31: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/31.jpg)
31 WOOT ’15 Networked Systems Laboratory
Definition of a link
Link =
• departure station • arrival station • departure time • arrival time • next link
![Page 32: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/32.jpg)
32 WOOT ’15 Networked Systems Laboratory
Station A Station B Station C
11:40 11:42 11:42 11:45
link α link β
Sd = Station A Sa = Station B Td = 11:40 Ta = 11:42 Next = link β
link α = link β =
Sd = Station B Sa = Station C Td = 11:42 Ta = 11:45 Next = None
![Page 33: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/33.jpg)
33 WOOT ’15 Networked Systems Laboratory
Searching Candidate Routes Input: [ (07:20, 07:22), (07:22, 07:25), (07:25, 07:28) ]
link 1: (07:20, 07:22, A, B) link 2: (07:20, 07:22, C, D) link 3: (07:20, 07:22, E, F)
link 4: (07:22, 07:25, B, G) link 5: (07:22, 07:25, D, H) link 6: (07:22, 07:26, F, L)
link 7: Matched ! (07:25, 07:28, G, J) link 8: (07:25, 07:27, H, K)
Next? Next?
×
×
Results: St. A link 1
St. B St. G St. J link 4 link 7
![Page 34: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/34.jpg)
34 WOOT ’15 Networked Systems Laboratory
Score calculation
Station A Station B
Station C Station D
Score: +5 (more popular)
Score: +1
Count # of Links Same origin / destination Different Td / Ta
![Page 35: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/35.jpg)
35 WOOT ’15 Networked Systems Laboratory
• Enumerate all the possible routes
→ extracted 6.4 billion routes – Sequence length = less than 15 links
– At most 2 line changes
• Compute the relationship between the number of links and the number of candidate routes.
Performance of the algorithm (1)
![Page 36: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/36.jpg)
36 WOOT ’15 Networked Systems Laboratory
a 6-links sequence yields almost 1 candidate route on average
![Page 37: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/37.jpg)
37 WOOT ’15 Networked Systems Laboratory
• More than 6.4 billion routes were searched within 74 𝑚𝑖𝑛𝑠
• A route was searched
within 74𝑚𝑖𝑛𝑠/6400000000 = 7.1𝜇𝑠𝑒𝑐 on average
Performance of the algorithm (2)
![Page 38: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/38.jpg)
38 WOOT ’15 Networked Systems Laboratory
Case study
A victim Sensor data
Route Detector
![Page 39: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/39.jpg)
39 WOOT ’15 Networked Systems Laboratory
S
W
V
S: Still W: Walk V: Vehicle
Case study 1 Results of activity detection
![Page 40: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/40.jpg)
40 WOOT ’15 Networked Systems Laboratory
Case study 1 Results of activity detection
S
W
V
S: Still W: Walk V: Vehicle
1st riding
![Page 41: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/41.jpg)
41 WOOT ’15 Networked Systems Laboratory
Case study 1 Results of activity detection
S
W
V
S: Still W: Walk V: Vehicle
Transfer
![Page 42: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/42.jpg)
42 WOOT ’15 Networked Systems Laboratory
Case study 1 Results of activity detection
S
W
V
S: Still W: Walk V: Vehicle
2nd riding
![Page 43: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/43.jpg)
43 WOOT ’15 Networked Systems Laboratory
Case study 1 Results of Dpt./Arr. time detection
(10:56, 10:58) → (10:58, 11:00) → (11:00, 11:03) → (change the line) → (11:10, 11:12) → (11:12, 11:14)
Candidate Routes
![Page 44: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/44.jpg)
44 WOOT ’15 Networked Systems Laboratory
Case study 1 Results of Candidate Routes Detection
Kokkai-gijido-mae
Kasumigaseki Ginza Tokyo Kanda
Akihabara
Edoga waba
shi
Goko kuji
Higashi ikebu kuro
Ikebu kuro
Kana mecho
Senka wa
transfer
transfer
Candidate 1: Score 2664
Candidate 2: Score 2277
![Page 45: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/45.jpg)
45 WOOT ’15 Networked Systems Laboratory
Case study 1 Results of Candidate Routes Detection
Kokkai-gijido-mae
Kasumigaseki Ginza Tokyo Kanda
Akihabara
Edogawabashi
Gokokuji
Higashi ikebuku
ro
Ikebukuro
Kanamecho
Senkawa
transfer
transfer
Candidate 1: Score 2664
Candidate 2: Score 2277
Correct!
![Page 46: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/46.jpg)
46 WOOT ’15 Networked Systems Laboratory
Case study 2
S
W
V
S: Still W: Walk V: Vehicle (0:10, 0:11) → (0:11, 0:13) → (0:13, 0:15) → (0:15, 0:17)
→ (0:17, 0:19) → (0:19, 0:21) → (0:21, 0:22)
![Page 47: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/47.jpg)
47 WOOT ’15 Networked Systems Laboratory
Case study 2 Results of Candidate Routes Detection
Tabata Nishi-nippori Nippori Ueno
Okachimachi
Candidate 1 ( The only ): Score 3892
Uguisudani
Akiha bara
Kanda
![Page 48: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/48.jpg)
48 WOOT ’15 Networked Systems Laboratory
Case study 2 Results of Candidate Routes Detection
Tabata Nishi-nippori Nippori Ueno
Okachimachi
Candidate 1 ( The only ): Score 3892
Uguisudani
Akiha bara
Kanda
Correct!
![Page 49: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/49.jpg)
49 WOOT ’15 Networked Systems Laboratory
Case study 3
S
W
V
S: Still W: Walk V: Vehicle
(21:27, 21:29) → (21:30, 21:32) → (21:33, 21:35)
→ (21:35, 21:37) → (21:37, 21:39)
![Page 50: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/50.jpg)
50 WOOT ’15 Networked Systems Laboratory
Case study 3 Results of Candidate Routes Detection
× None
![Page 51: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/51.jpg)
51 WOOT ’15 Networked Systems Laboratory
Why we failed to detect in Case 3 ?
DPT ARR
DPT ARR
DPT ARR
DPT ARR
DPT ARR
Scheduled 21:26 21:28 21:28 21:32 21:32 21:35 21:35 21:37 21:37 21:39
Observed 21:27 21:29 21:30 21:32 21:33 21:35 21:35 21:37 21:37 21:39
Detected 21:27 21:29 21:30 21:32 21:33 21:35 21:35 21:37 21:37 21:39
![Page 52: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/52.jpg)
52 WOOT ’15 Networked Systems Laboratory
Why we failed to detect in Case 3 ?
DPT ARR
DPT ARR
DPT ARR
DPT ARR
DPT ARR
Scheduled 21:26 21:28 21:28 21:32 21:32 21:35 21:35 21:37 21:37 21:39
Observed 21:27 21:29 21:30 21:32 21:33 21:35 21:35 21:37 21:37 21:39
Detected 21:27 21:29 21:30 21:32 21:33 21:35 21:35 21:37 21:37 21:39
DPT./ARR. time detection was perfect
![Page 53: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/53.jpg)
53 WOOT ’15 Networked Systems Laboratory
Why we failed to detect in Case 3 ?
DPT ARR
DPT ARR
DPT ARR
DPT ARR
DPT ARR
Scheduled 21:26 21:28 21:28 21:32 21:32 21:35 21:35 21:37 21:37 21:39
Observed 21:27 21:29 21:30 21:32 21:33 21:35 21:35 21:37 21:37 21:39
Detected 21:27 21:29 21:30 21:32 21:33 21:35 21:35 21:37 21:37 21:39
The train was delayed at the time of measurement
![Page 54: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/54.jpg)
54 WOOT ’15 Networked Systems Laboratory
Discussion - Train Operation
#1001 On time
#1002 On time
#1003 On time
#1004 On time
#8001 Delayed
#8002 Delayed
#8003 Suspended
#8004 Delayed
Detectable Undetectable × × )
![Page 55: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/55.jpg)
55 WOOT ’15 Networked Systems Laboratory
Discussion - Train Operation (cont. )
• Multiple observations – An adversary can figure out locations frequently visited by the target
in a statistical way
• Automatic train operation – It will work to increase the accuracy of train operations
![Page 56: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/56.jpg)
56 WOOT ’15 Networked Systems Laboratory
• Restricting access to raw sensor data – Requiring permission
– Wrapping in APIs
• Low-pass filtering – The trade offs between functionality and security
• Staying away from Japan
Countermeasures
![Page 57: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/57.jpg)
57 WOOT ’15 Networked Systems Laboratory
• A novel, proof-of-concept side-channel attack framework called RouteDetector was introduced
• It needs only sensors data which is not protected on the Android platform
• We successfully demonstrated that a route can be identified, used for a trip by train using timetables and route maps.
Conclusion
![Page 58: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/58.jpg)
58 WOOT ’15 Networked Systems Laboratory
Q&A
![Page 59: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/59.jpg)
59 WOOT ’15 Networked Systems Laboratory
![Page 60: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/60.jpg)
60 WOOT ’15 Networked Systems Laboratory
Discussion - Types of Vehicles
? ?
![Page 61: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/61.jpg)
61 WOOT ’15 Networked Systems Laboratory
Discussion - Types of Vehicles (cont. )
• An airplane • A mono rail
– Possible candidate to be attacked
• A bus average time waiting at traffic lights > average time waiting at bus stops
![Page 62: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/62.jpg)
62 WOOT ’15 Networked Systems Laboratory
Discussion – Detection errors
Escalator
V
• Our classifier can predict the activity of riding an escalator as "Vehicle"
• To distinguish between escalator and train, we apply a timing heuristics;
![Page 63: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/63.jpg)
63 WOOT ’15 Networked Systems Laboratory
Discussion – Detection errors (cont. )
• We consider a short "vehicle activity" (less than 60 seconds) as other activities
- Coping with long escalator ride, e.g., more than 60 seconds, is for future work.
![Page 64: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/64.jpg)
64 WOOT ’15 Networked Systems Laboratory
Map: https://www.google.co.jp/maps Boss icon: http://pictogram-free.com/03-mark/042-mark.html Location icon: http://www.cliparthut.com/map-icons-clip-art-clipart-W13lHd.html Train photo: http://blogs.yahoo.co.jp/yuuki_20140313/40442989.html Sensor image: http://fscomps.fotosearch.com/bigcomps/CSP/CSP139/k1398123.jpg Train icon: https://commons.wikimedia.org/wiki/File:Bahn_aus_Zusatzzeichen_1024-15.svg Timetable icon: http://simpleicon.com/book-2.html Airplane icon: http://www.sozai-library.com/wp-content/uploads/2013/05/00289-450x337.jpg Bus icon: http://4vector.com/i/free-vector-bus-symbol-black-clip -art_110561_Bus_Symbol_black_clip_art_hight.png Home icon: http://free-icon.org/data/dl_05/m_06.gif Bar icon: http://map-icon.com/material/eatanddrink/m_05.gif Malware icon: http://freeiconbox.com/icon/256/30992.png Smartphone with xyz axis: http://vnreview.vn/image/61/36/613648.jpg?t=1373940855536 Photo in the train :http://www.uraken.net/rail/alltrain/ec/syanai/207e.jpg Nexus7: http://i.ytimg.com/vi/Vj1koPa9FGQ/maxresdefault.jpg Htc j: http://adcdn.goo.ne.jp/images/sumaho/model/au/htc_j_butterfly_htl21_c.jpg
Image sources
![Page 65: Sensor-based Positioning System that Exploits Spatio-Temporal Regularity … · 2019-12-18 · WOOT ’15 Networked Systems Laboratory 1 Sensor-based Positioning System that Exploits](https://reader034.vdocuments.site/reader034/viewer/2022042116/5e93ec06a7efb54d1d05cffd/html5/thumbnails/65.jpg)
65 WOOT ’15 Networked Systems Laboratory
Acknowledgements • A part of this work was supported by JSPS Grant-in-Aid for
Challenging Exploratory Research (KAKENHI), Grant number 15K12038.