A Contact Sheet Approach toSearching Untagged Images on Smartphones
Wolfgang Richter*, Kiryong Ha*, Alok Shankar*, Ardalan Amiri Saniꝉ, Jan Harkes*, Lin Zhongꝉ, Mahadev Satyanarayanan*
(*Carnegie Mellon University, ꝉRice University)
Problem
Low Fidelity Evaluation
How do we efficiently search mobile devices in near real-time?
Lessons and Contributions
Solution: Contact Sheets
Two key factors:
CloudOlder,Uploaded
Recent,not uploaded
0
50
100
150
200
250
300
350
400
Full 640x480 320x240 160x120 80x60 40x30 20x20
Bandw
idth
(K
B)
Fidelity
To save energy and bandwidth, we degrade objects being searched and pull full fidelityversions in a just-in-time search.
Diamond ModificationsMISTAndroid Application
C2DM
PollingCamera
UI
Local Storage
Mobile API
Cloud API
MISTDataretriever Diamond GUI
SessionServerdiamondd
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
Full Fidelity 640x480 320x240 160x120 80x60 40x30 20x20
Energ
y C
onsu
mptio
n (
mJ)
Fidelity
No BatchingBatching
• Released Android Market App• Python Flask Web Application • Mobile API via HTTP • Cloud API via HTTP• Full Fidelity Retrieval • Push-based Google C2DM • Polling-based HTTP• Just-in-Time Search
How far can we degrade images in a search before it loses meaning?
Dataset: • 12, 963 Flickr Scraped ImagesFidelity Reduction: • Resolution • JPEG Quality FactorBenchmarked Three Computer Vision Algorithms:
• No One-Size-Fits-All• Significant Energy Savings• Significant Bandwidth Savings
Lessons
Help Us:
Energy Bandwidth Potential Savings?
Contributions
0
0. 2
0. 4
0. 6
0. 8
1
640x480 320x240 160x120 80x60 40x30 20x20
Filt
er
Acc
ura
cy
Fidelity
0.991
0.899
10095755025
0
0. 2
0. 4
0. 6
0. 8
1
640x480 320x240 160x120 80x60 40x30 20x20
Filt
er
Fa
lse
Ne
ga
tive
Ra
te
Fidelity
0.040
10095755025
0
0. 2
0. 4
0. 6
0. 8
1
640x480 320x240 160x120 80x60 40x30
Filt
er
Acc
ura
cy
Fidelity
0.907
0.803
10095755025
0
0. 2
0. 4
0. 6
0. 8
1
640x480 320x240 160x120 80x60 40x30
Filt
er
Fa
lse
Ne
ga
tive
Ra
te
Fidelity
0.124
0.042
10095755025
0
0. 2
0. 4
0. 6
0. 8
1
640x480 320x240 160x120 80x60
Filt
er
Fa
lse
Ne
ga
tive
Ra
te
Fidelity
0.575
0.281
10095755025
0
0. 2
0. 4
0. 6
0. 8
1
640x480 320x240 160x120 80x60
Filt
er
Acc
ura
cy
Fidelity
0.650
0.424
10095755025
RGB Histogram Face DetectionBrightness