virtual contact sheets - cs.cmu.eduworichte/talks/pdlretreat... · a contact sheet approach to...
TRANSCRIPT
Virtual Contact Sheets Wolfgang Richter
Kiryong Ha, Alok Shankar, Ardalan Amiri Sani*, Jan Harkes, Lin Zhong*, Mahadev Satyanarayanan
PARALLEL DATA LABORATORY Carnegie Mellon University
*Rice University
Crowdsourcing the News [1] “When every cell phone can record
video and take pictures, everyone is a potential news source.”
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 2
Who killed Ian Tomlinson?
CNN: Caught Red Handed [2]
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 3
I’m a sneaky thief.
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 4
Problem
Cloud
Recent, not uploaded
Older, uploaded
Fundamental Mobile Challenges • Real-time sync not possible:
• Energy • Infinite battery life desired
• Bandwidth • Increasingly metered and charged
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 5
How far can we degrade the objects in a search before it loses meaning?
Just-in-Time Search 1. Check virtual contact sheet
• Collection of fidelity-reduced images in the cloud
2. Pull full fidelity image on hit • Search full fidelity; present full fidelity results
to searcher
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 6
Cloud
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 7
Near Real-Time Sync (1) Search hit (2) Retrieve
Contact Sheet Analogy
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 8
Evaluating Feasibility [3]
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 9
Face Detection, RGB Histogram, Brightness Filter
Benchmark
Using Varying Resolution and JPEG Quality Factor
Energy Savings
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 10
0
1000
2000
3000
4000
5000
6000
7000
Full 640x480 320x240 160x120 80x60 40x30 20x20
En
erg
y C
on
sum
ptio
n (
mJ)
Fidelity
Energy
10 images = 5 seconds call time (Droid)
Bandwidth Savings
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 11
0
50
100
150
200
250
300
350
400
Full 640x480 320x240 160x120 80x60 40x30 20x20
Bandw
idth
(K
B)
Fidelity
Bandwidth
Orders of magnitude savings in bandwidth.
What about Search Accuracy?
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 12
0
0.2
0.4
0.6
0.8
1
640x480 320x240 160x120 80x60
Filt
er
Acc
ura
cy
Fidelity
0.650
0.424
10095755025
No one-size-fits-all sweet spot.
Face Detection
Conclusion & Future Work • Searches retain meaning
• No one-size-fits-all, but • Orders of magnitude bandwidth savings • Significant energy savings
• More search primitives? • Perceptual hashing • Face recognition • Texture Filters
• Better fidelity reduction? • Keep faces at high fidelity
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 13
Further Reading [1] TED. http://www.ted.com/talks/paul_lewis_crowdsourcing_the_news.html.
[2] CNN. http://www.cnn.com/2010/CRIME/08/24/new.jersey.theft.photo/index.html?hpt=C1.
[3] SimpleCV. http://simplecv.org/.
Jan Harkes, Mahadev Satyanarayanan, Ardalan Amiri Sani, and Benjamin Gilbert. A Contact Sheet Approach to Searching Untagged Images on Smartphones. CMU Technical Report CMU-CS-11-132.
Ardalan Amiri Sani, Wolfgang Richter, Xuan Bao, Trevor Narayan, Mahadev Satyanarayanan, Lin Zhong, and Romit Roy Choudhury. Opportunistic Content Search of Smartphones. Rice University Technical Report TR0627-2011.
Xuan Bao, Trevor Narayan, Ardalan Amiri Sani, Wolfgang Richter, Romit Roy Choudhury, Lin Zhong, and Mahadev Satyanarayanan. The Case for Context-Aware Compression. HotMobile 2011.
Jason Flinn and Mahadev Satyanarayanan. Energy-aware Adaptation for Mobile Applications. SOSP 1999.
Y. Nimmagadda, K. Kumar, and Y. H. Lu. Energy-efficient image compression in mobile devices for wireless transmission. ICME 2009.
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 14
Full Fidelity
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 15 12,000+ Flickr dataset
Average photo size: 374 KB
Reduce Resolution
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 16 12,000+ Flickr dataset
Average photo size: 221 KB
JPEG Quality Factor
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 17 12,000+ Flickr dataset
Average photo size: 6 KB
Bandwidth Savings Table
Wolfgang Richter © November 11 http://www.pdl.cmu.edu/ 18
Resolution 100 95 75 50 25 5
640x480 221 KB 114 KB 47 KB 31 KB 19 KB 6 KB
320x240 61 KB 33 KB 14 KB 10 KB 6 KB 2 KB
160x120 17 KB 10 KB 5 KB 3 KB 2 KB 826 B
80x60 5 KB 3 KB 2 KB 1 KB 892 B 460 B
40x30 2 KB 1 KB 725 B 588 B 484 B 345 B
20x20 1000 B 727 B 501 B 436 B 385 B 316 B