eye tracking for personalized photography steven scher (ucsc student) james davis (ucsc advisor)...
Post on 20-Dec-2015
216 views
TRANSCRIPT
Eye Tracking for Personalized Photography
Steven Scher (UCSC student)James Davis (UCSC advisor)
Sriram Swaminarayan (LANL advisor)
Who’s Behind the Camera?
Photo Creative Commonshttp://www.flickr.com/photos/jasonpratt
Who’s Behind the Camera?
Photo Creative Commonshttp://www.flickr.com/photos/scobleizer
Human-Computer Hybrid Computation
Photo Creative Commonshttp://www.flickr.com/photos/m500/
Photo Creative Commonshttp://www.flickr.com/photos/popculturegeek
Photo Creative Commonshttp://www.flickr.com/photos/aloha75
Human-Computer Hybrid ComputationSemi-automated “Photoshop” effects• Rotoscoping
– Keyframe-Based Tracking for Rotoscoping and Animation
• Tone Mapping– Interactive local
adjustment of tonal values
Human-Computer Hybrid ComputationAmazon Mechanical Turk
• Object Recognition as “20 questions”– Visual Recognition with Humans in the
Loop
• Object Tracking– Efficiently Scaling Up Video Annotation
with Crowdsourced Marketplaces
Human-Computer Hybrid ComputationEyetrack-automated “Photoshop” effects• Cropping
– Gaze-Based Interaction for Semi-Automatic Photo Cropping
Saliency:What is this a picture of?
Automatic Saliency Detection Harel et al
SaliencyItti, Koch, & Niebur
A Model of Saliency-based Visual Attention for Rapid Scene Analysis
Harel, Koch, & PeronaGraph Based Visual Saliency
• Feature extraction
• Activation
• Normalization
• Summation
Saliency:Who is this a picture of?
Eye Tracking
Half-Mirror
Infrared camera
Infrared Light
Future Expectations:EyeTracker in Camera Viewfinder
Viewfinder
Camera
Future Expectations:EyeTracker in Camera Viewfinder
Canon EOS A2“Eye Controlled Focus”1992
Photo Creative Commonshttp://www.flickr.com/photos/1080p/
Eye Tracks (1kHz)
350 400 450 500 550 600 650 700 750250
300
350
400
450
500
550
600
Eye Tracking
Seconds after finishing calibration
Eye
Trac
king
Err
or (p
ixel
s) o
n 80
0x60
0 sc
reen
Eye Tracks (1kHz)
350 400 450 500 550 600 650 700 750250
300
350
400
450
500
550
600
Gaussian Weighted According to Eyetracker’s Calibration accuracy
Logarithmically Flattened
Automatic Saliency DetectionItti et al
Automatic Saliency Detection Harel et al
Actual Eyetrack
Automatic Saliency DetectionItti et al
Automatic Saliency Detection Harel et al
Actual Eyetrack
Applications
Selective Defocus Content-Aware Retargeting
Applications
Selective Defocus Content-Aware Retargeting
Applications
Selective Defocus Content-Aware Retargeting
Applications
Selective Defocus Content-Aware Retargeting
Original Image
Selectively Defocused
Selectively Defocused
Selectively Defocused
Selectively Defocused
Applications
Selective Defocus Content-Aware Retargeting
Content Aware Retargeting
New Aspect Ratio
Content Aware Retargeting
Seam CarvingRemove one pixel from each row
New Aspect Ratio
Original Image
Simple Resizing
Actual Eyetracks
Eyetrack-based content-aware resizing
Simple resizing
Original Image
Automatic Saliency
Actual Eye Track
Content-Aware ResizingBased on Automatic Saliency
Content-Aware ResizingBased on Eye Tracks