jitter camera: high resolution video from a low resolution detector
DESCRIPTION
Jitter Camera: High Resolution Video from a Low Resolution Detector. Moshe Ben-Ezra, Assaf Zomet and Shree K. Nayar IEEE CVPR Conference June 2004, Washington DC, USA. Plasma Display Resolution 1366x768. MiniDV Camera. Resolution: 720x480. Digital Camera. Resolution: 2592x1944. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/1.jpg)
Jitter Camera: High Resolution Video from a Low
Resolution Detector
Moshe Ben-Ezra, Assaf Zomet and Shree K. Nayar
IEEE CVPR ConferenceJune 2004, Washington DC, USA
![Page 2: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/2.jpg)
Video Resolution
MiniDV Camera. Resolution: 720x480
Digital Camera. Resolution: 2592x1944
Plasma DisplayResolution 1366x768
![Page 3: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/3.jpg)
Fundamental Resolution Tradeoff
Spatial resolution (pixels)
Tem
pora
l res
olut
ion
(fps
)
30
330K720x480
Conventional video camera
130
3
3M2048x1536
Hi-resolution still Camera
![Page 4: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/4.jpg)
Super-Resolution
Sequence taken by a moving camera
High-Resolution computed image
Super-Resolution
Shechtman, Caspi, and Irani ECCV2002Zomet and S. Peleg. ICIP2000Baker and Kanade. CVPR2000Chiang and Boult, IVC2000
Capeland, Zisserman ICPR2000Elad and Feuer IP1997 Irani and Peleg GMIP1996
![Page 5: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/5.jpg)
Super Resolution
All SampledImages
y = (D G)x + z
Decimation
Blurring Op.
Hi Res. Image
Noise
![Page 6: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/6.jpg)
Motion Blur Hurts Us Again!
![Page 7: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/7.jpg)
Capture Images without Motion Blur
![Page 8: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/8.jpg)
Effect of Motion Blur on Super-Resolution
Input: No Motion Blur Super-Resolution Result
Input : With Motion Blur (known) Super-Resolution Result
![Page 9: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/9.jpg)
Quantifying The Affect of Motion Blur
Empirical tests: RMS error. Volume of Solutions (Linear Model):
y Ax z x A 1y A 1z
High-ResolutionImage
Noise (Quantization)
InputImages
Volume of Solutions 1/det(A)
Blur &Decimation
Baker and Kanade
![Page 10: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/10.jpg)
How Bad is Motion Blur for Super-Resolution?
Space of Super-Resolution Solutions
0 1 2 3 4 5
Motion blur in pixels
RMS Error After Super-Resolution
0 1 2 3 4 5
18
Motion blur in pixels
![Page 11: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/11.jpg)
Avoid Motion Blur using Jitter Sampling
Conventional Sampling
Time
Space
Spatial Jitter Sampling
Time
Space
![Page 12: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/12.jpg)
The Jitter CameraLens Detector
Micro-actuator
![Page 13: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/13.jpg)
The Jitter CameraLens Detector
Micro-actuator
Detector is a light weight device!
Jitter is instantaneous and synchronous
![Page 14: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/14.jpg)
Computer ControlledX Micro-Actuator
Computer ControlledY Micro-Actuator
Board Camera
Lens
![Page 15: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/15.jpg)
![Page 16: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/16.jpg)
1μm X Pixels
Y P
ixel
s
Jitter Mechanism Accuracy
Desired locations. Actual locations.
![Page 17: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/17.jpg)
Result: Resolution Chart
Super-Resolution ImageFour Images fromthe Jitter Camera
![Page 18: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/18.jpg)
Result: Color DeMosaicing and Super-Resolution
1 (out of 4) Jitter camera Image Super-Resolution
![Page 19: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/19.jpg)
Jitter Video (Stabilized)
How can we handle dynamic scenes?
![Page 20: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/20.jpg)
Adaptive Super-Resolution for Dynamic Scenes
Static blocks: 4 frames used.
Occlusions: 1 frame used.
Moving object: 2 - 4 frames used
![Page 21: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/21.jpg)
![Page 22: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/22.jpg)
I-3 I-2 I-1 I I+1 I+2 I+3
Adaptive Super-Resolution Algorithm
1. Estimate the aliasing error ‘’ (stdv) for each block Ik in I.2. Compute robust block matching between all pairs {I}{I1,2,3}. Use
‘’ as a scale factor for an M-Estimator error function.
3. For each block Ik try to find 3 matching blocks {Ix}k, s.t. : a) SSD(Ik, {Ix}k)-0.5 < 3b) {Ix}k are temporally closest to Ik (smallest x)
4. Apply super-resolution to the selected blocks.
The algorithm degrades gradually from 4-frames super-resolution to a single frame interpolation and deblurring.
![Page 23: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/23.jpg)
Scale EstimateMean 6.4, Stdv 14
Mean 10.5, Stdv 27
Mean 7.5, Stdv 16
Mean 15.2, Stdv 30
Mean 8.6, Stdv 17
Mean 17.7, Stdv 33
Low Res - Hi-ResAliasing Error(Simulated)
Low Res2nd derivative(Simulated)
![Page 24: Jitter Camera: High Resolution Video from a Low Resolution Detector](https://reader035.vdocuments.site/reader035/viewer/2022062500/568152ba550346895dc0dd28/html5/thumbnails/24.jpg)