Super-Resolution

Presented By : Rashmi Pandey

Guided By : Prof. Purvi Rekh

Basic terminology

Low-Resolution (LR):

Pixel density within an image is small, therefore

offering less details.

High-Resolution (HR):

Pixel density within an image is larger, therefore

offering more details.

Super resolution (SR):

Obtaining a HR image from one or multiple LR

images

Useful in many practical cases where multiple

frames of the same scene can be obtained. It is

including video applications, medical imaging and

satellite imaging.

Synthetic zooming of region of interest (ROI) is

another important application in surveillance,

forensic, scientific, medical, and satellite imaging.

Applications of SR images

LR image Super Resolved image

(Conti......)

(Conti......)

Reduce the pixel size by increasing no. of pixels

per unit area.

Problem : Amount of light available per pixel also

decreases

Increase chip-size

Problem : Increase of capacitance leads storage problem

SR image Reconstruction

Advantage : Cost less and computationally effective

How to increase image resolution?

Basic Premise for SR

Introduction of Super Resolution

(Conti......)

Observation model

Assumed known

X

High- Resolution

Image H

H

Blur

1

N

F =I 1

F N

Geometric

Warp

D

D 1

N

Decimation

V 1

V N

Additive Noise

Y 1

Y N

Low- Resolution

Images

N1kkkkkk VXY

FHD

N1kkkkkk VXY

FHD

The Model as One Equation......

VX

V

V

V

X

Y

Y

Y

Y

N

2

1

NNN

222

111

N

2

1

H

FHD

FHD

FHD

Scheme for Super Resolution

Registration

Interpolation onto the HR grid

Deblurring and removing noise

Registration Represents the estimation of motion information

between the LR images and the reference LR

image

Interpolation on to Grid Obtain uniformly spaced HR image from

nonuniformaly spaced composite of LR

images

Deblurring and Removing Noise

Image restoration is applied to up sampled

image to remove blurring and noise

SR considering Registration Error

Blur Identification

Computationally Efficient SR Algorithm

Advanced Issues in SR

Non-uniform Interpolation

Classical method-multiple LR images

Example Based SR

SR from single image

Frequency domain approach

SR Image Reconstruction Algorithms

This algorithm performs exactly the 3 stages

presented before:

Registration, Interpolation and Restoration

Advantages

Few computational power

Real-time applications possible

Disadvantages

Only works exactly, when blur and noise

characteristics are the same for all LR images

Restoration step ignores errors caused in the

interpolation step

1.Non-uniform interpolation

Non-uniform interpolation SR reconstruction results by (a) nearest

neighbor interpolation, (b) bilinear interpolation, (c) non-uniform

interpolation using four LR images, and (d) de-bluring part (c).

(a) (b)

(c) (d)

2.Classical multi-frame based SR

Given: A set of low-quality images:

Required: Fusion of these images into a higher resolution image

How?

Actual super-resolution reconstruction result using sub pixel misalignment

Disadvantages : Limited only small increases in resolution

Algorithm uses a training set to learn the fine

details of an image at low resolution.

Maintain image database with HR/LR image pairs

Replace similar LR patches with corresponding HR

patches.

3.Example Based SR

+

LR HR

Disadvantages : Not guaranteed to provide HR image

Combine multi image SR with example based SR

Without use external source

Patch Redundancy :

Use patch redundancy in same scale to model multi

image super resolution problem

Use patch redundancy in different scales to model

example based super resolution problem

4.SR from single image

Bi-cubic interpolation Using Single Image

LR

Experimental Results

Experimental Results

Nearest Neighbor SR from Single image

LR

This algorithm is based on 3 principles:

Shifting property of the Fourier transform (FT)

Aliasing relationship between continuous FT of HR

image and the DFT of LR images (See below figure )

Band limited HR images

Advantages:

Clear demonstration of

LR and HR relationship

Capable of reducing h/w

complexity

Disadvantages:

Lack of data correlation in

frequency domain

5. Frequency Domain Approach

Conclusion

References 1. M. Elad and A. Feuer, “Restoration of Single Super-Resolution Image From Several Blurred,

Noisy and Down-Sampled Measured Images”, the IEEE Trans. on Image Processing, Vol. 6, no. 12, pp. 1646-58, December 1997.

2. M. Elad and A. Feuer, “Super-Resolution Restoration of Continuous Image Sequence - Adaptive Filtering Approach”, the IEEE Trans. on Image Processing, Vol. 8. no. 3, pp. 387-395, March 1999.

3. M. Elad and A. Feuer, “Super-Resolution reconstruction of Continuous Image Sequence”, the IEEE Trans. On Pattern Analysis and Machine Intelligence (PAMI), Vol. 21, no. 9, pp. 817-834, September 1999.

4. M. Elad and Y. Hel-Or, “A Fast Super-Resolution Reconstruction Algorithm for Pure Translational Motion and Common Space Invariant Blur”, the IEEE Trans. on Image Processing, Vol.10, No. 8, pp.1187-93, August 2001.

5. S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, “Fast and Robust Multi-Frame Super-resolution”, IEEE Trans. On Image Processing, Vol. 13, No. 10, pp. 1327-1344, October 2004.

6. S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advanced and Challenges in Super-Resolution", the International Journal of Imaging Systems and Technology, Vol. 14, No. 2, pp. 47-57, Special Issue on high-resolution image reconstruction, August 2004.

7. S. Farsiu, M. Elad, and P. Milanfar, “Multi-Frame Demosaicing and Super-Resolution of Color Images”, IEEE Trans. on Image Processing, vol. 15, no. 1, pp. 141-159, Jan. 2006.

8. S. Farsiu, M. Elad, and P. Milanfar, "Video-to-Video Dynamic Superresolution for Grayscale and Color Sequences," EURASIP Journal of Applied Signal Processing, Special Issue on Superresolution Imaging , Volume 2006, Article ID 61859, Pages 1–15.

9. D. Glasner, S. Bagon and M. Irani, "Super-resolution from a single image," in IEEE 12th

International Conference on Computer Vision (ICCV 2009), Kyoto, Japan, Sep. 29 - Oct. 2,

2009, pp. 349-356.