ee 7730: image analysis i introduction. bahadir k. gunturk2 ee 7730 dr. bahadir k. gunturk office:...
TRANSCRIPT
![Page 1: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/1.jpg)
EE 7730: Image Analysis I
Introduction
![Page 2: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/2.jpg)
Bahadir K. Gunturk 2
EE 7730
Dr. Bahadir K. Gunturk Office: EE 225 Email: [email protected] Tel: 8-5621 Office Hours: MW 2:40 – 4:30 Class Hours: MWF 1:40 – 2:30 (CEBA-3129)
![Page 3: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/3.jpg)
Bahadir K. Gunturk 3
EE 7730
We will learn the fundamentals of digital image processing, computer vision, and digital video processing
Lecture slides, problems sets, solutions, study materials, etc. will be posted on the class website. [www.ece.lsu.edu/gunturk/EE7730]
Textbook is not required. References:
Gonzalez/Woods, Digital Image Processing, Prentice-Hall, 2/e. Forsyth/Ponce, Computer Vision: A Modern Approach, Prentice-Hall. Duda, Hart, and Stork, Pattern Classification, John Wiley&Sons, 2001. Tekalp, Digital Video Processing, 1995 Jain, Fundamentals of Digital Image Processing, Prentice-Hall.
![Page 4: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/4.jpg)
Bahadir K. Gunturk 4
Grading Policy
Your grade will be based on Problem Sets + Semester Project: 35% Midterm: 30% Final: 35%
Problem Sets Theoretical problems and MATLAB assignments 4-5 Problem Sets Individually or in two-person teams
Semester Project Each student will give a 15 minute presentation
![Page 5: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/5.jpg)
Bahadir K. Gunturk 5
EE 7740 Image Analysis II
Semester Project Possible project topics will be provided in a month Projects will be done individually Projects will involve MATLAB or C/C++ implementation Each student will give a 15 minute presentation at the end of
the semester
![Page 6: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/6.jpg)
Bahadir K. Gunturk 6
EE 7740 Image Analysis II
Image Analysis I - Outline Digital image fundamentals 2D Fourier transform, sampling, Discrete Cosine Transfrom Image enhancement Human visual system and color image processing Image restoration Image compression Image segmentation Morphology Introduction to digital video processing
![Page 7: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/7.jpg)
Bahadir K. Gunturk 7
Digital Image AcquisitionSensor array
When photons strike, electron-hole pairs are generated on sensor sites.
Electrons generated are collected over a certain period of time.
The number of electrons are converted to pixel values. (Pixel is short for picture element.)
![Page 8: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/8.jpg)
Bahadir K. Gunturk 8
Digital Image Acquisition
Two types of quantization:1. There are finite number of
pixels. (Spatial resolution)2. The amplitude of pixel is
represented by a finite number of bits. (Gray-scale resolution)
![Page 9: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/9.jpg)
Bahadir K. Gunturk 9
Digital Image Acquisition
![Page 10: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/10.jpg)
Bahadir K. Gunturk 10
Digital Image Acquisition• 256x256 - Found on very cheap
cameras, this resolution is so low that the picture quality is almost always unacceptable. This is 65,000 total pixels.
• 640x480 - This is the low end on most "real" cameras. This resolution is ideal for e-mailing pictures or posting pictures on a Web site.
• 1216x912 - This is a "megapixel" image size -- 1,109,000 total pixels -- good for printing pictures.
• 1600x1200 - With almost 2 million total pixels, this is "high resolution." You can print a 4x5 inch print taken at this resolution with the same quality that you would get from a photo lab.
• 2240x1680 - Found on 4 megapixel cameras -- the current standard -- this allows even larger printed photos, with good quality for prints up to 16x20 inches.
• 4064x2704 - A top-of-the-line digital camera with 11.1 megapixels takes pictures at this resolution. At this setting, you can create 13.5x9 inch prints with no loss of picture quality.
![Page 11: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/11.jpg)
Bahadir K. Gunturk 11
Matrix Representation of Images A digital image can be written as a matrix
1 2
[0,0] [0,1] [0, 1]
[1,0] [1,1] [1, 1][ , ]
[ 1,0] [ 1, 1]MxN
x x x N
x x x Nx n n
x M x M N
35 45 20
43 64 52
10 29 39
![Page 12: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/12.jpg)
Bahadir K. Gunturk 12
Image Resolution
![Page 13: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/13.jpg)
Bahadir K. Gunturk 13
Bit Depth – Grayscale Resolution
8 bits 7 bits
6 bits 5 bits
![Page 14: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/14.jpg)
Bahadir K. Gunturk 14
Bit Depth – Grayscale Resolution
4 bits 3 bits2 bits 1 bit
![Page 15: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/15.jpg)
Bahadir K. Gunturk 15
Digital Color Images
1 2[ , ]Rx n n
1 2[ , ]Gx n n
1 2[ , ]Bx n n
![Page 16: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/16.jpg)
Bahadir K. Gunturk 16
Video
= vertical position
= horizontal position
= frame number
1 2 3[ , , ]x n n n
1n
2n
3n
~24 frames per second.
![Page 17: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/17.jpg)
Bahadir K. Gunturk 17
Why do we process images?
To facilitate their storage and transmission To prepare them for display or printing To enhance or restore them To extract information from them To hide information in them
![Page 18: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/18.jpg)
Bahadir K. Gunturk 18
Image Processing Example
Image Restoration
Original image Blurred Restored by Wiener filter
![Page 19: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/19.jpg)
Bahadir K. Gunturk 19
Image Processing Example
Noise Removal
Noisy image Denoised by Median filter
![Page 20: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/20.jpg)
Bahadir K. Gunturk 20
Image Processing Example
Image Enhancement
Histogram equalization
![Page 21: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/21.jpg)
Bahadir K. Gunturk 21
Image Processing Example Artifact Reduction in Digital Cameras
Original scene Captured by a digital camera
Processed to reduce artifacts
![Page 22: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/22.jpg)
Bahadir K. Gunturk 22
Image Processing Example
Image Compression
Original image 64 KB
JPEG compressed 15 KB
JPEG compressed 9
KB
![Page 23: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/23.jpg)
Bahadir K. Gunturk 23
Image Processing Example
Object Segmentation
“Rice” image Edges detected using Canny filter
![Page 24: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/24.jpg)
Bahadir K. Gunturk 24
Image Processing Example
Resolution Enhancement
![Page 25: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/25.jpg)
Bahadir K. Gunturk 25
Image Processing Example
Watermarking
Original image
Hidden message
Generate watermark
Watermarked image
Secret key
![Page 26: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/26.jpg)
Bahadir K. Gunturk 26
Image Processing Example
Face Recognition
Surveillance video
Search in the database
![Page 27: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/27.jpg)
Bahadir K. Gunturk 27
Image Processing Example
Fingerprint Matching
![Page 28: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/28.jpg)
Bahadir K. Gunturk 28
Image Processing Example
Segmentation
![Page 29: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/29.jpg)
Bahadir K. Gunturk 29
Image Processing Example
Texture Analysis and Synthesis
Pattern repeated Computer generated
Photo
![Page 30: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/30.jpg)
Bahadir K. Gunturk 30
Image Processing Example
Face detection and tracking
http://www-2.cs.cmu.edu/~har/faces.html
![Page 31: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/31.jpg)
Bahadir K. Gunturk 31
Image Processing Example
Face Tracking
![Page 32: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/32.jpg)
Bahadir K. Gunturk 32
Image Processing Example
Object Tracking
![Page 33: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/33.jpg)
Bahadir K. Gunturk 33
Image Processing Example
Virtual Controls
![Page 34: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/34.jpg)
Bahadir K. Gunturk 34
Image Processing Example
Visually Guided Surgery
![Page 35: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/35.jpg)
Bahadir K. Gunturk 35
Cameras
First camera was invented in 16th century. It used a pinhole to focus light rays onto a wall or
translucent plate.
Take a box, prick a small hole in one of its sides with a pin, and then replace the opposite side with a translucent plate.
Place a candle on the pinhole side, you will see an inverted image of the candle on the translucent plate.
![Page 36: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/36.jpg)
Bahadir K. Gunturk 36
Perspective Projection
Perspective projection equations
' ' 'x y z
x y z
![Page 37: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/37.jpg)
Bahadir K. Gunturk 37
Pinhole Camera Model If the pinhole were really reduced to a point, exactly one light ray would
pass through each point in the image plane. In reality, each point in the image place collects light from a cone of rays.
![Page 38: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/38.jpg)
Bahadir K. Gunturk 38
Pinhole Cameras
Pinhole too big - many directions are averaged, blurring the image Pinhole too small - diffraction effects blur the image
![Page 39: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/39.jpg)
Bahadir K. Gunturk 39
Cameras With Lenses
Most cameras are equipped with lenses. There are two main reasons for this:
To gather light. For an ideal pinhole, a single light ray would reach each point the image plane. Real pinholes have a finite size, so each point in the image plane is illuminated by a cone of light rays. The larger the hole, the wider the cone and the brighter the image => blurry pictures. Shrinking the pinhole produces sharper images, but reduces the amount of light and may introduce diffraction effects.
To keep the picture in sharp focus while gathering light from a large area.
![Page 40: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/40.jpg)
Bahadir K. Gunturk 40
Compound Lens Systems
![Page 41: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/41.jpg)
Bahadir K. Gunturk 41
Real Lenses
Rays may not focus at a single point.
Spherical aberration
Spherical aberration can be eliminated completely by designing aspherical lenses.
![Page 42: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/42.jpg)
Bahadir K. Gunturk 42
Real Lenses
Chromatic aberration
The index of refraction is a function of wavelength. Light at different wavelengths follow different paths.
![Page 43: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/43.jpg)
Bahadir K. Gunturk 43
Real Lenses
Chromatic Aberration
![Page 44: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/44.jpg)
Bahadir K. Gunturk 44
Real Lenses Special lens systems using two or more pieces of glass with different
refractive indeces can reduce or eliminate this problem. However, not even these lens systems are completely perfect and still can lead to visible chromatic aberrations.
![Page 45: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/45.jpg)
Bahadir K. Gunturk 45
Real Lenses Barrel Distortion & Pincushion Distortion
Stop (Aperture)
Causes of distortion
(normal)
Chief ray
![Page 46: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/46.jpg)
Bahadir K. Gunturk 46
Real Lenses Barrel Distortion & Pincushion Distortion
Distorted Corrected
http://www.vanwalree.com/optics/distortion.htmlhttp://www.dpreview.com/learn/?/Image_Techniques/Barrel_Distortion_Correction_01.htm
![Page 47: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/47.jpg)
Bahadir K. Gunturk 47
Real Lenses
Vignetting effect in a two-lens system. The shaded part of the beam never reaches the second lens. The brightness drop in the image perimeter.
![Page 48: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/48.jpg)
Bahadir K. Gunturk 48
Real Lenses
Optical vignetting example. Left: f/1.4. Right: f/5.6.
f-number
focal length to diameter ratio
![Page 49: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/49.jpg)
Bahadir K. Gunturk 49
Real Lenses
Long exposure time
Short exposure time
![Page 50: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/50.jpg)
Bahadir K. Gunturk 50
Real Lenses
Flare
Hood may prevent flares
![Page 51: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/51.jpg)
Bahadir K. Gunturk 51
Real Lenses
Flare
![Page 52: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/52.jpg)
Bahadir K. Gunturk 52
Compound Lens Systems
http://www.dpreview.com/learn/?/glossary/http://www.cartage.org.lb/en/themes/Sciences/Physics/Optics/Optical/Lens/Lens.htmhttp://www.vanwalree.com/optics.html
![Page 53: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/53.jpg)
Bahadir K. Gunturk 53
Digital Camera Pipeline
Auto-exposure algorithms measure brightness over discrete scene regions to compensate for overexposed or underexposed areas by manipulating shutter speed and/or aperture size. The net goals here are to maintain relative contrast between different regions in the image and to achieve a good overall quality.
(from Katz and Gentile)
![Page 54: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/54.jpg)
Bahadir K. Gunturk 54
Digital Camera Pipeline
Auto-focus algorithms divide into two categories. Active methods use infrared or ultrasonic emitters/receivers to estimate the distance between the camera and the object being photographed. Passive methods, on the other hand, make focusing decisions based on the received image in the camera.
![Page 55: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/55.jpg)
Bahadir K. Gunturk 55
Digital Camera Pipeline
Lens distortion correctionThis set of algorithms accounts for the physical properties of lenses that warp the output image compared to the actual scene the user is viewing. Different lenses can cause different distortions; for instance, wide-angle lenses create a "barrel distortion", while telephoto lenses create a "pincushion distortion“.
![Page 56: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/56.jpg)
Bahadir K. Gunturk 56
Digital Camera Pipeline
Vignetting (shading distortion) reduces image brightness in the area around the lens. Chromatic aberration causes color fringes around an image. The media processor needs to mathematically transform the image in order to correct for these distortions.
![Page 57: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/57.jpg)
Bahadir K. Gunturk 57
Digital Camera Pipeline
Sensor's output needs to be gamma-corrected to account for eventual display, as well as to compensate for nonlinearities in the sensor's capture response.
![Page 58: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/58.jpg)
Bahadir K. Gunturk 58
Digital Camera Pipeline
Image stability compensation, or hand-shaking correction is another area of preprocessing. Here, the processor adjusts for the translational motion of the received image, often with the help of external transducers that relate the real-time motion profile of the sensor.
![Page 59: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/59.jpg)
Bahadir K. Gunturk 59
Digital Camera Pipeline
White balance is another important stage of preprocessing. When we look at a scene, regardless of lighting conditions, our eyes tend to normalize everything to the same set of natural colors. For instance, an apple looks deep red to us whether we're indoors under fluorescent lighting, or outside in sunny weather. However, an image sensor's "perception" of color depends largely on lighting conditions, so it needs to map its acquired image to appear natural in its final output. This mapping can be done either manually or automatically.
![Page 60: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/60.jpg)
Bahadir K. Gunturk 60
Digital Camera Pipeline
Demosaicking (Bayer interpolation) estimates missing color samples in single-chip cameras.
![Page 61: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/61.jpg)
Bahadir K. Gunturk 61
Digital Camera Pipeline
In this stage, the interpolated RGB image is transformed to the targeted output color space (if not already in the right space). For compression or display to a television, this will usually involve an RGBYCbCr matrix transformation, often with another gamma correction stage to accommodate the target display. The YCbCr outputs may also be chroma subsampled at this stage to the standard 4:2:2 format for color bandwidth reduction with little visual impact.
![Page 62: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/62.jpg)
Bahadir K. Gunturk 62
Digital Camera Pipeline
PostprocessingIn this phase, the image is perfected via a variety of filtering operations before being sent to the display and/or storage media. For instance, edge enhancement, pixel thresholding for noise reduction, and color-artifact removal are all common at this stage.
![Page 63: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/63.jpg)
Bahadir K. Gunturk 63
Digital Camera Pipeline
Display / Compress / StoreOnce the image itself is ready for viewing, the image pipe branches off in two different directions. In the first, the postprocessed image is output to the target display, usually an integrated LCD screen (but sometimes an NTSC/PAL television monitor, in certain camera modes). In the second, the image is sent to the media processor's compression algorithm, where industry-standard compression techniques (JPEG, for instance) are applied before the picture is stored locally in some storage medium (e.g., Flash memory card).
![Page 64: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/64.jpg)
Bahadir K. Gunturk 64
Review: Linear Systems
We define a system as a unit that converts an input function into an output function.
System operatorIndependent variable
![Page 65: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/65.jpg)
Bahadir K. Gunturk 65
Linear Systems
Then the system H is called a linear system.
where fi(x) is an arbitrary input in the class of all inputs {f(x)}, and gi(x) is the corresponding output.
Let
If
A linear system has the properties of additivity and homogeneity.
![Page 66: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/66.jpg)
Bahadir K. Gunturk 66
Linear Systems
for all fi(x) {f(x)} and for all x0.
The system H is called shift invariant if
This means that offsetting the independent variable of the input by x0 causes the same offset in the independent
variable of the output. Hence, the input-output relationship remains the same.
![Page 67: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/67.jpg)
Bahadir K. Gunturk 67
Linear Systems
The operator H is said to be causal, and hence the system described by H is a causal system, if there is no output before there is an input. In other words,
A linear system H is said to be stable if its response to any bounded input is bounded. That is, if
where K and c are constants.
![Page 68: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/68.jpg)
Bahadir K. Gunturk 68
Linear Systems
(a)
ax
(x-a)
A unit impulse function, denoted (a), is defined by the expression
![Page 69: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/69.jpg)
Bahadir K. Gunturk 69
Linear Systems
A unit impulse function, denoted (a), is defined by the expression
Then
![Page 70: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/70.jpg)
Bahadir K. Gunturk 70
Linear Systems
is called the impulse response of H.
The term
From the previous slide
It states that, if the response of H to a unit impulse [i.e., h(x, )], is known, then response to any input f can be computed using the preceding integral. In other words, the response of a linear system is characterized completely by its impulse response.
![Page 71: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/71.jpg)
Bahadir K. Gunturk 71
Linear Systems
and the integral becomes
If H is a shift-invariant system, then
This expression is called the convolution integral. It states that the response of a linear, fixed-parameter system is completely characterized by the convolution of the input with the system impulse response.
![Page 72: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/72.jpg)
Bahadir K. Gunturk 72
Linear Systems
[ ]* [ ] [ ] [ ]m
f n h n f m h n m
Convolution of two functions is defined as
In the discrete case
( )* ( ) ( ) ( )f x h x f h x d
![Page 73: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/73.jpg)
Bahadir K. Gunturk 73
Linear Systems
1 2
1 2 1 2 1 2 1 1 2 2[ , ]** [ , ] [ , ] [ , ]m m
f n n h n n f m m h n m n m
1 2[ , ]h n n is a linear filter.
In the 2D discrete case
![Page 74: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/74.jpg)
Bahadir K. Gunturk 74
Example
1 1 11
1 1 19
1 1 1
* =
![Page 75: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/75.jpg)
Bahadir K. Gunturk 75
Example
* =1 1 1
1 8 1
1 1 1
![Page 76: EE 7730: Image Analysis I Introduction. Bahadir K. Gunturk2 EE 7730 Dr. Bahadir K. Gunturk Office: EE 225 Email: bahadir@ece.lsu.edubahadir@ece.lsu.edu](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649edd5503460f94bee46f/html5/thumbnails/76.jpg)
Bahadir K. Gunturk 76
Try MATLAB
f=imread(‘saturn.tif’);
figure; imshow(f);
[height,width]=size(f);
f2=f(1:height/2,1:width/2);
figure; imshow(f2);
[height2,width2=size(f2);
f3=double(f2)+30*rand(height2,width2);
figure;imshow(uint8(f3));
h=[1 1 1 1; 1 1 1 1; 1 1 1 1; 1 1 1 1]/16;
g=conv2(f3,h);
figure;imshow(uint8(g));