computer vision - umartalhaumartalha.weebly.com/.../8382263/02_lecture_cv_nust_pnec.pdf ·...

Computer Vision

EE837

NUST-PNEC Computer Vision EE 837 Fall 2014

Image processing basics

Lecture 02 Computer Vision EE837


• Chapter 2

David A. Forsyth and Jean Ponce, “Computer Vision A Modern Approach”, 2nd edition, Prentice Hall, Inc., 2003.

• For image processing details

Rafael C. Gonzalez, Richard E. Woods, “Digital Image Processing”, 3rd edition, Prentice Hall, 2007

Suggested readings


• Binary

• Gray Scale

• Color

Images


Image as a Function

• We can think of an image as a function, f,

• f: R2 R

– f (x, y) gives the intensity at position (x, y)

• A color image is just three functions pasted together. We can write this as a “vector-valued” function:

r(x, y)

f (x, y) = g(x, y)

b (x, y)


Binary Images

• Images with only two values (0 or 1)

• Simple to process and analyze

• Very useful for industrial applications

• Obtained from gray-level (or color) image f(x, y) by thresholding

• Characteristic Function

b(x, y) = 1 if f(x, y) < T

0 if f(x, y) >= T


0: Black1: White

p

q

X

Y

Row 1

Row q

1 1 1

0 0 0 00

1

Binary Images


10 5 9

100

Gray scale image


Gray scale image


Color Image

Red, Green, Blue channels


Image thresholding


Region has brighter or darker color, etc.

If pixel > threshold

then pixel = 1

else pixel = 0

Image thresholding


clc; clear all; close allsign = imread('StopSign.jpg','jpg'); red = (sign(:,:,1)>120) & (sign(:,:,2)<100) & (sign(:,:,3)<80); out = red*200; imwrite(out, 'SegStopSign.jpg', 'jpg');subplot(1,2,1); imshow('StopSign.jpg'); title('Stop Sign - Original');subplot(1,2,2); imshow('SegStopSign.jpg'); title('Stop Sign - Segmented');

Image thresholding


clc; clear all; close allsign = imread('Coins.jpg','jpg'); red = (sign(:,:,1)>80) & (sign(:,:,2)<240) & (sign(:,:,3)<140); out = red*200; imwrite(out, 'SegCoins.jpg', 'jpg');subplot(1,2,1); imshow('Coins.jpg'); title('Coins - Original');subplot(1,2,2); imshow('SegCoins.jpg'); title('Coins - Segmented');

Image histogram


• Light Variations

• Camera Electronics

• Surface Reflectance

• Lens

Image noise


• I(x,y) : the true pixel values

• n(x,y) : the noise at pixel (x,y)

ˆ , , , I x y I x y n x y

Image noise


2

22,

n

n x y e

Gaussian noise


• p is uniformly distributed random variable

• l is threshold

• smin and smax are constant

,ˆ ,

min max min

I x y p lI x y

s r s s p l

Salt and Pepper noise


• Derivative: Rate of change

– Speed is a rate of change of a distance

– Acceleration is a rate of change of speed

• Average (Mean)

– Dividing the sum of N values by N

Image derivatives


( ) ( )lim ( )0

df f x f x xf x fx x

dx x

ds

vdt

dva

dt

Derivative

Speed

Acceleration


2 4

32 4

y x x

dyx x

dx

sin

cos ( 1)

xy x e

dy xx edx

Examples


)()()(

lim 0 xfx

xxfxf

dx

dfx

)(1

)1()(xf

xfxf

dx

df

)()1()( xfxfxfdx

df

Discrete derivative


Take value of f(x) at present instant and subtract from the previous

)()1()1( xfxfxfdx

df

)()1()( xfxfxfdx

df

)()1()( xfxfxfdx

df Backward difference

Forward difference

Central difference

Discrete derivative – Finite difference


05201551050)(

005150550)(

2020202510101510)(

xf

xf

xf

Derivative Masks

Backward difference

Forward difference

Central difference

[-1 1]

[1 -1]

[-1 0 1]

Example


( , )f x y

y

x

f

f

y

yxfx

yxf

yxf),(

),(

),(

22),( yx ffyxf

y

x

f

f1tan

Given function

Gradient vector

Gradient magnitude

Gradient direction

Derivatives in 2-D


Image is 2D so we will have 2 partial derivatives

00000

0010100

0010100

0010100

00000

xI

101

101

101

3

1xfDerivative masks

101

101

101

3

1yf

2020201010

2020201010

2020201010

2020201010

2020201010

I

Derivatives of images


Averaging is one way to get rid of noise in any pixel

(-10+0+20) + (-10+0+20) + (-10+0+20) = 30

00000

00000

00000

00000

00000

yI

2020201010

2020201010

2020201010

2020201010

2020201010

I

Derivatives of images


2

2

2)(

x

exg

( ) .011 .13 .6 1 .6 .13 .011g x

2 2

2

( )

2( , )

x y

g x y e

Gaussian Filter


1

x=-3 x=-2 x=-1 x=0 x=1 x=2 x=3

• Most common natural model

• Smooth function, it has infinite number of derivatives

• Fourier Transform of Gaussian is Gaussian

• Convolution of a Gaussian with itself is a Gaussian

• There are cells in eye that perform Gaussian filtering

Properties of Gaussian


• Mean

n

I

n

IIII

n

i

i

n

121

n

Iw

n

IwIwIwI

n

i

ii

nn

12211

Averages

• Weighted mean


Image filtering


• Modify pixels in an image based on some function of a local neighborhood ofthe pixels

10 5 3

4 5 1

1 1 7

Some function

Local image data Modified image data

7

pf

• Replace each pixel by a linear combination of its neighbors

• We don’t want to only do this at a single pixel, of course, but want instead to “run the kernel over the whole image”.

1 1 2 2 3 3

4 4 5 5 6 6

7 7 8 8 9 9

f h f h f h f h

f h f h f h

f h f h f h

( , ) ( , )k l

f h f k l h i k j l

Correlation


f1 f2 f3

f4 f5 f6

f7 f8 f9

f = imageh = kernel/filter

h1 h2 h3

h4 h5 h6

h7 h8 h9

1 9 2 8 3 7

4 6 5 5 6 4

7 3 8 2 9 1

*f h f h f h f h

f h f h f h

f h f h f h

* ( , ) ( , )k l

f h f k l h i k j l

Convolution


f1 f2 f3

f4 f5 f6

f7 f8 f9

f = imageh = kernel/filter

h9 h8 h7

h6 h5 h4

h3 h2 h1

h1 h2 h3

h4 h5 h6

h7 h8 h9

h7 h8 h9

h4 h5 h6

h1 h2 h3

X-flip

Y-flip

)1,1()1,1()1,0()1,()1,1()1,1(

)0,1(),1()0,0(),()0,1(),1(

)1,1()1,1()1,0()1,()1,1()1,1(

hyxfhyxfhyxf

hyxfhyxfhyxf

hyxfhyxfhyxfhf

x

f

x

h

1

1

1

1

),(),(i j

jihiyixfhf

Convolution


-1,0 0,1 1,1

-1,0 0,0 1,0

-1,-1 0,-1 1,-1

Coordinates

*

2 2 2 3

2 1 3 3

2 2 1 2

1 3 2 2

Convolution example


1 -1 -1

1 2 -1

1 1 1

Apply

h? ? ? ?

? ? ? ?

? ? ? ?

? ? ? ?

1 1 1

-1 2 1

-1 -1 1

Rotate

f f * h

2 2 2 3

2 1 3 3

2 2 1 2

1 3 2 2

1 1 1

-1 2 1

-1 -1 1

Convolution example


1 -1 -1

1 2 -1

1 1 1

Apply

h5 ? ? ?

? ? ? ?

? ? ? ?

? ? ? ?

1 1 1

-1 2 1

-1 -1 1

Rotate

f

f * h

2*2 + 1*2 + (-1)*2 + 1*1 = 5

2 2 2 3

2 1 3 3

2 2 1 2

1 3 2 2

1 1 1

-1 2 1

-1 -1 1

Convolution example


1 -1 -1

1 2 -1

1 1 1

Apply

h5 4 ? ?

? ? ? ?

? ? ? ?

? ? ? ?

1 1 1

-1 2 1

-1 -1 1

Rotate

f

f * h

2*2+1*2+(-1)*2+1*1 = 5-1*2+2*2+1*2-1*2-1*1+1*3= 4

2 2 2 3

2 1 3 3

2 2 1 2

1 3 2 2

1 1 1

-1 2 1

-1 -1 1

Convolution example


1 -1 -1

1 2 -1

1 1 1

Apply

h5 4 4 ?

? ? ? ?

? ? ? ?

? ? ? ?

1 1 1

-1 2 1

-1 -1 1

Rotate

f

f * h

2*2+1*2+(-1)*2+1*1 = 5-1*2+2*2+1*2-1*2-1*1+1*3= 4-1*2+2*2+1*3-1*1-1*3+1*3= 4

2 2 2 3

2 1 3 3

2 2 1 2

1 3 2 2

1 1 1

-1 2 1

-1 -1 1

Convolution example


1 -1 -1

1 2 -1

1 1 1

Apply

h5 4 4 -2

? ? ? ?

? ? ? ?

? ? ? ?

1 1 1

-1 2 1

-1 -1 1

Rotate

f

f * h

2*2+1*2+(-1)*2+1*1 = 5-1*2+2*2+1*2-1*2-1*1+1*3= 4-1*2+2*2+1*3-1*1-1*3+1*3= 4-1*2+2*3-1*3-1*3 = -2

2 2 2 3

2 1 3 3

2 2 1 2

1 3 2 2

1 1 1

-1 2 1

-1 -1 1

Convolution example


1 -1 -1

1 2 -1

1 1 1

Apply

h5 4 4 -2

9 ? ? ?

? ? ? ?

? ? ? ?

1 1 1

-1 2 1

-1 -1 1

Rotate

f

f * h

2*2+1*2+(-1)*2+1*1 = 5-1*2+2*2+1*2-1*2-1*1+1*3= 4-1*2+2*2+1*3-1*1-1*3+1*3= 4-1*2+2*3-1*3-1*3 = -21*2+1*2+2*2+1*1-1*2+1*2= 9

Convolution example


And so on ……..

Boarder handling


for interior pixels wherethere is full overlap, we know what to do.

but what values do weuse for pixels that are “off the image” ?

Boarder handling


• One of the simplest methods is zero-padding

Boarder handling


• Replication – replace each off-image pixel with the value from the nearest pixel that is in the image.

Boarder handling


• Reflection – reflect pixel values at the border (as if there was a little mirror there)

Boarder handling


• Wraparound – when going off the right border of the image, you wrap around to the left border. Similarly, when leaving the bottom of the image you reenter at the top. Basically, the image is a big donut

0 0 0

0 1 0

0 0 0

*


Filtering Examples

No effect

0 0 0

0 0 1

0 0 0

*


Filtering Examples

1 1 1

1 1 1

1 1 19

1*


Filtering Examples

Averaging

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

25

1*


Filtering Examples

Original

Original

Pixel offset

Pixel offset

Filtered

Filtered

0

0

8

8

4

0.3

0.3

2.4

8

4

6

4.8


Blurring Examples

*


Filtering Gaussian

Gaussian Smoothing Smoothing by Averaging


Gaussian vs Basic Averaging

Weighted averaging Constant weight

Gaussian Noise After Gaussian SmoothingAfter Averaging


Noise Filtering

Salt & Pepper Noise After Gaussian SmoothingAfter Averaging


Noise Filtering

Does not look anything like what we have seen

Magnitude of the FT

Images in Fourier domain


Images in Fourier domain

Does not look anything like what we have seen

Magnitude of the FT


Original image FFT of original image Low-pass filter

Low-pass image FFT of low-pass image

Low-pass Filtering

Let the low frequencies pass and eliminating the high frequencies.

Generates image with overallshading, but not much detail


Original image FFT of original image High-pass filter

High-pass image FFT of high-pass image

High-pass Filtering

Lets through the high frequencies (the detail), but eliminates the low frequencies (the overall shape). It acts like an edge enhancer.


Boosting High Frequencies

Original image FFT of original image High-boost filter

High boosted image FFT of high-boosted image



Most information at low frequencies!

Fun with Fourier Spectra


• conv: 1-D Convolution

C = conv(A, B) convolves vectors A and B

• conv2: Two dimensional convolution

C = conv2(A, B) performs the 2-D convolution

of matrices A and B.

Some basic MATLAB functions


• filter2: Two-dimensional digital filter

Y = filter2(B,X) filters the data in X with the 2-D

FIR filter in the matrix B

The result, Y, is computed using 2-D correlation and is

the same size as X

filter2 uses CONV2 to do most of the work. 2-D correlation

is related to 2-D convolution by a 180 degree rotation of

the filter matrix



• gradient: Approximate gradient

[FX,FY] = gradient(F) returns the numerical gradient of the matrix F. FX corresponds to dF/dx, FY corresponds to dF/dy

• mean: Average or mean value

For vectors, mean(X) is the mean value (average) of the elements in X



• special: Create predefined 2-D filters

H = fspecial(TYPE) creates a two-dimensional filter H of the specified type. Possible values for TYPE are:

'average' averaging filter;

'gaussian' Gaussian low-pass filter

'laplacian’ filter approximating the 2-D Laplacian operator

'log' Laplacian of Gaussian filter

'prewitt' Prewitt horizontal edge-emphasizing filter

'sobel' Sobel horizontal edge-emphasizing filter

• Example: H=fspecial('gaussian',7,1)

Creates a 7x7 Gaussian filter with variance 1



???

Final project ideas!!!


• I will upload Matlab tutorial on Yahoo group

• Please take a look if you are unfamiliar with Matlab or the image toolbox

• I will upload Homework 1. Due date will be mentioned on it

Yahoo Group update and homework


computer vision - umartalhaumartalha.weebly.com/.../8382263/02_lecture_cv_nust_pnec.pdf ·...

Documents