filters - web.cs.hacettepe.edu.trpinar/courses/cmp719/lectures/filte… · • image filters in...
TRANSCRIPT
![Page 1: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/1.jpg)
Filters
CMP 719 – Computer Vision
Pinar Duygulu
Hacettepe University
![Page 2: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/2.jpg)
Today’s topics
• Image Formation
• Image filters in spatial domain– Filter is a mathematical operation of a grid of numbers– Smoothing, sharpening, measuring texture
• Image filters in the frequency domain– Filtering is a way to modify the frequencies of images– Denoising, sampling, image compression
• Templates and Image Pyramids– Filtering is a way to match a template to the image– Detection, coarse-to-fine registration
![Page 3: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/3.jpg)
Images as functions
Source: S. Seitz
![Page 4: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/4.jpg)
Images as functions
• We can think of an image as a function, f, from
R2 to R:• f( x, y ) gives the intensity at position ( x, y )
• Realistically, we expect the image only to be defined over a
rectangle, with a finite range:
– f: [a,b] x [c,d] [0, 255]
• A color image is just three functions pasted
together. We can write this as a “vector-valued”
function:
Source: S. Seitz
( , )
( , ) ( , )
( , )
r x y
f x y g x y
b x y
![Page 5: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/5.jpg)
Digital images• In computer vision we operate on digital (discrete) images:
• Sample the 2D space on a regular grid
• Quantize each sample (round to nearest integer)
• Image thus represented as a matrix of integer values.
Adapted from S. Seitz
2D
1D
![Page 6: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/6.jpg)
Images as discrete functions
• Cartesian Coordinates
Source: Fei Feli Li, Stanford University
![Page 7: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/7.jpg)
Today’s topics
• Image Formation
• Image filters in spatial domain– Filter is a mathematical operation of a grid of numbers– Smoothing, sharpening, measuring texture
• Image filters in the frequency domain– Filtering is a way to modify the frequencies of images– Denoising, sampling, image compression
• Templates and Image Pyramids– Filtering is a way to match a template to the image– Detection, coarse-to-fine registration
![Page 8: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/8.jpg)
Zebras vs. Dalmatians
Both zebras and dalmatians have black and white pixels in about the same number
– if we shuffle the images point-wise processing is not affected
Need to measure properties relative to small neighborhoods of pixels- find different image patterns
![Page 9: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/9.jpg)
Filteringg [m,n] f [m,n]
We want to remove unwanted sources of variation, and keep the information relevant for whatever task we need to solve
Source: Torralba, MIT
![Page 10: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/10.jpg)
Filters
• Filtering:
– Form a new image whose pixels are a combination of original pixel values
- compute function of local neighborhood at each position
• Goals:
• Extract useful information from the images
Features (textures, edges, corners, distinctive points, blobs…)
• Modify or enhance image properties:
super-resolution; in-painting; de-noising, resizing
• Detect patterns
Template matching
Source: Fei Feli Li, Stanford University; James Hays, Brown
![Page 11: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/11.jpg)
Smooth/Sharpen Images... Find edges... Find waldo…Source: Darrell, Berkeley
![Page 12: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/12.jpg)
Source: Fei Feli Li, Stanford University
![Page 13: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/13.jpg)
Common types of noise
– Salt and pepper noise: random occurrences of black and white pixels
– Impulse noise: random occurrences of white pixels
– Gaussian noise: variations in intensity drawn from a Gaussian normal distribution
Source: S. SeitzSource: Darrell, Berkeley
![Page 14: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/14.jpg)
Gaussian noise
Fig: M. Hebert
>> noise = randn(size(im)).*sigma;
>> output = im + noise;
Source: Darrell, Berkeley
![Page 15: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/15.jpg)
First attempt at a solution
• Let’s replace each pixel with an average of all the values in its neighborhood
• Assumptions:
– Expect pixels to be like their neighbors
– Expect noise processes to be independent from pixel to pixel
Source: Darrell, Berkeley
![Page 16: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/16.jpg)
First attempt at a solution
• Let’s replace each pixel with an average of all the values in its neighborhood
• Moving average in 1D:
Source: S. MarschnerSource: Darrell, Berkeley
![Page 17: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/17.jpg)
Weighted Moving Average
• Can add weights to our moving average
• Weights [1, 1, 1, 1, 1] / 5
Source: S. MarschnerSource: Darrell, Berkeley
![Page 18: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/18.jpg)
Weighted Moving Average
• Non-uniform weights [1, 4, 6, 4, 1] / 16
Source: S. MarschnerSource: Darrell, Berkeley
![Page 19: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/19.jpg)
Moving Average In 2D
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Source: S. SeitzSource: Darrell, Berkeley
![Page 20: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/20.jpg)
Moving Average In 2D
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Source: S. SeitzSource: Darrell, Berkeley
![Page 21: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/21.jpg)
Moving Average In 2D
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Source: S. SeitzSource: Darrell, Berkeley
![Page 22: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/22.jpg)
Moving Average In 2D
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20 30
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Source: S. SeitzSource: Darrell, Berkeley
![Page 23: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/23.jpg)
Moving Average In 2D
0 10 20 30 30
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Source: S. SeitzSource: Darrell, Berkeley
![Page 24: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/24.jpg)
Moving Average In 2D
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20 30 30 30 20 10
0 20 40 60 60 60 40 20
0 30 60 90 90 90 60 30
0 30 50 80 80 90 60 30
0 30 50 80 80 90 60 30
0 20 30 50 50 60 40 20
10 20 30 30 30 30 20 10
10 10 10 0 0 0 0 0
Source: S. SeitzSource: Darrell, Berkeley
![Page 25: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/25.jpg)
Correlation filtering
Say the averaging window size is 2k+1 x 2k+1:
Loop over all pixels in neighborhood around image pixel F[i,j]
Attribute uniform weight to each pixel
Now generalize to allow different weights depending on neighboring pixel’s relative position:
Non-uniform weights
Source: Darrell, Berkeley
![Page 26: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/26.jpg)
Correlation filtering
Filtering an image: replace each pixel with a linear combination of its neighbors.
The filter “kernel” or “mask” H[u,v] is the prescription for the weights in the linear combination.
This is called cross-correlation, denoted
Source: Darrell, Berkeley
![Page 27: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/27.jpg)
Averaging Filter
adapted from Darrell and Freeman, MIT
![Page 28: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/28.jpg)
Averaging Filter
adapted from Darrell and Freeman, MIT
Averaging Filter
![Page 29: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/29.jpg)
Averaging Filter
adapted from Darrell and Freeman, MIT
![Page 30: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/30.jpg)
Averaging Filter
adapted from Darrell and Freeman, MIT
![Page 31: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/31.jpg)
Averaging filter
• What values belong in the kernel H for the moving average example?
0 10 20 30 30
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
111
111
111
“box filter”
?
Source: Darrell, Berkeley
![Page 32: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/32.jpg)
Smoothing by averaging
depicts box filter: white = high value, black = low value
original filtered
Source: Darrell, Berkeley
![Page 33: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/33.jpg)
Example
Source: Martial Hebert, CMU
![Page 34: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/34.jpg)
Example
Source: Martial Hebert, CMU
![Page 35: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/35.jpg)
Example
Source: Martial Hebert, CMU
![Page 36: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/36.jpg)
Smoothing by averaging
Source: Martial Hebert, CMU
![Page 37: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/37.jpg)
Smoothing by averaging
Source: Martial Hebert, CMU
![Page 38: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/38.jpg)
Smoothing by averaging
Source: Martial Hebert, CMU
![Page 39: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/39.jpg)
Smoothing by averaging
Source: Martial Hebert, CMU
![Page 40: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/40.jpg)
Gaussian filter
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
1 2 1
2 4 2
1 2 1
• What if we want nearest neighboring pixels to have the most influence on the output?
This kernel is an approximation of a Gaussian function:
Source: S. SeitzSource: Darrell, Berkeley
A weighted average that weights pixels at its centermuch more strongly than its boundaries
![Page 41: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/41.jpg)
Smoothing with a Gaussian
Source: Darrell, Berkeley
![Page 42: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/42.jpg)
Smoothing with a Gaussian
Source: Martial Hebert, CMU
![Page 43: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/43.jpg)
Smoothing with a Gaussian
Result of blurring using a uniform local model
Produces a set of narrow vertical horizontal and vertical bars – ringing effect
Result of blurring using a set of Gaussian weights
Source: David Forsyth, UIUC
![Page 44: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/44.jpg)
Smoothing with a Gaussian
Source: Martial Hebert, CMU
![Page 45: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/45.jpg)
Gaussian filters• What parameters matter here?
• Size of kernel or mask
– Note, Gaussian function has infinite support, but discrete filters use finite kernels
σ = 5 with 10 x 10 kernel
σ = 5 with 30 x 30 kernel
Source: Darrell, Berkeley
![Page 46: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/46.jpg)
Gaussian filters• What parameters matter here?
• Variance of Gaussian: determines extent of smoothing
σ = 2 with 30 x 30 kernel
σ = 5 with 30 x 30 kernel
Source: Darrell, Berkeley
![Page 47: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/47.jpg)
Smoothing with a Gaussian
If is small : the smoothing will have little effect
If is larger : neighboring pixels will have larger weights resulting in consensus of the neighbors
If is very large : details will disappear along with the noise
Source: Martial Hebert, CMU
![Page 48: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/48.jpg)
Gaussian filter
=1
=2
=4
Source: Torralba, MIT
![Page 49: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/49.jpg)
Gaussian smoothing to remove noise
Source: Martial Hebert, CMU
![Page 50: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/50.jpg)
Gaussian smoothing to remove noise
Source: Martial Hebert, CMU
![Page 51: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/51.jpg)
Smoothing with a Gaussian
The effects of smoothingEach row shows smoothingwith gaussians of differentwidth; each column showsdifferent realizations of an image of gaussian noise.
Source: David Forsyth, UIUC
![Page 52: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/52.jpg)
Smoothing with a Gaussian
• Filtered noise is sometimes useful
– looks like some natural textures, can be used to simulate fire, etc.
Source: David Forsyth, UIUC
![Page 53: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/53.jpg)
Gaussian kernel
Source: from Michael Black
![Page 54: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/54.jpg)
Matlab>> hsize = 10;
>> sigma = 5;
>> h = fspecial(‘gaussian’ hsize, sigma);
>> mesh(h);
>> imagesc(h);
>> outim = imfilter(im, h);
>> imshow(outim);
outimSource: Darrell, Berkeley
![Page 55: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/55.jpg)
Smoothing with a Gaussian
for sigma=1:3:10
h = fspecial('gaussian‘, fsize, sigma);
out = imfilter(im, h);
imshow(out);
pause;
end
…
Parameter σ is the “scale” / “width” / “spread” of the Gaussian kernel, and controls the amount of smoothing.
Source: Darrell, Berkeley
![Page 56: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/56.jpg)
Convolution
• Convolution: – Flip the filter in both dimensions (bottom to top, right to left)
– Then apply cross-correlation
Notation for convolution operator
F
H
Source: Darrell, Berkeley
![Page 57: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/57.jpg)
Convolution vs. correlationConvolution
Cross-correlation
For a Gaussian or box filter, how will the outputs differ?
If the input is an impulse signal, how will the outputs differ?Source: Darrell, Berkeley
![Page 58: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/58.jpg)
Predict the filtered outputs
000
010
000
* = ?
000
100
000* = ?
111111111
000020000
-* = ?
Source: Darrell, Berkeley
![Page 59: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/59.jpg)
Practice with linear filters
adapted from Darrell and Freeman, MIT
![Page 60: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/60.jpg)
Practice with linear filters
adapted from Darrell and Freeman, MIT
![Page 61: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/61.jpg)
Practice with linear filters
000
010
000
Original Filtered
(no change)
Source: D. LoweSource: Darrell, Berkeley
![Page 62: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/62.jpg)
Impulse
0 0 0 0 0
0 0 0 0 0
0 0 1 0 0
0 0 0 0 0
0 0 0 0 0
f [m,n] I g h[m k,n l]g[k,l]k,l
g[m,n]
h[m,n]
f[m,n]
=
Source: Torralba, MIT
![Page 63: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/63.jpg)
Practice with linear filters
000
100
000
Original
?
Source: D. LoweSource: Darrell, Berkeley
![Page 64: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/64.jpg)
Practice with linear filters
000
100
000
Original Shifted left
by 1 pixel
with
correlation
Source: D. LoweSource: Darrell, Berkeley
![Page 65: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/65.jpg)
Shifts
0 0 0 0 0
0 0 0 0 0
0 0 0 0 1
0 0 0 0 0
0 0 0 0 0
f [m,n] I g h[m k,n l]g[k,l]k,l
g[m,n]
h[m,n]
f[m,n]
=
2pixels
Source: Torralba, MIT
![Page 66: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/66.jpg)
Practice with linear filters
adapted from Darrell and Freeman, MIT
![Page 67: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/67.jpg)
Practice with linear filters
adapted from Darrell and Freeman, MIT
![Page 68: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/68.jpg)
Practice with linear filters
adapted from Darrell and Freeman, MIT
![Page 69: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/69.jpg)
Sharpening
adapted from Darrell and Freeman, MIT
![Page 70: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/70.jpg)
Sharpening
adapted from Darrell and Freeman, MIT
![Page 71: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/71.jpg)
Practice with linear filters
Original
111111111
000020000
-
Sharpening filter- Accentuates differences with local average
Source: D. LoweSource: Darrell, Berkeley
![Page 72: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/72.jpg)
Filtering examples: sharpening
Source: Darrell, Berkeley
![Page 73: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/73.jpg)
Rectangular filter
g[m,n]
h[m,n]
=
f[m,n]
Source: Torralba, MIT
![Page 74: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/74.jpg)
Source: Fei Fei Li, Stanford
![Page 75: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/75.jpg)
Rectangular filter
g[m,n]
h[m,n]
=
f[m,n]
Source: Torralba, MIT
![Page 76: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/76.jpg)
Rectangular filter
g[m,n]
h[m,n]
=
f[m,n]
Source: Torralba, MIT
![Page 77: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/77.jpg)
Integral image
Source: Torralba, MIT
![Page 78: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/78.jpg)
Shift invariant linear system
• Shift invariant:
– Operator behaves the same everywhere, i.e. the value of the output depends on the pattern in the image neighborhood, not the position of the neighborhood.
• Linear:
– Superposition: h * (f1 + f2) = (h * f1) + (h * f2)
– Scaling: h * (k f) = k (h * f)
Source: Darrell, Berkeley
![Page 79: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/79.jpg)
Properties of convolution• Linear & shift invariant
• Commutative:
f * g = g * f
• Associative
(f * g) * h = f * (g * h)
• Identity:
unit impulse e = […, 0, 0, 1, 0, 0, …]. f * e = f
• Differentiation:
Source: Darrell, Berkeley
![Page 80: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/80.jpg)
Separability
• In some cases, filter is separable, and we can factor into two steps:
– Convolve all rows
– Convolve all columns
Source: Darrell, Berkeley
![Page 81: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/81.jpg)
Separability
• In some cases, filter is separable, and we can factor into two steps: e.g.,
What is the computational complexity advantage for a separable filter of size k x k, in terms of number of operations per output pixel?
f * (g * h) = (f * g) * h
g
h
f
Source: Darrell, Berkeley
![Page 82: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/82.jpg)
Advantages of separability
adapted from Larry Davis, University of Maryland
![Page 83: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/83.jpg)
Seperable Gaussian
![Page 84: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/84.jpg)
Advantages of Gaussians
adapted from Larry Davis, University of Maryland
![Page 85: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/85.jpg)
Effect of smoothing filters
Additive Gaussian noise Salt and pepper noise
Source: Darrell, Berkeley
![Page 86: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/86.jpg)
Median filter
• No new pixel values
introduced
• Removes spikes: good
for impulse, salt &
pepper noise
Source: Darrell, Berkeley
![Page 87: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/87.jpg)
Median filter
Salt and pepper noise
Median filtered
Source: M. Hebert
Plots of a row of the image
Source: Darrell, Berkeley
![Page 88: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/88.jpg)
Median filter
• Median filter is edge preserving
Source: Darrell, Berkeley
![Page 89: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/89.jpg)
Boundary issues
• What is the size of the output?
• MATLAB: filter2(g, f, shape)
– shape = ‘full’: output size is sum of sizes of f and g
– shape = ‘same’: output size is same as f
– shape = ‘valid’: output size is difference of sizes of f and g
f
gg
gg
f
gg
gg
f
gg
gg
full same valid
Source: S. LazebnikSource: Darrell, Berkeley
![Page 90: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/90.jpg)
Boundary issues• What about near the edge?
– the filter window falls off the edge of the image
– need to extrapolate
– methods:
• clip filter (black)
• wrap around
• copy edge
• reflect across edge
Source: S. MarschnerSource: Darrell, Berkeley
![Page 91: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/91.jpg)
Boundary issues
• What about near the edge?
– the filter window falls off the edge of the image
– need to extrapolate
– methods (MATLAB):
• clip filter (black): imfilter(f, g, 0)
• wrap around: imfilter(f, g, ‘circular’)
• copy edge: imfilter(f, g, ‘replicate’)
• reflect across edge: imfilter(f, g, ‘symmetric’)
Source: S. MarschnerSource: Darrell, Berkeley
![Page 92: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/92.jpg)
Borders
From Rick’s bookSource: Torralba, MIT
![Page 93: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/93.jpg)
Today’s topics
• Image Formation
• Image filters in spatial domain– Filter is a mathematical operation of a grid of numbers– Smoothing, sharpening, measuring texture
• Image filters in the frequency domain– Filtering is a way to modify the frequencies of images– Denoising, sampling, image compression
• Templates and Image Pyramids– Filtering is a way to match a template to the image– Detection, coarse-to-fine registration
![Page 94: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/94.jpg)
Why does the Gaussian give a nice smooth image, but the square filter give edgy artifacts?
Gaussian Box filter
Source: James Hays, Brown
![Page 95: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/95.jpg)
Why does a lower resolution image still make sense to us? What do we lose?
Image: http://www.flickr.com/photos/igorms/136916757/ Slide: HoiemSource: James Hays, Brown
![Page 96: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/96.jpg)
Jean Baptiste Joseph Fourier (1768-1830)
had crazy idea (1807):Any univariate function can be rewritten as a weighted sum of sines and cosines of different frequencies.
• Don’t believe it? – Neither did Lagrange,
Laplace, Poisson and other big wigs
– Not translated into English until 1878!
• But it’s (mostly) true!– called Fourier Series
– there are some subtle restrictions
...the manner in which the author arrives at these equations is not exempt of difficulties and...his
analysis to integrate them still leaves something to be desired on the score of generality and even rigour.
Laplace
LagrangeLegendre
Source: James Hays, Brown
![Page 97: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/97.jpg)
A sum of sinesOur building block:
Add enough of them to get any signal f(x) you want!
xAsin(
Source: James Hays, Brown
![Page 98: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/98.jpg)
Filtering in spatial domain-101
-202
-101
* =
Source: James Hays, Brown
![Page 99: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/99.jpg)
Filtering in frequency domain
FFT
FFT
Inverse FFT
=
Slide: HoiemSource: James Hays, Brown
![Page 100: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/100.jpg)
Why does the Gaussian give a nice smooth image, but the square filter give edgy artifacts?
Gaussian Box filter
Filtering
Source: James Hays, Brown
![Page 101: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/101.jpg)
Gaussian
Source: James Hays, Brown
![Page 102: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/102.jpg)
Box Filter
Source: James Hays, Brown
![Page 103: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/103.jpg)
Throw away every other row and column to create a 1/2 size image
Subsampling by a factor of 2
Source: James Hays, Brown
![Page 104: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/104.jpg)
• 1D example (sinewave):
Source: S. Marschner
Aliasing problem
Source: James Hays, Brown
![Page 105: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/105.jpg)
Source: S. Marschner
• 1D example (sinewave):
Aliasing problem
Source: James Hays, Brown
![Page 106: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/106.jpg)
Subsampling without pre-filtering
1/4 (2x zoom) 1/8 (4x zoom)1/2
Slide by Steve SeitzSource: James Hays, Brown
![Page 107: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/107.jpg)
Subsampling with Gaussian pre-filtering
G 1/4 G 1/8Gaussian 1/2
Slide by Steve SeitzSource: James Hays, Brown
![Page 108: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/108.jpg)
Today’s topics
• Image Formation
• Image filters in spatial domain– Filter is a mathematical operation of a grid of numbers– Smoothing, sharpening, measuring texture
• Image filters in the frequency domain– Filtering is a way to modify the frequencies of images– Denoising, sampling, image compression
• Templates and Image Pyramids– Filtering is a way to match a template to the image– Detection, coarse-to-fine registration
![Page 109: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/109.jpg)
Template matching
Scene
Template (mask)
A toy example
Source: Darrell, Berkeley
![Page 110: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/110.jpg)
Template matching
Template
Detected template
Source: Darrell, Berkeley
![Page 111: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/111.jpg)
Template matching
Detected template Correlation map
Source: Darrell, Berkeley
![Page 112: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/112.jpg)
Where’s Waldo?
Scene
Template
Source: Darrell, Berkeley
![Page 113: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/113.jpg)
Where’s Waldo?
Scene
Template
Source: Darrell, Berkeley
![Page 114: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/114.jpg)
Where’s Waldo?
Detected template Correlation map
Source: Darrell, Berkeley
![Page 115: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/115.jpg)
Template matching
Scene
Template
What if the template is not identical to some subimage in the scene?
Source: Darrell, Berkeley
![Page 116: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/116.jpg)
Template matching
Detected template
Template
Match can be meaningful, if scale, orientation, and general appearance is right.
Source: Darrell, Berkeley
![Page 117: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/117.jpg)
Application
Figure from “Computer Vision for Interactive Computer Graphics,” W.Freeman et al, IEEE Computer Graphics and Applications, 1998 copyright 1998, IEEE
![Page 118: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/118.jpg)
Template matching• Goal: find in image
• Main challenge: What is a good similarity or distance measure between two patches?– Correlation
– Zero-mean correlation
– Sum Square Difference
– Normalized Cross Correlation
Slide: HoiemSource: Hays, Brown
![Page 119: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/119.jpg)
Matching with filters• Goal: find in image
• Method 0: filter the image with eye patch
Input Filtered Image
],[],[],[,
lnkmflkgnmhlk
What went wrong?
response is stronger for higher intensity
f = imageg = filter
Slide: HoiemSource: Hays, Brown
![Page 120: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/120.jpg)
Slide: Hoiem
Matching with filters• Goal: find in image
• Method 1: filter the image with zero-mean eye
Input Filtered Image (scaled) Thresholded Image
)],[()],[(],[,
lnkmgflkfnmhlk
True detections
False detections
mean of f
Source: Hays, Brown
![Page 121: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/121.jpg)
Slide: Hoiem
Matching with filters• Goal: find in image
• Method 2: SSD
Input 1- sqrt(SSD) Thresholded Image
2
,
)],[],[(],[ lnkmflkgnmhlk
True detections
Source: Hays, Brown
![Page 122: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/122.jpg)
Matching with filters• Goal: find in image
• Method 2: SSD
Input 1- sqrt(SSD)
2
,
)],[],[(],[ lnkmflkgnmhlk
What’s the potential downside of SSD?
Slide: HoiemSource: Hays, Brown
SSD is sensitive to average intensity
![Page 123: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/123.jpg)
Matching with filters• Goal: find in image
• Method 3: Normalized cross-correlation
5.0
,
2
,
,
2
,
,
)],[()],[(
)],[)(],[(
],[
lk
nm
lk
nm
lk
flnkmfglkg
flnkmfglkg
nmh
Matlab: normxcorr2(template, im)
mean image patchmean template
Slide: HoiemSource: Hays, Brown
![Page 124: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/124.jpg)
Slide: Hoiem
Matching with filters• Goal: find in image
• Method 3: Normalized cross-correlation
Input Normalized X-Correlation Thresholded Image
True detections
Source: Hays, Brown
![Page 125: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/125.jpg)
Matching with filters• Goal: find in image
• Method 3: Normalized cross-correlation
Input Normalized X-Correlation Thresholded Image
True detections
Slide: HoiemSource: Hays, Brown
![Page 126: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/126.jpg)
Q: What is the best method to use?
A: Depends
• SSD: faster, sensitive to overall intensity
• Normalized cross-correlation: slower, invariant to local average intensity and contrast
Source: Hays, Brown
![Page 127: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/127.jpg)
Q: What if we want to find larger or smaller eyes?
![Page 128: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/128.jpg)
adapted from Michael Black, Brown University
A: Image Pyramid
![Page 129: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/129.jpg)
Review of Sampling
Low-Pass Filtered Image
Image
GaussianFilter Sample
Low-Res Image
Slide: HoiemSource: Hays, Brown
![Page 130: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/130.jpg)
Gaussian Pyramid
adapted from Michael Black, Brown University
![Page 131: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/131.jpg)
Gaussian pyramid
Source: ForsythSource: Hays, Brown
![Page 132: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/132.jpg)
Template Matching with Image Pyramids
Input: Image, Template1. Match template at current scale
2. Downsample image
3. Repeat 1-2 until image is very small
4. Take responses above some threshold, perhaps with non-maxima suppression
Slide: HoiemSource: Hays, Brown
![Page 133: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/133.jpg)
Coarse-to-fine Image Registration1. Compute Gaussian pyramid
2. Align with coarse pyramid
3. Successively align with finer pyramids
– Search smaller range
Why is this faster?
Are we guaranteed to get the same result?
Slide: HoiemSource: Hays, Brown
![Page 134: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/134.jpg)
Laplacian filter
Gaussianunit impulse
Laplacian of Gaussian
Source: LazebnikSource: Hays, Brown
![Page 135: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/135.jpg)
Laplacian pyramid
Source: ForsythSource: Hays, Brown
![Page 136: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/136.jpg)
Computing Gaussian/Laplacian Pyramid
http://sepwww.stanford.edu/~morgan/texturematch/paper_html/node3.html
Can we reconstruct the original from the laplacian pyramid?
Source: Hays, Brown
![Page 137: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/137.jpg)
Texture segmentation
Malik & Perona, 1990. Preattentive texture discrimination with early vision mechanisms.
![Page 138: Filters - web.cs.hacettepe.edu.trpinar/courses/CMP719/lectures/filte… · • Image filters in spatial domain –Filter is a mathematical operation of a grid of numbers –Smoothing,](https://reader033.vdocuments.site/reader033/viewer/2022053002/5f0612fc7e708231d4162912/html5/thumbnails/138.jpg)
• Early processing in humans filters for various orientations and scales of frequency
• Perceptual cues in the mid-high frequencies dominate perception
• When we see an image from far away, we are effectively subsampling it
Early Visual Processing: Multi-scale edge and blob filters
Clues from Human Perception
Source: Hays, Brown