filtering images in the spatial domain chapter 3b...
TRANSCRIPT
![Page 1: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/1.jpg)
1
Filtering Images in the Spatial Domain
Chapter 3b G&W Ross Whitaker
(modified by Guido Gerig) School of Computing
University of Utah
![Page 2: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/2.jpg)
2
Overview
• Correlation and convolution • Linear filtering
– Smoothing, kernels, models – Detection – Derivatives
• Nonlinear filtering – Median filtering – Bilateral filtering – Neighborhood statistics and nonlocal filtering
![Page 3: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/3.jpg)
3
. . . .
. . . .
. . . .
. . . .
100 130 104 99 …
87 95 103 150 …
50 36 150 104 …
20 47 205 77 …
. . . .
. . . .
Filter
0.0 0.1 0.0
0.1 0.6 0.1
0.0 0.1 0.0
Input image Output image
Cross Correlation • Operation on image neighborhood and small …
– “mask”, “filter”, “stencil”, “kernel” • Linear operations within a moving window
![Page 4: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/4.jpg)
4
. . . .
. . . .
. . . .
. . . .
100 130 104 99 …
87 95 103 150 …
50 36 150 104 …
20 47 205 77 …
. . . .
. . . .
Filter
0.0 0.1 0.0
0.1 0.6 0.1
0.0 0.1 0.0
Input image Output image
Cross Correlation • Operation on image neighborhood and small …
– “mask”, “filter”, “stencil”, “kernel” • Linear operations within a moving window
![Page 5: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/5.jpg)
5
. . . .
. . . .
. . . .
. . . .
100 130 104 99 …
87 95 103 150 …
50 36 150 104 …
20 47 205 77 …
. . . .
. . . .
Filter
0.0 0.1 0.0
0.1 0.6 0.1
0.0 0.1 0.0
Input image Output image
Cross Correlation • Operation on image neighborhood and small …
– “mask”, “filter”, “stencil”, “kernel” • Linear operations within a moving window
![Page 6: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/6.jpg)
6
. . . .
. . . .
. . . .
. . . .
100 130 104 99 …
87 95 103 150 …
50 36 150 104 …
20 47 205 77 …
. . . .
. . . .
Filter
0.0 0.1 0.0
0.1 0.6 0.1
0.0 0.1 0.0
Input image Output image
0.0*87 + 0.1*95 + 0.0*103
+ 0.1*50 + 0.6*36 + 0.1*150
+ 0.0*20 + 0.1*47 + 0.0*205 = 55.8
Cross Correlation • Operation on image neighborhood and small …
– “mask”, “filter”, “stencil”, “kernel” • Linear operations within a moving window
![Page 7: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/7.jpg)
7
. . . .
. . . .
. . . .
. . . .
100 130 104 99 …
87 95 103 150 …
50 36 150 104 …
20 47 205 77 …
. . . .
. . . .
Filter
0.0 0.1 0.0
0.1 0.6 0.1
0.0 0.1 0.0
Input image Output image
0.0*87 + 0.1*95 + 0.0*103
+ 0.1*50 + 0.6*36 + 0.1*150
+ 0.0*20 + 0.1*47 + 0.0*205 = 55.8
Cross Correlation • Operation on image neighborhood and small …
– “mask”, “filter”, “stencil”, “kernel” • Linear operations within a moving window
55.8
![Page 8: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/8.jpg)
8
. . . .
. . . .
. . . .
. . . .
100 130 104 99 …
87 95 103 150 …
50 36 150 104 …
20 47 205 77 …
. . . .
. . . .
Filter
0.0 0.1 0.0
0.1 0.6 0.1
0.0 0.1 0.0
Input image Output image
0.0*87 + 0.1*95 + 0.0*103
+ 0.1*50 + 0.6*36 + 0.1*150
+ 0.0*20 + 0.1*47 + 0.0*205 = 55.8
Cross Correlation • Operation on image neighborhood and small …
– “mask”, “filter”, “stencil”, “kernel” • Linear operations within a moving window
55.8
![Page 9: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/9.jpg)
9
. . . .
. . . .
. . . .
. . . .
100 130 104 99 …
87 95 103 150 …
50 36 150 104 …
20 47 205 77 …
. . . .
. . . .
Filter
0.0 0.1 0.0
0.1 0.6 0.1
0.0 0.1 0.0
Input image Output image
Cross Correlation • Operation on image neighborhood and small …
– “mask”, “filter”, “stencil”, “kernel” • Linear operations within a moving window
55.8
![Page 10: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/10.jpg)
10
. . . .
. . . .
. . . .
. . . .
100 130 104 99 …
87 95 103 150 …
50 36 150 104 …
20 47 205 77 …
. . . .
. . . .
Filter
0.0 0.1 0.0
0.1 0.6 0.1
0.0 0.1 0.0
Input image Output image
Cross Correlation • Operation on image neighborhood and small …
– “mask”, “filter”, “stencil”, “kernel” • Linear operations within a moving window
![Page 11: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/11.jpg)
11
. . . .
. . . .
. . . .
. . . .
100 130 104 99 …
87 95 103 150 …
50 36 150 104 …
20 47 205 77 …
. . . .
. . . .
Filter
0.0 0.1 0.0
0.1 0.6 0.1
0.0 0.1 0.0
Input image Output image
Cross Correlation • Operation on image neighborhood and small …
– “mask”, “filter”, “stencil”, “kernel” • Linear operations within a moving window
![Page 12: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/12.jpg)
12
0.0*95 + 0.1*103 + 0.0*150
+ 0.1*36 + 0.6*150 + 0.1*104
+ 0.0*47 + 0.1*205 + 0.0*77 = 134.8
. . . .
. . . .
. . . .
. . . .
100 130 104 99 …
87 95 103 150 …
50 36 150 104 …
20 47 205 77 …
. . . .
. . . .
Filter
0.0 0.1 0.0
0.1 0.6 0.1
0.0 0.1 0.0
Input image Output image
Cross Correlation • Operation on image neighborhood and small …
– “mask”, “filter”, “stencil”, “kernel” • Linear operations within a moving window
![Page 13: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/13.jpg)
13
0.0*95 + 0.1*103 + 0.0*150
+ 0.1*36 + 0.6*150 + 0.1*104
+ 0.0*47 + 0.1*205 + 0.0*77 = 134.8
. . . .
. . . .
. . . .
. . . .
100 130 104 99 …
87 95 103 150 …
50 36 150 104 …
20 47 205 77 …
. . . .
. . . .
Filter
0.0 0.1 0.0
0.1 0.6 0.1
0.0 0.1 0.0
Input image Output image
Cross Correlation • Operation on image neighborhood and small …
– “mask”, “filter”, “stencil”, “kernel” • Linear operations within a moving window
134.8
![Page 14: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/14.jpg)
14
Cross Correlation
• 1D
• 2D
![Page 15: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/15.jpg)
15
Correlation: Technical Details
• How to filter boundary?
? ? ? ? ?
?
?
?
?
![Page 16: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/16.jpg)
16
Correlation: Technical Details • Boundary conditions
– Boundary not filtered (keep it 0) – Pad image with amount (a,b)
• Constant value or repeat edge values – Cyclical boundary conditions
• Wrap or mirroring
![Page 17: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/17.jpg)
17
Correlation: Technical Details
• Boundaries – Can also modify kernel – no longer
correlation • For analysis
– Image domains infinite – Data compact (goes to zero far away from
origin)
![Page 18: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/18.jpg)
18
Correlation: Properties
• Shift invariant
![Page 19: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/19.jpg)
19
Correlation: Properties
• Shift invariant
• Linear
Compact notation
![Page 20: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/20.jpg)
20
Filters: Considerations
• Normalize – Sums to one – Sums to zero (some cases, see later)
• Symmetry – Left, right, up, down – Rotational
• Special case: auto correlation
![Page 21: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/21.jpg)
21
0 0 0
0 1 0
0 0 0
Examples 1
![Page 22: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/22.jpg)
22
0 0 0
0 1 0
0 0 0
Examples 1
![Page 23: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/23.jpg)
23
0 0 0
0 1 0
0 0 0
1 1 1
1 1 1
1 1 1
1/9 *
Examples 1
![Page 24: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/24.jpg)
24
0 0 0
0 1 0
0 0 0
1 1 1
1 1 1
1 1 1
1/9 *
Examples 1
![Page 25: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/25.jpg)
25
1 1 1
1 1 1
1 1 1
1/9 *
Examples 1
![Page 26: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/26.jpg)
26
1 1 1
1 1 1
1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1/9 *
1/25 *
Examples 2
![Page 27: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/27.jpg)
27
Smoothing and Noise Noisy image 5x5 box filter
![Page 28: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/28.jpg)
28
Noise Analysis • Consider an a simple image I() with additive,
uncorrelated, zero-mean noise of variance s • What is the expected rms error of the
corrupted image? • If we process the image with a box filter of
size 2a+1 what is the expected error of the filtered image?
![Page 29: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/29.jpg)
30
Other Filters
• Disk – Circularly symmetric, jagged in discrete
case • Gaussians
– Circularly symmetric, smooth for large enough stdev
– Must normalize in order to sum to one • Derivatives – discrete/finite differences
– Operators
![Page 30: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/30.jpg)
Gaussian Kernel
31
![Page 31: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/31.jpg)
Gaussian Kernel
32
Normalization to 1.0
![Page 32: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/32.jpg)
Box versus Gaussian
33
![Page 33: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/33.jpg)
34
• Discrete
• Continuous
• Same as cross correlation with kernel transposed around each axis
• The two operations (correlation and convolution) are the same if the kernel is symmetric about axes
Convolution
reflection of w
Java demo: http://www.jhu.edu/signals/convolve/ http://www.jhu.edu/signals/discreteconv2/index.html
![Page 34: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/34.jpg)
35
Convolution: Properties
• Shift invariant, linear • Commutative
• Associative
• Others (discussed later): – Derivatives, convolution theorem, spectrum…
![Page 35: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/35.jpg)
36
Computing Convolution
![Page 36: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/36.jpg)
37
Computing Convolution
• Compute time – MxM mask – NxN image
![Page 37: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/37.jpg)
38
Computing Convolution
• Compute time – MxM mask – NxN image
O(M2N2) “for” loops are nested 4 deep
![Page 38: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/38.jpg)
39
Computing Convolution
• Compute time – MxM mask – NxN image
• Special case: separable
O(M2N2) “for” loops are nested 4 deep
O(M2N2) O(MN2)
Two 1D kernels
= *
![Page 39: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/39.jpg)
40
Separable Kernels
• Examples – Box/rectangle – Bilinear interpolation – Combinations of
partial derivatives • d2f/dxdy
– Gaussian • Only filter that is both
circularly symmetric and separable
• Counter examples – Disk – Cone – Pyramid
![Page 40: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/40.jpg)
41
Separability
![Page 41: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/41.jpg)
Digital Images: Boundaries are “Lines” or “Discontinuities”
Example: Characterization of discontinuities? 42 Source: http://web.media.mit.edu/~maov/classes/vision09/lect/09_Image_Filtering_Edge_Detection_09.pdf
![Page 42: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/42.jpg)
Digital Images: Boundaries are “Lines” or “Discontinuities”
Example: Characterization of discontinuities? 43 Source: http://web.media.mit.edu/~maov/classes/vision09/lect/09_Image_Filtering_Edge_Detection_09.pdf
![Page 43: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/43.jpg)
44 Source: http://web.media.mit.edu/~maov/classes/vision09/lect/09_Image_Filtering_Edge_Detection_09.pdf
![Page 44: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/44.jpg)
45 Source: http://web.media.mit.edu/~maov/classes/vision09/lect/09_Image_Filtering_Edge_Detection_09.pdf
![Page 45: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/45.jpg)
46
Derivatives: Finite Differences
![Page 46: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/46.jpg)
47
0 -1 0
0 0 0
0 1 0
0 0 0
-1 0 1
0 0 0
Derivative Example
![Page 47: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/47.jpg)
48 Source: http://web.media.mit.edu/~maov/classes/vision09/lect/09_Image_Filtering_Edge_Detection_09.pdf
![Page 48: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/48.jpg)
49 Source: http://web.media.mit.edu/~maov/classes/vision09/lect/09_Image_Filtering_Edge_Detection_09.pdf
![Page 49: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/49.jpg)
50 Source: http://web.media.mit.edu/~maov/classes/vision09/lect/09_Image_Filtering_Edge_Detection_09.pdf
![Page 50: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/50.jpg)
51
Pattern Matching
![Page 51: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/51.jpg)
52
Pattern Matching/Detection
• The optimal (highest) response from a filter is the autocorrelation evaluated at position zero
• A filter responds best when it matches a pattern that looks itself
• Strategy – Detect objects in images by correlation with
“matched” filter
![Page 52: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/52.jpg)
53
Matched Filter Example
![Page 53: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/53.jpg)
54
Matched Filter Example: Correlation of template with image
![Page 54: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/54.jpg)
55
Matched Filter Example: Thresholding of correlation results
![Page 55: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/55.jpg)
56
Matched Filter Example: High correlation → template found
![Page 56: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/56.jpg)
Summary of 9/15
57
• Spatial Filtering – Consider neighborhood information – Special consideration at boundary
? ? ? ? ?
?
?
?
?
![Page 57: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/57.jpg)
Summary of 9/15
58
• Common Filters
1 1 1
1 1 1
1 1 1
1/9 *
Box Gaussian Derivative
![Page 58: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/58.jpg)
Box versus Gaussian
59
![Page 59: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/59.jpg)
Gaussian Filtering
60
![Page 60: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/60.jpg)
Gaussian Filtering
61
![Page 61: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/61.jpg)
62 Source: http://web.media.mit.edu/~maov/classes/vision09/lect/09_Image_Filtering_Edge_Detection_09.pdf
![Page 62: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/62.jpg)
Cross-correlation and Convolution
63
• Cross-correlation
• Convolution
![Page 63: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/63.jpg)
Key Concepts
64
• Separability
O(M2N2) O(MN2) =
![Page 64: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/64.jpg)
65
Nonlinear Methods For Filtering
• Median filtering • Bilateral filtering • Neighborhood statistics and nonlocal
filtering
![Page 65: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/65.jpg)
66
Median Filtering
• For each neighborhood in image – Sliding window – Usually odd size (symmetric) 5x5, 7x7,…
• Sort the greyscale values • Set the center pixel to the median • Important: use “Jacobi” updates
– Separate input and output buffers – All statistics on the original image old new
![Page 66: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/66.jpg)
67
Median vs Gaussian
Original + Gaussian
Noise
3x3 Box 3x3 Median
![Page 67: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/67.jpg)
68
Median Filter
• Issues – Boundaries
• Compute on pixels that fall within window – Computational efficiency
• What is the best algorithm?
• Properties – Removes outliers (replacement noise – salt and
pepper) – Window size controls size of structures – Preserves straight edges, but rounds corners and
features
![Page 68: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/68.jpg)
69
Replacement Noise • Also: “shot noise”, “salt&pepper” • Replace certain % of pixels with samples from pdf • Best filtering strategy: filter to avoid outliers
![Page 69: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/69.jpg)
70
Smoothing of S&P Noise • It’s not zero mean (locally) • Averaging produces local biases
![Page 70: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/70.jpg)
71
Smoothing of S&P Noise • It’s not zero mean (locally) • Averaging produces local biases
![Page 71: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/71.jpg)
72
Median Filtering
Median 5x5 Median 3x3
![Page 72: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/72.jpg)
73
Median Filtering
Median 5x5 Median 3x3
![Page 73: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/73.jpg)
74
Median Filtering • Iterate
Median 3x3 2x Median 3x3
![Page 74: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/74.jpg)
75
Median Filtering
• Image model: piecewise constant (flat)
![Page 75: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/75.jpg)
76
Median Filtering
• Image model: piecewise constant (flat)
Ordering
Output
![Page 76: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/76.jpg)
77
Median Filtering
• Image model: piecewise constant (flat)
![Page 77: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/77.jpg)
78
Median Filtering
• Image model: piecewise constant (flat)
Ordering
Output
![Page 78: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/78.jpg)
79
Order Statistics • Median is special case of order-statistics filters • Instead of weights based on neighborhoods, weights
are based on ordering of data Neighborhood Ordering
Filter
Neighborhood average (box) Median filter
Trimmed average (outlier removal)
![Page 79: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/79.jpg)
80 Source: http://web.media.mit.edu/~maov/classes/vision09/lect/09_Image_Filtering_Edge_Detection_09.pdf
![Page 80: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/80.jpg)
81
Piecewise Flat Image Models
• Image piecewise flat -> average only within similar regions
• Problem: don’t know region boundaries
![Page 81: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/81.jpg)
82
Piecewise-Flat Image Models
• Assign probabilities to other pixels in the image belonging to the same region
• Two considerations – Distance: far away pixels are less likely to
be same region – Intensity: pixels with different intensities
are less likely to be same region
![Page 82: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/82.jpg)
Gaussian: Blur Comes from Averaging across Edges
*
*
*
input output
Same Gaussian kernel everywhere. Source: http://people.csail.mit.edu/sparis/bf_course/slides/03_definition_bf.ppt
![Page 83: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/83.jpg)
Bilateral Filter No Averaging across Edges
*
*
*
input output
The kernel shape depends on the image content.
[Aurich 95, Smith 97, Tomasi 98]
Source: http://people.csail.mit.edu/sparis/bf_course/slides/03_definition_bf.ppt
![Page 84: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/84.jpg)
85
Main Idea
Distance (kernel/pdf) Distance (pdf)
Prob pixel belongs to same region as i
Prob pixel belongs to same region as i
intensity position
![Page 85: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/85.jpg)
Bilateral Filter Definition: an Additional Edge Term
Source: http://people.csail.mit.edu/sparis/bf_course/slides/03_definition_bf.ppt
space weight
not new
range weight
new
normalization factor
new
S
IIIGGW
IBFq
qqp
p
p qp ||||||1
][rs
Same idea: weighted average of pixels.
![Page 86: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/86.jpg)
Illustration a 1D Image
• 1D image = line of pixels
• Better visualized as a plot
pixel
intensity
pixel position
Source: http://people.csail.mit.edu/sparis/bf_course/slides/03_definition_bf.ppt
![Page 87: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/87.jpg)
space
Gaussian Blur and Bilateral Filter Gaussian blur
space
p
q
S
IGIGBq
qp qp ||||][
Source: http://people.csail.mit.edu/sparis/bf_course/slides/03_definition_bf.ppt
![Page 88: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/88.jpg)
space
Gaussian Blur and Bilateral Filter
space range normalization
Gaussian blur
S
IIIGGW
IBFq
qqp
p
p qp ||||||1
][rs
Bilateral filter [Aurich 95, Smith 97, Tomasi 98]
space
space range
p
p
q
q
S
IGIGBq
qp qp ||||][
Source: http://people.csail.mit.edu/sparis/bf_course/slides/03_definition_bf.ppt
![Page 89: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/89.jpg)
90
• Neighborhood – sliding window • Weight contribution of neighbors according to:
• G is a Gaussian (or lowpass), as is H, N is neighborhood, – Often use G(rij) where rij is distance between pixels – Update must be normalized for the samples used in this
(particular) summation • Spatial Gaussian with extra weighting for intensity
– Weighted average in neighborhood with downgrading of intensity outliers
Bilateral Filter
Tomasi, Manduchi: http://en.wikipedia.org/wiki/Bilateral_filter http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/MANDUCHI1/Bilateral_Filtering.html
normalization: all weights add up to 1
![Page 90: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/90.jpg)
Bilateral Filter
91
When the bilateral filter is centered, say, on a pixel on the bright side of the boundary, the similarity function s assumes values close to one for pixels on the same side, and values close to zero for pixels on the dark side. The similarity function is shown in figure 1(b) for a 23x23 filter support centered two pixels to the right of the step in figure 1(a).
Replaces the pixel value at x with an average of similar and nearby pixel values.
![Page 91: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/91.jpg)
92
Bilateral Filtering
Bilateral Gaussian Blurring
Replaces the pixel value at x with an average of similar and nearby pixel values.
![Page 92: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/92.jpg)
93
Bilateral Filtering
Bilateral Gaussian Blurring
![Page 93: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/93.jpg)
94
Nonlocal Averaging
• Recent algorithm – NL-means, Baudes et al., 2005 – UINTA, Awate & Whitaker, 2005
• Different model – No need for piecewise-flat – Images consist of some set of pixels with similar
neighborhoods → average several of those • Scattered around
– General area of a pixel – All around
• Idea – Average sets of pixels with similar neighborhoods
![Page 94: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/94.jpg)
UINTA: Unsupervised Information-Theoretic Adaptive Filtering : Excellent Introduction and
Additional Readings (Suyash P. Awate)
95
http://www.cs.utah.edu/~suyash/pubs/uinta/
Suyash P. Awate, Ross T. Whitaker Unsupervised, Information-Theoretic, Adaptive Image Filtering with Applications to Image Restoration IEEE Trans. Pattern Analysis & Machine Intelligence (TPAMI) 2006, Vol. 28, Num. 3, pp. 364-376
![Page 95: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/95.jpg)
96
Nonlocal Averaging
• Strategy: – Average pixels to alleviate noise – Combine pixels with similar neighborhoods
• Formulation – ni,j – vector of pixels values, indexed by j,
from neighborhood around pixel i
= – vector
![Page 96: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/96.jpg)
97
Nonlocal Averaging Formulation
• Distance between neighborhoods
• Kernel weights based on distances
• Pixel values of k neighborhoods: fk
22
2,
)( ,,
kid
edKw kiki
![Page 97: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/97.jpg)
98
Averaging Pixels Based on Weights
• For each pixel, i, choose a set of pixel locations k: – k = 1, …., M – Average them together based on
neighborhood weights (prop. to intensity pattern difference)
k
M
k
kiw
i fwg M
k ki
1
,1
1 ,
![Page 98: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/98.jpg)
99
Nonlocal Averaging
![Page 99: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/99.jpg)
100
Some Details
• Window sizes: good range is 5x5->11x11
• How to choose samples: – Random samples from around the image
• UINTA, Awate&Whitaker – Block around pixel (bigger than window,
e.g. 51x51) • NL-means
• Iterate – UNITA: smaller updates and iterate
![Page 100: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/100.jpg)
101
NL-Means Algorithm
• For each pixel, p – Loop over set of pixels nearby – Compare the neighorhoods of those pixels
to the neighborhood of p and construct a set of weights
– Replace the value of p with a weighted combination of values of other pixels
• Repeat… but 1 iteration is pretty good
![Page 101: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/101.jpg)
102
Results
Noisy image (range 0.0-1.0) Bilateral filter (3.0, 0.1)
![Page 102: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/102.jpg)
103
Results
Bilateral filter (3.0, 0.1) NL means (7, 31, 1.0)
![Page 103: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/103.jpg)
104
Results
Bilateral filter (3.0, 0.1) NL means (7, 31, 1.0)
![Page 104: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/104.jpg)
105
Less Noisy Example
![Page 105: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/105.jpg)
106
Less Noisy Example
![Page 106: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/106.jpg)
107
Results
Original Noisy Filtered
![Page 107: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/107.jpg)
108
Checkerboard With Noise
Original Noisy Filtered
![Page 108: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/108.jpg)
109
Quality of Denoising
• , joint entropy, and RMS- error vs. number of iterations
![Page 109: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/109.jpg)
110
MRI Head
![Page 110: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/110.jpg)
111
MRI Head
![Page 111: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/111.jpg)
112
Fingerprint
![Page 112: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/112.jpg)
113
Fingerprint
![Page 113: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/113.jpg)
114
Results
Original Noisy Filtered
![Page 114: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/114.jpg)
115
Results
Original Noisy Filtered
![Page 115: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/115.jpg)
116
Results
Original Noisy Filtered
![Page 116: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/116.jpg)
117
Fractal
Original Noisy Filtered
![Page 117: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/117.jpg)
118
Piecewise Constant • Several 10s of Iterations • Tends to obliterate rare events (e.g. corners)
![Page 118: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/118.jpg)
119
Texture, Structure
![Page 119: Filtering Images in the Spatial Domain Chapter 3b G&Wgerig/CS6640-F2014/Materials/spatial_filtering_GG.pdfFiltering Images in the Spatial Domain Chapter 3b G&W Ross Whitaker (modified](https://reader035.vdocuments.site/reader035/viewer/2022062505/5eda1bdfb3745412b570c7c2/html5/thumbnails/119.jpg)
120