image analysis lecture 9.1 -segmentation · segmentation image segmentation is the process of...
TRANSCRIPT
![Page 1: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/1.jpg)
Image AnalysisLecture 9.1 - Segmentation
Idar Dyrdal
![Page 2: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/2.jpg)
SegmentationImage segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong together
The goal is to divide the image into meaningful and/or perceptually uniform regions
Segmentation is typically used to locate objects and boundaries of physical entities in the scene
The segmentation process utilize available image information (intensity, color, texture, pixel position, …).
2
![Page 3: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/3.jpg)
Segmentation (2)
First step in image analysis:Going from pixels to objects or object parts (physical items or scene elements)Paves the way for object feature extraction followed byObject recognition (Classification)
Principles:ThresholdingEdge basedRegion basedAutomatic (supervised) or interactive (unsupervised)
3
![Page 4: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/4.jpg)
Color based segmentation - three categories
4Original image Segmented image
![Page 5: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/5.jpg)
Semantic Segmentation (meaningful regions)
Road
Building
Sky
Building
5
![Page 6: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/6.jpg)
Segmentation methods
Active contours (Snakes, Scissors, Level Sets)Split and merge (Watershed, Divisive & agglomerative clustering, Graph-based segmentation)Gray level thresholdingK-means (parametric clustering)Mean shift (non-parametric clustering)Normalized cutsGraph cuts
6
Supervised color based segmentation (region growing)
![Page 7: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/7.jpg)
Segmentation by thresholding
7
Number of pixels
Otsu’s method:• Automatic clustering based thresholding• Minimization of intra-class variance• Analog to Fisher’s Discriminant Analysis
Gray level
![Page 8: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/8.jpg)
Thresholding with Otsu’s method
8
3 thresholds
4 classes
![Page 9: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/9.jpg)
Binary segmentation – foreground vs. background
9
Threshold between two populations Threshold at given percentile
Number of pixels Number of pixels
Gray level Gray level
ForegroundBackground
Background
Foreground
![Page 10: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/10.jpg)
Binary thresholding – Object detection
10
Thermal image Thresholded image (Otsu’s method)
Global threshold selection threshold too low for detection of the object of interest
![Page 11: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/11.jpg)
Manual thresholding
11
Medium threshold High threshold
![Page 12: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/12.jpg)
Local thresholding
12
Threshold computed from gray level statistics in selected window (Otsu’s method)
![Page 13: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/13.jpg)
Local thresholding using edge information
13
Edge image (Canny edge detectorapplied to selected window)
Thresholded window
Threshold = average gray level along edges
![Page 14: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/14.jpg)
Object detection in video sequences (visible light)
14
Change detection
Absolute difference image (Current image - time averaged background image)
Thresholding of difference image, i.e. Otsu’s method
Requires fixed camera (or registration of images)
Daylight video frame Thresholded difference image
![Page 15: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/15.jpg)
Segmentation by clustering
15
Original image
Pixels represented as pointsin feature space
Segmented image
![Page 16: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/16.jpg)
K-means (parametric) clustering
1. Select K points (for example randomly) as initial cluster centers
2. Assign each sample to nearest cluster center
3. Compute new cluster centers (i.e. sample means)
4. Repeat steps 2 and 3 until no further re-assignments are possible.
16
Unlabeled dataset
![Page 17: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/17.jpg)
K-means clustering
Initial cluster centers (red, green and blue points) Samples assigned to nearest cluster center
17
![Page 18: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/18.jpg)
K-means clustering
Re-computed cluster centers Samples re-assigned to new cluster centers
18
![Page 19: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/19.jpg)
K-means clustering
Re-computed cluster centers Final clustering
19
![Page 20: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/20.jpg)
K-means clustering using color
20
Original image Clustered image – 10 clusters
![Page 21: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/21.jpg)
Mean shift (non-parametric) segmentation
Segmentation by clustering of the pixels in the image (e.g. using color and position)Non-parametric method (using the so called Parzen window technique) to find modes (i.e. peaks) in the density functionAll pixels climbing to the same peak are assigned to the same region.
(Szeliski: Computer Vision – Algorithms and Applications)
21
![Page 22: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/22.jpg)
Mean shift segmentation
22
Original imagea
b
Plot of a vs. b for each pixel in Lab transformed image
![Page 23: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/23.jpg)
Parzen Method
23
Example:
Window (kernel) function:
Density estimate (smoothing of point cloud):
![Page 24: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/24.jpg)
Mean shift segmentation
24
Gradient ascent (hill climbing)
Labeled point cloud Segmented image
![Page 25: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/25.jpg)
Mean Shift Segmentation - example
25Original image Segmented in five categories
![Page 26: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/26.jpg)
Active contours
(Szeliski: Computer Vision – Algorithms and Applications)
26
Fitting of curves to object boundaries:Snakes (fitting of spline curves to strong edges)Intelligent scissors (interactive specification of curves clinging to object boundaries)Level set techniques (evolving boundaries as the zero set of a characteristic function).
These methods iteratively move towards a final solution.
![Page 27: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/27.jpg)
Active Contours - example
27
Original image Segmented image
![Page 28: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/28.jpg)
Split and merge methods
Principles:Region based methodsRecursive splitting of the image based on region statisticsHierarchical merging of pixels and regionsCombined splitting and merging
Methods:Watershed segmentationRegion splitting (divisive clustering)Region merging (agglomerative clustering)Graph-based segmentation
(Szeliski: Computer Vision – Algorithms and Applications)
28
![Page 29: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/29.jpg)
Agglomerative clustering
29
Dendrogram
Dis
tanc
e m
easu
re
Distance measures
![Page 30: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/30.jpg)
Normalized cuts
Separation of groups with weak affinities (similarities) between nearby pixels
(Szeliski: Computer Vision – Algorithms and Applications)
30
![Page 31: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/31.jpg)
Graph cuts
(Szeliski: Computer Vision – Algorithms and Applications)
31
Energy-based methods for binary segmentation:
Grouping of pixels with similar statisticsMinimization of pixel-based energy functionRegion-based and boundary-based energy termsImage represented as a graphCutting of weak edges, i.e. low similarity between corresponding pixels.
![Page 32: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/32.jpg)
Graph cuts - example
32
Original image Segmented image
![Page 33: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/33.jpg)
Morphological operations
Non-linear filteringTypically used to clean up binary imagesErosion: replace pixel value with minimum in local neighborhoodDilation: replace pixel value with maximum in local neighborhoodStructuring element used to define the local neighborhood:
A shape (in blue) and its morphological dilation (in green) and erosion (in yellow) by a diamond-shaped structuring element.
33
(Renato Keshet 2008)
![Page 34: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/34.jpg)
Morphological operations - Erosion
34
Structuring element (disk shaped)
"1"
"0"
"1"
![Page 35: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/35.jpg)
Morphological operations - Dilation
35
Structuring element (disk shaped)
![Page 36: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/36.jpg)
Opening = Erosion + Dilation
36
![Page 37: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/37.jpg)
Closing = Dilation + Erosion
37
![Page 38: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/38.jpg)
Opening - example
38
Segmented image (Active Contours) Result of opening
![Page 39: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/39.jpg)
Closing - example
39
Segmented image Result of closing
![Page 40: Image Analysis Lecture 9.1 -Segmentation · Segmentation Image segmentation is the process of partitioning a digital image into multiple parts, i.e. find groups of pixels that belong](https://reader033.vdocuments.site/reader033/viewer/2022051606/602a96c6e1b4797ccb000350/html5/thumbnails/40.jpg)
Summary
Image Segmentation:Thresholding techniquesClustering methods for segmentationMorphological operations
More information:Szeliski 3.3.2 and 5.1 - 5.5
40