chapter 14 clustering and unsupervised classification classification a. dermanis
TRANSCRIPT
![Page 1: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/1.jpg)
CHAPTER 14CHAPTER 14
Clustering andClustering andUnsupervised ClassificationUnsupervised Classification
CLASSIFICATIONCLASSIFICATION
A. Dermanis
![Page 2: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/2.jpg)
m = x1
N i xi
CT = ST
1
N
i xi
ST = (x – mi)(x – mi)T
xi
Si = (x – mi)(x – mi)T
Ci = Si
1ni
mi = xxi
1ni
Clustering = dividing of N pixels into K classes ω1, ω2, …, ωK Clustering = dividing of N pixels into K classes ω1, ω2, …, ωK
global mean
ClusteringClustering
total scatter matrix:
mean of class ωi :scatter matrix of class ωi :
total covariance matrix:
covariance matrix of class ωi :
A. Dermanis
![Page 3: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/3.jpg)
Sex = ni (mi – m)(mi – m)T i
i xi
Sin = Si = (x – mi)(x – mi)T i
Clustering criteria Clustering criteria
overall compactness of the clusters internal scatter matrix
degree of distinction between the clusters external scatter matrix
Optimal algorithm: Sin = min and Sex = max (simultaneously)
Problem: How many clusters ? (K = ?)
Extreme choice: K = N (each pixel a different class) k = {xk}
Extreme choice: K = 1 (all pixels in a single class) Sin = ST, Sex = 0
ST = Sin + Sex = constantST = Sin + Sex = constant
mk = xk, Sk = 0, Sin = Sk = 0 = min, Sex = ST =maxk
A. Dermanis
![Page 4: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/4.jpg)
F
E
D
C
B
AG
GLO
ME
RA
TIV
E
DIV
ISIV
E
1 2 3 4 5 6A
Hierarchical Clustering
Agglomerative clustering:
Unifying at each step the two closest clusters
Divisive clustering :
Dividing at each step the most disperse cluster into two new clusters
Needed:
Unification criteria.Division criteria and procedures.
A. Dermanis
![Page 5: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/5.jpg)
A B D
F
E
D
C
B
1 2
3
4
5
6
AG
GLO
ME
RA
TIV
E
DIV
ISIV
E
1 2 3 4 5 6A
Hierarchical Clustering
C E F
A. Dermanis
![Page 6: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/6.jpg)
Distance between two clusters (alternatives): Distance between two clusters (alternatives):
i kki nn
d x x
xx ||||1
)()( |||| xxxxxx T
mean distance:
||||min,
min xxxx
ki
d
minimum distance:
||||max,
max xxxx
ki
d
maximum distance:
Used in agglomerative and divisive clustering Used in agglomerative and divisive clustering
A. Dermanis
![Page 7: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/7.jpg)
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8 9
10
11
1213
14
15
16
The K-means or migrating means algorithm The K-means or migrating means algorithm
A. Dermanis
![Page 8: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/8.jpg)
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8 9
10
11
1213
14
15
16
The K-means or migrating means algorithm The K-means or migrating means algorithm
Step 0:
Selection of K = 3 pixels as initial positions of means
A. Dermanis
![Page 9: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/9.jpg)
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8 9
10
11
1213
14
15
16
Step 1:
Assignment each pixels to the clusterof its closest mean
Calculation of the new meansfor each cluster
The K-means or migrating means algorithm The K-means or migrating means algorithm
A. Dermanis
![Page 10: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/10.jpg)
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8 9
10
11
1213
14
15
16
Step 2:
Assignment each pixels to the clusterof its closest mean
Calculation of the new meansfor each cluster
The K-means or migrating means algorithm The K-means or migrating means algorithm
A. Dermanis
![Page 11: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/11.jpg)
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8 9
10
11
1213
14
15
16
Step 3:
Assignment each pixels to the clusterof its closest mean
Calculation of the new meansfor each cluster
The K-means or migrating means algorithm The K-means or migrating means algorithm
A. Dermanis
![Page 12: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/12.jpg)
1 2 3 4 5 6 7 8 9
1
2
3
4
5
6
7
8
9
1
2
3
4
5
6
7
8 9
10
11
1213
14
15
16
Step 4:
Assignment each pixels to the clusterof its closest mean
All pixels remain in the same cluster.Means remain the same.
Termination of the algorithm !
The K-means or migrating means algorithm The K-means or migrating means algorithm
A. Dermanis
![Page 13: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/13.jpg)
The Isodata AlgorithmThe Isodata Algorithm
A variant of the K means algorithm.In each step one of 3 additional procedures can be used:
A variant of the K means algorithm.In each step one of 3 additional procedures can be used:
1. Cluster ELIMINATIONELIMINATION
2. Cluster UNIFICATIONUNIFICATION
3. Cluster DIVISIONDIVISION
Eliminate clusterswith very few pixels
Unify pairs of clustersVery close to each other
Divide large clusterswhich are elongatedInto two clusters
A. Dermanis
![Page 14: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/14.jpg)
The Isodata AlgorithmThe Isodata Algorithm
1. Cluster ELIMINATIONELIMINATION
Eliminate clusterswith very few pixels
A. Dermanis
![Page 15: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/15.jpg)
The Isodata AlgorithmThe Isodata Algorithm
2. Cluster UNIFICATIONUNIFICATION
Unify pairs of clustersVery close to each other
A. Dermanis
![Page 16: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/16.jpg)
The Isodata AlgorithmThe Isodata Algorithm
3. Cluster DIVISIONDIVISION
Divide large clusterswhich are elongatedInto two clusters
A. Dermanis
![Page 17: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/17.jpg)
The unification process The unification process
The division process The division process
The Isodata AlgorithmThe Isodata Algorithm
m2
m1
m2+kσ2
m2–kσ2
A. Dermanis
![Page 18: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/18.jpg)
K-means: 5 classes
K-means: 7 classes K-means: 9 classes
K-means: 3 classes
Examples ofclassifiction usingthe K-mean algorithm
Examples ofclassifiction usingthe K-mean algorithm
A. Dermanis
![Page 19: CHAPTER 14 Clustering and Unsupervised Classification CLASSIFICATION A. Dermanis](https://reader033.vdocuments.site/reader033/viewer/2022061609/56649c745503460f94927fc4/html5/thumbnails/19.jpg)
ISODATA : 3 classes ISODATA : 5 classes
ISODATA : 7 classes ISODATA : 9 classes
Examples ofclassifiction usingthe ISODATA algorithm
Examples ofclassifiction usingthe ISODATA algorithm
A. Dermanis