spectral clustering between friends
DESCRIPTION
spectral clustering between friends. One of these things is not like the other…. spectral clustering (a la Ng-Jordan-Weiss). data. similarity graph. edges have weights w ( i , j ). e.g. the Laplacian. diagonal matrix D. Normalized Laplacian :. energy. Normalized Laplacian :. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/1.jpg)
spectral clustering between friends
![Page 2: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/2.jpg)
One of these things is not like the other…
![Page 3: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/3.jpg)
![Page 4: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/4.jpg)
spectral clustering (a la Ng-Jordan-Weiss)
data similarity graphedges have weights w(i,j)
e.g.
![Page 5: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/5.jpg)
the Laplacian
diagonal matrix D
Normalized Laplacian:
![Page 6: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/6.jpg)
energy
Normalized Laplacian:
![Page 7: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/7.jpg)
spectral embedding
Normalized Laplacian:Compute first k eigenvectors: v1, v2 , …, vk
![Page 8: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/8.jpg)
clustering
Run k–means to cluster the points
![Page 9: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/9.jpg)
spectral clustering
Sidi, et. al. 2011 [TelAviv-SFU]
Many, many variants…
it’s amazing!
it’s mediocre!
it’s antiquated
Many opinions
… what to prove?
![Page 10: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/10.jpg)
why should spectral clustering work?
spectral embedding
k perfect clusters
![Page 11: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/11.jpg)
graph expansion
Expansion: For a subset S µ V, define
E(S) = set of edges with one endpoint in S.
S
![Page 12: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/12.jpg)
graph expansion
Expansion: For a subset S µ V, define
E(S) = set of edges with one endpoint in S.
S1
Theorem [Cheeger70, Alon-Milman85, Sinclair-Jerrum89]: ¸22 · ½G (2) ·
p2̧ 2
½G (k) = minfmaxÁ(Si ) : S1;S2; : : : ;Sk µ V disjointgk-way expansion constant:
S2
S3
S4
“most important result in spectral graph theory” -- Wikipedia
![Page 13: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/13.jpg)
Miclo’s conjecture
Higher-order Cheeger Conjecture [Miclo 08]:
¸k2 · ½G (k) · C(k)
p¸k
for some C(k) depending only on k.
For every graph G and k 2 N, we have
[Lee-OveisGharan-Trevisan 2012]:True with
This bound for C(k) is tight.Algorithm of Ng-Jordan-Weiss works, changing the last step.
S1
S2
S3
S4
![Page 14: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/14.jpg)
the clustering step
Run k–means to cluster the points
we do random projection
random space partition
![Page 15: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/15.jpg)
Miclo’s conjecture
Higher-order Cheeger Conjecture [Miclo 08]:
¸k2 · ½G (k) · C(k)
p¸k
for some C(k) depending only on k.
For every graph G and k 2 N, we have
[Lee-OveisGharan-Trevisan 2012]:True with
This bound for C(k) is tight.Algorithm of Ng-Jordan-Weiss works, changing the last step.
S1
S2
S3
S4
![Page 16: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/16.jpg)
hybrid algorithms
Suppose the data has some nice low-dimensional structure
Spectral embedding could losethat information:Back in a high-dimensional space
![Page 17: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/17.jpg)
hybrid algorithms
Suppose the data has some nice low-dimensional structure
Use spectral embedding distances to deform the data
Do clustering on transformed data set
![Page 18: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/18.jpg)
unraveling the mysteries of complexity
![Page 19: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/19.jpg)
the unique games conjecture
Consider linear equations in two variables, modulo a prime pVariables: x1, x2, …, xn
x12 + x2 = 4x4 – 3 x7 = 1
x9 + 8 x12 = 9…
If there exists a solution that satisfies 99% of the equations,can you find one that satisfies 10%?
Conjectured to be NP-hard [Khot 2002]
![Page 20: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/20.jpg)
a spectral attack
Construct a graph with one vertex for every variable, and anedge whenever two variables occur in the same constraint.
x12 + x2 = 4x4 – 3 x7 = 1
x9 + 8 x12 = 9…A “good” solution to the equations implies a partition of thegraph into p nice clusters!
![Page 21: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/21.jpg)
a spectral attack
Higher-order Cheeger Theorem:For every graph G and k 2 N, we have
S1
S2
S3
S4
Unnecessary for large k:[Arora-Barak-Steurer 2010]
A better asymptotic dependence would disprove the UGC.
![Page 22: spectral clustering between friends](https://reader035.vdocuments.site/reader035/viewer/2022062323/5681645e550346895dd632cf/html5/thumbnails/22.jpg)