geometric diffusions as a tool for harmonic analysis and structure definition of data
DESCRIPTION
The second-round discussion* on. Geometric diffusions as a tool for harmonic analysis and structure definition of data. By R. R. Coifman et al. * The first-round discussion was led by Xuejun; * The third-round discussion is to be led by Nilanjan. Diffusion Maps. Purpose - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/1.jpg)
Geometric diffusions as a tool for harmonic analysis and structure definition of dataBy R. R. Coifman et al.
The second-round discussion* on
* The first-round discussion was led by Xuejun;
* The third-round discussion is to be led by Nilanjan.
![Page 2: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/2.jpg)
Diffusion Maps
• Purpose - finding meaningful structures and geometric descriptions of a data set X.
- dimensionality reduction
• Why?
The high dimensional data is often subject to
a large quantity of constraints (e.g. physical laws)
that reduce the number of degrees of freedom.
![Page 3: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/3.jpg)
• Markov Random Walk
KDxxKxxK
xxpaA
jji
jiijij
1
),(),(
)]|([][
• Symmetric Kernel
ji
jji
iji
jiij a
xxKxxK
xxKa ~
),(),(
),(~
Diffusion Maps
Many works propose to use first few eigenvectors of A as a low representation of data (without rigorous justification).
2/12/1~ ADDA
• Relationship
![Page 4: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/4.jpg)
• Diffusion maps
)(
)(
)( 11
00
x
x
x m
m
m
• Spectral Decomposition of A
Diffusion Maps
n
jninnij xxa )()(~ 2
where 01 210
• Spectral Decomposition of Am
n
jninmn
mij xxa )()(~ 2)(
![Page 5: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/5.jpg)
• Diffusion distance of m-step
2
)()()(,
2
)()(
~2~~)(
jmim
mij
mjj
miijim
xx
aaaxxD
• Interpretation
dzzxazxaxxD jm
im
jim
2)2/()2/(,
2 ),(~),(~)(
Diffusion Distance
The diffusion distance measures the rate of connectivity between xi and xj by paths of length m in the data.
![Page 6: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/6.jpg)
Diffusion vs. Geodesic Distance
),(),( .. BCDBAD geodgeod ),(),( BCDBAD mm
![Page 7: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/7.jpg)
Data Embedding
)(,),(),(001100 xxxxx d
md
mm
• By mapping the original data into (often )
• The diffusion distance can be accurately approximated
dRx 0dRx0dd
)(1),( 22 mm eOyxyxD
![Page 8: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/8.jpg)
Example: curves
Umist face database: 36 pictures (92x112 pixels) of the same person being randomly permuted.
Goal: recover the geometry of the data set.
![Page 9: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/9.jpg)
.,0)cos(
on tosimilar very a graph obtains oneordered,- re isnumbers ofset this When image.each to
numbera real assigns ioneigenfunct second The 1
t
Original ordering
Re-ordering
The natural parameter (angle of the head) is recovered, the data points are re-organized and the structure is identified as a curve with 2 endpoints.
![Page 10: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/10.jpg)
Original set: 1275 images (75x81 pixels) of the word “3D”.
Example: surface
![Page 11: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/11.jpg)
Diffusion Wavelet
• A function f defined on the data admits a multiscale representation of the form:
• Need a method compute and efficiently represent the powers Am.
1
0 iondecompositwavelet
1
scalecoarsest
00
s
s
sss fAAfAf
![Page 12: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/12.jpg)
• Multi-scale analysis of diffusion
Discretize the semi-group {At:t>0} of the powers of A at a logarithmic scale
which satisfy
Diffusion Wavelet
![Page 13: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/13.jpg)
Diffusion Wavelet
![Page 14: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/14.jpg)
• The detail subspaces
• Downsampling, orthogonalization, and operator compression
- diffusion maps: X is the data set
}:{ , jkjj Xk
}:{0 Xkk
A - diffusion operator, G – Gram-Schmidt ortho-normalization, M - AG
![Page 15: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/15.jpg)
Diffusion multi-resolution analysis on the circle. Consider 256 points on the unit circle, starting with 0,k=k and with the standard diffusion. Plot several scaling functions in each approximation space Vj.
![Page 16: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/16.jpg)
Diffusion multi-resolution analysis on the circle. We plot the compressed matrices representing powers of the diffusion operator. Notice the shrinking of the size of the matrices which are being compressed at the different scales.
![Page 17: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/17.jpg)
Multiscale Analysis of MDPs
[1] S. Mahadevan, “Proto-value Functions: Developmental Reinforcement Learning”, ICML05.
[2] S. Mahadevan, M. Maggioni, “Value Function Approximation with Diffusion Wavelets and Laplacian Eigenfunctions”, NIPS05.
[3] M. Maggioni, S. Mahadevan, “Fast Direct Policy Evaluation using Multiscale Analysis of Markov Diffusion Processes”, ICML06.
![Page 18: Geometric diffusions as a tool for harmonic analysis and structure definition of data](https://reader036.vdocuments.site/reader036/viewer/2022062316/56816784550346895ddc9923/html5/thumbnails/18.jpg)
To be discussed a third-round led by Nilanjan
Thanks!