analysis meets graphs · 2019-02-28 · functional analysis compressive sensing stochastic...

Analysis meets Graphs

Matthew Begue

Norbert Wiener CenterDepartment of Mathematics

University of Maryland, College Park

Why study graphs?

Graph theory has developed into a useful tool in appliedmathematics.Vertices correspond to different sensors, observations, or datapoints. Edges represent connections, similarities, or correlationsamong those points.

Wikipedia graph

Graphs in pure mathematics too!

Why study Analysis?

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Why study Analysis?

Real AnalysisComplex AnalysisPartial Differential EquationsOrdinary Differential EquationsHARMONIC ANALYSISProbabilityDifferential GeometryFunctional AnalysisCompressive SensingStochastic ProcessesMeasure TheoryBrownian MotionOperator TheorySpectral TheoryMathematical ModelingNumerical Analysis

Analysis toolbox

TranslationConvolutionModulationFourier Transform

Analysis toolbox

TranslationConvolutionModulationFourier Transform

Analysis toolbox

TranslationConvolutionModulationFourier Transform

Analysis toolbox

TranslationConvolutionModulationFourier Transform

Outline

1 Fourier Transform

2 Modulation

3 Convolution

4 Translation

Motivation

In the classical setting, the Fourier transform on R is given by

f (ξ) =

∫R

f (t)e−2πiξt dt = 〈f ,e2πiξt〉.

This is precisely the inner product of f with the eigenfunctions ofthe Laplace operator.

Motivation

In the classical setting, the Fourier transform on R is given by

f (ξ) =

∫R

f (t)e−2πiξt dt = 〈f ,e2πiξt〉.

This is precisely the inner product of f with the eigenfunctions ofthe Laplace operator.

Graph Laplacian

DefinitionThe pointwise formulation for the Laplacian acting on a functionf : V → R is

∆f (x) =∑y∼x

f (x)− f (y).

For a finite graph, the Laplacian can be represented as a matrix.Let D denote the N × N degree matrix, D = diag(dx ).Let A denote the N × N adjacency matrix,

A(i , j) =

{1, if xi ∼ xj0, otherwise.

Then the unweighted graph Laplacian can be written as

L = D − A.

Spectrum of the Laplacian

L is a real symmetric matrix and therefore has nonnegativeeigenvalues {λk}N−1

k=0 with associated orthonormal eigenvectors{ϕk}N−1

k=0 .If G is finite and connected, then we have

0 = λ0 < λ1 ≤ λ2 ≤ · · · ≤ λN−1.

Easy to show that ϕ0 ≡ 1/√

N.

Data Sets - Minnesota Road Network

−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943

44

45

46

47

48

49

50

(a) λ3

−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943

44

45

46

47

48

49

50

(b) λ4

−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943

44

45

46

47

48

49

50

(c) λ5

−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943

44

45

46

47

48

49

50

(d) λ6

−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943

44

45

46

47

48

49

50

(e) λ7

−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943

44

45

46

47

48

49

50

(f) λ8

Figure : Eigenfunctions corresponding to the first six nonzero eigenvalues.Minnesota road graph (2642 vertices)

Data Sets - Sierpinski gasket graph approximation

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

(a) λ3

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

(b) λ4

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

(c) λ5

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

(d) λ6

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

(e) λ7

−1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

(f) λ8

Figure : Eigenfunctions corresponding to the first six nonzero eigenvalues.Level-8 graph approximation to Sierpinski gasket (9843 vertices)

Data Sets - Sierpinski gasket graph approximation

−1

−0.5

0

0.5

1

0

0.5

1

1.5

2

−0.02

−0.015

−0.01

−0.005

0

0.005

0.01

0.015

0.02

(a) λ3

−1

−0.5

0

0.5

1

0

0.5

1

1.5

2

−0.02

−0.015

−0.01

−0.005

0

0.005

0.01

(b) λ4

−1

−0.5

0

0.5

1

0

0.5

1

1.5

2

−0.02

−0.015

−0.01

−0.005

0

0.005

0.01

0.015

(c) λ5

−1

−0.5

0

0.5

1

0

0.5

1

1.5

2

−0.02

−0.015

−0.01

−0.005

0

0.005

0.01

0.015

0.02

(d) λ6

−1

−0.5

0

0.5

1

0

0.5

1

1.5

2

−0.02

−0.015

−0.01

−0.005

0

0.005

0.01

0.015

0.02

(e) λ7

−1

−0.5

0

0.5

1

0

0.5

1

1.5

2

−0.02

−0.015

−0.01

−0.005

0

0.005

0.01

0.015

0.02

(f) λ8

Figure : Eigenfunctions corresponding to the first six nonzero eigenvalues.Level-8 graph approximation to Sierpinski gasket (9843 vertices)

Graph Fourier Transform

DefinitionThe graph Fourier transform is defined as

f (λl ) = 〈f , ϕl〉 =N∑

n=1

f (n)ϕ∗l (n).

Notice that the graph Fourier transform is only defined on values ofσ(L).The inverse Fourier transform is then given by

f (n) =N−1∑l=0

f (λl )ϕl (n).

Outline

1 Fourier Transform

2 Modulation

3 Convolution

4 Translation

Graph Modulation

In Euclidean setting, modulation is multiplication of a Laplacianeigenfunction.

DefinitionFor any k = 0,1, ...,N − 1 the graph modulation operator Mk , isdefined as

(Mk f )(n) =√

Nf (n)ϕk (n).

On R,

modulation in the time domain = translation in the frequencydomain

Mξf (ω) = f (ω − ξ).

Example - Classical Setting

Example - Movie

G =SG6f (λl ) = δ2(l) =⇒ f = ϕ2.

−1

−0.5

0

0.5

1

0

0.5

1

1.5

2

−0.01

−0.005

0

0.005

0.01

0.015

0.02

Example - Movie

G =Minnesotaf (λl ) = δ2(l) =⇒ f = ϕ2.

−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943

44

45

46

47

48

49

50

Outline

1 Fourier Transform

2 Modulation

3 Convolution

4 Translation

Graph Convolution - Motivation and Definition

Classically, for signals f ,g ∈ L2(R) we define the convolution as

f ∗ g(t) =

∫R

f (u)g(t − u) du.

However, there is no clear analogue of translation in the graphsetting. So we exploit the property

( f ∗ g)(ξ) = f (ξ)g(ξ),

and then take inverse Fourier transform.

DefinitionFor f ,g : V → R, we define the graph convolution of f and g as

f ∗ g(n) =N−1∑l=0

f (λl )g(λl )ϕl (n).

Outline

1 Fourier Transform

2 Modulation

3 Convolution

4 Translation

Graph Translation

For signal f ∈ L2(R), the translation operator, Tu, can be thoughtof as a convolution with δu.On R we can calculateδu(k) =

∫R δu(x)e−2πikx dx = e−2πiku(= ϕ∗

k (u)).Then by taking the convolution on R we have

(Tuf )(t) = (f∗δu)(t) =

∫R

f (k)δu(k)ϕk (t) dk =

∫R

f (k)ϕ∗k (u)ϕk (t) dk

DefinitionFor any f : V → R the graph translation operator, Ti , is defined to be

(Ti f )(n) =√

N(f ∗ δi )(n) =√

NN−1∑l=0

f (λl )ϕ∗l (i)ϕl (n).

Example - Classical Setting

Translation in the time domain = Modulation in the frequency domain

Example - Movie

G =Minnesotaf = 11

−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943

44

45

46

47

48

49

50

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5 6−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Example - Movie

G =Minnesotaf (λl ) = e−5λl

−98 −97 −96 −95 −94 −93 −92 −91 −90 −89

43

44

45

46

47

48

49

50

−0.1

−0.05

0

0.05

0 1 2 3 4 5 6 7−0.02

0

0.02

0.04

0.06

0.08

0.1

0.12

Example - Movie

G =SG6f (λl ) = e−5λl

−1

−0.5

0

0.5

1

0

0.5

1

1.5

2

−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

0.1

0 1 2 3 4 5 6−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Example - Movie

G =Minnesotaf ≡ 1

−98 −97 −96 −95 −94 −93 −92 −91 −90 −8943

44

45

46

47

48

49

50

−5

0

5

10

0 1 2 3 4 5 6−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Nice Properties of Graph Translation

Theorem

For any f ,g : V → R and i , j ∈ {1,2, ...,N} then1 Ti (f ∗ g) = (Ti f ) ∗ g = f ∗ (Tig).2 TiTj f = TjTi f .

Not-so-nice Properties of Translation operator

The translation operator is not isometric

‖Ti f‖`2 6= ‖f‖`2

In general, the set of translation operators {Ti}Ni=1 do not form a

group like in the classical Euclidean setting.

TiTj 6= Ti+j

In general, Can we even hope for TiTj = Ti•j for some semigroupoperation, • : {1, ...,N} × {1, ...,N} → {1, ...,N}?

Theorem (B. & O.)

Given a graph, G, with eigenvector matrix Φ = [ϕ0| · · · |ϕN−1]. Graphtranslation on G is a semigroup, i.e. TiTj = Ti•j for some semigroupoperator • : {1, ...,N} × {1, ...,N} → {1, ...,N}, only if Φ = (1/

√N)H,

where H is a Hadamard matrix.

Not-so-nice Properties of Translation operator

The translation operator is not isometric

‖Ti f‖`2 6= ‖f‖`2

In general, the set of translation operators {Ti}Ni=1 do not form a

group like in the classical Euclidean setting.

TiTj 6= Ti+j

In general, Can we even hope for TiTj = Ti•j for some semigroupoperation, • : {1, ...,N} × {1, ...,N} → {1, ...,N}?

Theorem (B. & O.)

Given a graph, G, with eigenvector matrix Φ = [ϕ0| · · · |ϕN−1]. Graphtranslation on G is a semigroup, i.e. TiTj = Ti•j for some semigroupoperator • : {1, ...,N} × {1, ...,N} → {1, ...,N}, only if Φ = (1/

√N)H,

where H is a Hadamard matrix.

Not-so-nice Properties of Translation operator

The translation operator is not isometric

‖Ti f‖`2 6= ‖f‖`2

In general, the set of translation operators {Ti}Ni=1 do not form a

group like in the classical Euclidean setting.

TiTj 6= Ti+j

In general, Can we even hope for TiTj = Ti•j for some semigroupoperation, • : {1, ...,N} × {1, ...,N} → {1, ...,N}?

Theorem (B. & O.)

Given a graph, G, with eigenvector matrix Φ = [ϕ0| · · · |ϕN−1]. Graphtranslation on G is a semigroup, i.e. TiTj = Ti•j for some semigroupoperator • : {1, ...,N} × {1, ...,N} → {1, ...,N}, only if Φ = (1/

√N)H,

where H is a Hadamard matrix.

Not-so-nice Properties of Translation operator

The translation operator is not isometric

‖Ti f‖`2 6= ‖f‖`2

In general, the set of translation operators {Ti}Ni=1 do not form a

group like in the classical Euclidean setting.

TiTj 6= Ti+j

In general, Can we even hope for TiTj = Ti•j for some semigroupoperation, • : {1, ...,N} × {1, ...,N} → {1, ...,N}?

Theorem (B. & O.)

Given a graph, G, with eigenvector matrix Φ = [ϕ0| · · · |ϕN−1]. Graphtranslation on G is a semigroup, i.e. TiTj = Ti•j for some semigroupoperator • : {1, ...,N} × {1, ...,N} → {1, ...,N}, only if Φ = (1/

√N)H,

where H is a Hadamard matrix.

Why should we care?

With these time-frequency operators generalized tovertex-frequency operators, we are able to convert many niceresults and theories from harmonic analysis to the graph setting.

Wavelets and Wavelet TransformWindowed/Short-Time Fourier Transform (STFT)Windowed Fourier Frames

Why should we care?

Why should we care?

Thank you!