vectorized laplacians for dealing with high-dimensional...

The Power Grid Graph drawn by Derrick Bezanson 2010

Vectorized Laplacians for dealing with high-dimensional data sets

Fan Chung Graham University of California, San Diego

Saturday, August 11, 2012

Singer + Wu 2011, C. + Zhao 2012

Vectorized Laplacians ----- Graph gauge theory

Earlier work (3 papers 1992-1994) with Shlomo Sternberg, Bert Kostanton the vibrational spectra and spins of the molecules.

New directions inorganizing/analyzinghigh-dimensional data sets,with applications in- vector diffusion maps, - graphics, imaging, sensing- resource allocation with dynamic demands


=A5

B={(12345),(54321),(12)(34)}!A5

V a group H!

.u v u = gv for g B,a chosen subset of H! "!

Cayley graphs G = (V,E)Γ


adjacency matrix !

0

0

irreduciblerepresentations

V a group H!

.u v u = gv for g B,a chosen subset of H! "!

Cayley graphs G = (V,E)

Frobenius 1897

Γ


Eigenvalues of the buckyball

Roots of

• with multiplicity 5

(x2 + x − t 2 + t −1)(x3 − tx2 − x2 − t 2x − 3x + t 3 − t 2 + t + 2) = 0


(x2 + x − t 2 −1)(x2 + x − (t +1)2 ) = 0


(x2 + (2t +1)x + t 2 + t −1)(x4 − 3x3 + (−2t 2 + t −1)x2 +

with weight 1 for single bonds and t for double bonds

(3t 2 − 4t + 8)x + t 4 − t 3 + t 2 + 4t − 4) = 0

• with multiplicity 1x − t − 2 = 0

52 + 42 + 32 + 32 +1= 60Saturday, August 11, 2012

The spectrum of a graph

1( ) ( ( ) ( ))y xx

f x f x f yd

! = "#!

Discrete Laplace operator

2

2 ( )jf xx!"! 1( ) ( ){ }j jf x f x

x x+! !" "! !

In a path Pn 1

1

1( ) ( ) ( )2

( ) ( )

{( )

( )}

j j j

j j

f x f x f x

f x f x

+

!

" = ! !

! !

For


acts on Γ = (V ,E), ΔIn a graph

In general, we consider

acts on Δ (V ,X) = f :V → X{ }.

Δ = Δ180×180

Δ = Δnd×nd

V = n

F

F (V ,R) = { f :V → R}

Example 1: Vibrational spectra of molecules

Example 2: Vector Diffusion Maps

(V ,Rd ).

(V ,R3).F

F

geometric datatime based datastatistical data


Vibrations and spins of the Buckyball Noise in the data sets, etc.

Connections: a classical framework for dealing with


= {E Ex: x ∈V}Vector bundle


= {E Ex: x ∈V}

Connection T

Vector bundle


Vector bundle = {

Tx,e :

E

Connection Tfor x V, e =(x,y) E∈ ∈→Ey Ex

Tx,e Ty,e = id

Example 1:

Vibrational spectra of molecules

Spins of the Buckyball

Ex: x ∈V}



= {h :V → R3}

uh(u)

h(v)

(V ,R3)F

the space of displacements

v



Potential energy

Hooke’s law: h :V → R3

uvh(u)h(v)

= ku ,v

u + h(u)− v − h(v) − u − v( )u ,v∑ 2

≈ ku ,v wu ,v ⋅(h(u)− h(v))u ,v∑ 2

where wu ,v =

u − vu − v

,

= ku ,v < h(u)− h(v),Aeu ,v∑ (h(u)− h(v)) >

Ae = wu ,v ⊗wu ,vt


The vibrational spectrum (infrared)of the Buckyball


Vector bundle

Tx,e :

Selection sec s(u)∈ s∈forE

secEEu

Connection T e =(x,y) E

→Ey Ex

Tx,e Ty,e = id

Suppose a group G acts on V as graph automorphisms.

For a G and u V, ∈ ∈ →Eu Eaua :→Eu Eabu a b = ab :

(a ⋅T )x,e = aTa−1x,a−1ea−1

G acts as automorphisms on connections:

for x V,∈

∈

Define QT ,A(s) = (s(x)−Tx,ex,e∑ s(y)) ⋅Ax,e(s(x)−Tx,es(y))

= {E Ex: x ∈V}


a G-invariant connection on defines an

Theorem: C.+Sternberg 1992

E

Γ =

Example 1: Vibrations and spins on the Buckyball

F = Av,beb∈B∑⎛⎝⎜

⎞⎠⎟⊗ I − Av,betb ⊗ r(b)

b∈B∑

tb ∈Auto(X)

QT ,A(s) = (s(x)−Tx,ex,e∑ s(y)) ⋅Ax,e(s(x)−Tx,es(y))

can be represented as

In a homogenous graph G/H with edge generating

= (Γ,X)E

isomorphism , such that the operator

set B,

For the Buckyball G = SL(2,5), PSL(2,5) A5,Isotropy subgroup Z2

G


Examples of connection graphs

data point uOuv



Spins of the buckyball

data point v

rotation

Singer + Wu 2011


Examples of connection Laplacians

data point uOuv


data point v

rotation

Singer + Wu 2011

DataAt a node u,

In a network,

local PCA vector in Rd

dimension reduction


Consistency of connection Laplacians

For a connected connection graph G of dimension d,

For

< f ,L f >= f (u)Ouv − f (v)(i, j )∈E∑ 2

wuv

f :V → Rd



For a connected connection graph G of dimension d, the following statements are equivalent:

The 0 eigenvalue has multiplicity d.

All eigenvalues have multiplicity d.

For each vertex v, we haveOuv =Ou

−1Ov

Ov ∈SO(d),such that for all (u,v)∈E.

For any two vertices u and v, any two paths P, P’, Oxy

xy∈P∏ = Ozw

zw∈P '∏



For a connected connection graph G of dimension d,

if the 0 eigenvalue has multiplicity d, then

we can find

Theorem:

such that Ouv = Ou−1OvOv ∈SO(d),

for all (u,v)∈E.OuMethod 1: For a fixed vertex u, choose a frame

consisting of any d orthonormal vectors.For any v, define Ov = Ovivi+1

i∏ where

is any path joining u and v. u = v0,v1,...,vt

Method 2: Use eigenvectors. Singer+Spielman 2012Saturday, August 11, 2012

Dealing with noise in the data

Random connection Laplacians

PageRank algorithms

Graph limits and graphlets

.........

Graph sparsification algorithms

Several combinatorial approaches


A random graph model for any given expected degree sequence

vj

Pr(Xij = 1) = Pr(vi vj ) = pij

viExample: The Erdos-Renyi model G(n, p)

with pij all equal to . p``

X = Xij( )

Xij = Xji ,1≤ i ≤ j ≤ n.

Independent random indicator variables


A matrix concentration inequality

Theorem: C. + Radcliffe 2011

Xi : independent random Hermitian matrixn × n

Xi − E(Xi ) ≤ M

X = Xii=1

m

∑ satisfies

Pr X − E(X ) > a( ) ≤ 2n exp −a2

2v2 + 2Ma / 3⎛⎝⎜

⎞⎠⎟

where v2 = var(Xii=1

m

∑ )

with

Ahlswede-Winter,Cristofides-Markstrom, Oliveira, Gross, Recht, Tropp,....

Then


Eigenvalues of random graphs with general distributions


G : a random graph, where Pr(vi vj ) = pij .

λi (A)− λi (A) ≤ 4dmax log(2n / ε )for all i, ,if the maximum degree satisfies

adjacency matrix of GA :

expected adjacency matrixA : Aij = pij

With probability , eigenvalues of and satisfyA1− ε

dmax > 49 log( 2nε ).

1≤ i ≤ n

A


Eigenvalues of the normalized Laplacian for random graphs with general distributions


G : a random graph, where Pr(vi vj ) = pij .

λi (L)− λi (L) ≤3log(4n / ε )

δfor all i, ,and

adjacency matrix of GA :expected adjacency matrixA : Aij = pij

δ >10 logn.1≤ i ≤ n

diagonal degree matrix of GDL = I − D−1/2AD−1/2

if the minimum degree satisfiesL = I − D−1/2AD−1/2 ,

for all i, ,if the minimum degree satisfies

With probability , eigenvalues of 1− εsatisfy




PageRank algorithms


.........




Graph sparsification

For a graph G=(V,E,W),

| hG (S) − hH (S) | ≤ εhG (S).

we say H is an ε-sparsifier if for all we have

S ⊆ V

O(n logn)Benczur and Karger 1994 --- sparsifier with size in running time

O(n2 )Spielman and Sriastava 2008 --- using effective resistance running time

O(n)

Batson,Spielman and Sriastava 2009 --- sparsifier running time

O(n)O(nm)


Graph sparsification

Sparsification Sampling edges

| hG (S) − hH (S) | ≤ εhG (S)

e

The cut edges are more likely to be sampled in order to satisfy, for all S ⊆ V ,

We need to rank edges. Saturday, August 11, 2012



PageRank algorithms for ranking edges


.........




Two equivalent ways to define PageRank p=pr(!,s)

(1) (1 )p s pW! != + "

0

(1 ) ( )t t

t

p sW! !"

=

= #$(2)

s = 1 1 1( , ,...., )n n n

Definition of PageRank

the (original) PageRank

some “seed”, e.g.,s = (1,0,....,0)

personalized PageRank

PageRank for ranking vertices

jumping constant

seed


PageRank as generalized Green’s function

Discrete Green’s function Chung+Yau 2000

restricted to

L = D1/2ΔD−1/2 = λkPk

k=1

n−1

∑Laplacian

(ker L )⊥

Green’s function G =

1λk

Pkk=1

n−1

∑

LG = I

The general Green’s function Gβ =

1β + λk

Pkk=1

n−1

∑

PageRank pr(α, s) = βsD−1/2GβD1/2 , β = 2α 1−α


Ranking pages using Green’s function and electrical network theory

: a weighted graphG = (V ,E,W ) wu ,v

vu

injected current functionconductance

1

iV :V →

−1 iE :E→ induced current function

B(e,v) =

1 if v is e 's head

−1 if v is e 's tail0 otherwise

⎧

⎨⎪⎪

⎩⎪⎪

Kirchoff’s current law: iV = iEB

Ohm’s current law: iE = f BTWvoltage potantial function

L = BTWB



iV = fV BTW B = fV L

Kirchoff’s current law: iV = iEB

Ohm’s current law: iE = f BTW

fV = iVD−1/2G D−1/2

R(u,v) = fV (χu − χv )T

= iVD−1/2G D−1/2 (χu − χv )

T

= (χu − χv ) D−1/2G D−1/2 (χu − χv )

T

B ' =D1/2

B⎡

⎣⎢

⎤

⎦⎥

Wβ =βI 0 0 W⎡

⎣⎢

⎤

⎦⎥

Lβ = βI + L

β β

β

LβGβ = Iβ

β

β

prα (s) = sD−1/2GβD

1/2 where β =2α1−α



Rα (u,v) = hα (u,v) + hα (v,u)

=prα ,u (u)du

−prα ,u (v)dv

+prα ,v (v)dv

−prα ,v (u)du

The Green value of (u,v)


Vectorized Laplacian and vectorized PageRank

A connection graph G = (V ,E,O,w)

A connection matrix

A =

wuvOuv if (u,v)∈E,0d×d otherwise.

⎧⎨⎩

fLf T = wuv f (u)Ouv − f (v)

(u ,v)∈E∑ 2

L =D − A D(v,v) = duId×d


Vectorized Laplacian and vectorized PageRank

A connection graph G = (V ,E,O,w)

The transition probability matrix

P =D−1A

p(α,v) = α s + (1−α ) pZ

Z =

I + P2

The vectorized PageRank



Rα (u,v) =pα ,u (u)du

−pα ,u (v)dv

+pα ,v (v)dv

−pα ,v (u)du

The Green value of (u,v)

which the sampling subroutine is based upon.




PageRank algorithms for ranking edges


.........




Examples of connection Laplacians


Example 2: Electron microscopy Vector Diffusion Maps

Spins of the buckyball

Managing millions of images, photos.Example 3: Graphics, vision, ...


Millions of photos in a box!

Navigate using connection graphs.

- finding consistent subgraphs, paths ...


Feature detectionDetect features using SIFT [Lowe, IJCV 2004]


Feature matchingMatch features between each pair of images


Image connectivity graph

(graph layout produced using the Graphviz toolkit: http://www.graphviz.org/)Part of an image connection graphBuilding Rome in a day

Agarwal, Snavely, Simon, Seitz, Szeliski 2009Saturday, August 11, 2012

The Power Grid Graph drawn by Derrick Bezanson 2010

Vectorized Laplacians for dealing with high-dimensional data sets

Fan Chung Graham University of California, San Diego


Thank you.


Image connectivity graph

(graph layout produced using the Graphviz toolkit: http://www.graphviz.org/)Part of an image connection graphBuilding Rome in a day

Agarwal, Snavely, Simon, Seitz, Szeliski 2009Saturday, August 11, 2012

An example of connection graphsBuilding Rome in a day

Agarwal, Snavely, Simon, Seitz, Szeliski 2009


vectorized laplacians for dealing with high-dimensional...

Documents