# nips yomikai 1226

Post on 20-Jul-2015

1.086 views

Category:

## Documents

Embed Size (px)

TRANSCRIPT

• An Inverse Power Method for

Nonlinear

Eigenproblems with Applications in

1-Spectral Clustering and Sparse PCA

NIPS 2010, 1133

@niam

12/26

1

• p-LaplacianGraph-cutp=1

PCASparse

(p=2)IPM/

1. (p>=1)

2. IPM

3. IPMAll proofs had to be omitted due to space restrictions and

can be found in the supplementary material.

2

Algorithm

• (Power Method)

A/

xk+1 =Axk

xk

3

• Courant-Fischer

4

(2)

http://www.math.meiji.ac.jp/~mk/labo/text/generalized-eigenvalue-

problem/node16.html

Courant-Fischer

• 5

(2) (3)

A:n x nfRn

(convex) (even)p-homogeneous

S(f)=0 f=0Lipschitz

p

Gp-homogeneous.

R,S

vR, v

• 6

(2)

FF

R(f)=, S(f)=Bf Cf = 0Bf=CfA=C-1B(1)C-1f!=0=0fS(f)=0f=0

• (general differential)

Fnonconvex, nondifferential

FconvexFF0

7

• Theorem 2.2

8

fF=R(f)/S(f)critical pointF(f)=0()-()=0, =R(f)/S(f)Af-f=0, =/fAA

2.2S

• (Power Method)

A/

xk+1 =Axk

xk

9

• Inverse Power Method

10

Inverse Power MethodA

fk

• Algorithm 1

11

2-norm

necessary to guarantee descent

• inner optimization problem (p=1)

12

inner optimization problems(fk)1-Laplaciansparse PCAp>1

• 13

Algorithm 1*

• Application2: sparse PCA

14

XPCAf

f*sparse1-norm

IPM

Application 1p-Laplacian)Application 2PCA)Applicaiton2straight-forward

• Algorithm 1

15

inner optimization problem

• Algorithm 4: Sparse PCA

16

4.inner optimization problem

• Application 1: p-Laplacian, Graph-

cut

Graph laplacian (2-Laplacian)

()2

second eigenvector (2f=0

p-Laplacian:

17

=1/2 fT (D-W) f

graph Laplacian

• Cheeger cut

18

Vweight matrixWG

h_RCC

1-Laplacian

• median zerosecond eigenvectorh_RCCsecond eigenvector

Gram-Schmidt

Algorithm 1median zero

19

• Algorithm 1

20

• Algorithm 3

21

median 0 median

vs(f)median4.

RCCp=2Laplacian

• Graph cut

22

1-Laplacian 2-Laplacian

two-moonsGraph-cut

2000two-moons100

• Graph-cut 2

23

K=10Graph-cut

• Sparse PCA