nips yomikai 1226

Download Nips yomikai 1226

Post on 20-Jul-2015

1.086 views

Category:

Documents

1 download

Embed Size (px)

TRANSCRIPT

  • An Inverse Power Method for

    Nonlinear

    Eigenproblems with Applications in

    1-Spectral Clustering and Sparse PCA

    NIPS 2010, 1133

    @niam

    12/26

    1

  • p-LaplacianGraph-cutp=1

    PCASparse

    (p=2)IPM/

    1. (p>=1)

    2. IPM

    3. IPMAll proofs had to be omitted due to space restrictions and

    can be found in the supplementary material.

    2

    Algorithm

  • (Power Method)

    A/

    xk+1 =Axk

    xk

    3

  • Courant-Fischer

    4

    (2)

    http://www.math.meiji.ac.jp/~mk/labo/text/generalized-eigenvalue-

    problem/node16.html

    Courant-Fischer

  • 5

    (2) (3)

    A:n x nfRn

    (convex) (even)p-homogeneous

    S(f)=0 f=0Lipschitz

    p

    Gp-homogeneous.

    R,S

    Lipschitz0Rademacher

    vR, v

  • 6

    (2)

    R,Squadratic function(1)

    FF

    R(f)=, S(f)=Bf Cf = 0Bf=CfA=C-1B(1)C-1f!=0=0fS(f)=0f=0

  • (general differential)

    Fnonconvex, nondifferential

    FconvexFF0

    7

  • Theorem 2.2

    8

    fF=R(f)/S(f)critical pointF(f)=0()-()=0, =R(f)/S(f)Af-f=0, =/fAA

    2.2S

  • (Power Method)

    A/

    xk+1 =Axk

    xk

    9

  • Inverse Power Method

    10

    Inverse Power MethodA

    fk

  • Algorithm 1

    11

    2-norm

    necessary to guarantee descent

  • inner optimization problem (p=1)

    12

    inner optimization problems(fk)1-Laplaciansparse PCAp>1

  • 13

    Algorithm 1*

  • Application2: sparse PCA

    14

    XPCAf

    f*sparse1-norm

    IPM

    Application 1p-Laplacian)Application 2PCA)Applicaiton2straight-forward

  • Algorithm 1

    15

    inner optimization problem

  • Algorithm 4: Sparse PCA

    16

    4.inner optimization problem

  • Application 1: p-Laplacian, Graph-

    cut

    Graph laplacian (2-Laplacian)

    ()2

    second eigenvector (2f=0

    p-Laplacian:

    17

    =1/2 fT (D-W) f

    graph Laplacian

  • Cheeger cut

    18

    Vweight matrixWG

    h_RCC

    1-Laplacian

  • median zerosecond eigenvectorh_RCCsecond eigenvector

    Gram-Schmidt

    Algorithm 1median zero

    19

  • Algorithm 1

    20

  • Algorithm 3

    21

    median 0 median

    vs(f)median4.

    RCCp=2Laplacian

  • Graph cut

    22

    1-Laplacian 2-Laplacian

    two-moonsGraph-cut

    2000two-moons100

  • Graph-cut 2

    23

    K=10Graph-cut

  • Sparse PCA

    3Data Setcadinality (

    24

  • Graph-cutPCASparsep=1

    Sparsep=1IPM

    25

  • 26