![Page 1: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/1.jpg)
EE 290A: Generalized Principal Component Analysis
Lecture 5: Generalized Principal Component Analysis
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 1
![Page 2: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/2.jpg)
Last time
GPCA: Problem definition Segmentation of multiple hyperplanes
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 2
![Page 3: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/3.jpg)
Recover subspaces from vanishing polynomial
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 3
![Page 4: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/4.jpg)
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 4
![Page 5: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/5.jpg)
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 5
![Page 6: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/6.jpg)
This Lecture
Segmentation of general subspace arrangements knowing the number of subspaces
Subspace segmentation without knowing the number of subspaces
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 6
![Page 7: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/7.jpg)
An Introductory Example
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 7
![Page 8: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/8.jpg)
Make use of the vanishing polynomials
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 8
![Page 9: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/9.jpg)
Recover Mixture Subspace Models
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 9
![Page 10: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/10.jpg)
Question: How to choose one representative point per subspace? (some loose answers)1. In noise-free case, randomly pick one.2. In noisy case, choose one close to the zero
set of vanishing polynomials. (How?)
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 10
![Page 11: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/11.jpg)
Summary
Using the vanishing polynomials, GPCA converts a CAE problem to a closed-form solution.
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 11
![Page 12: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/12.jpg)
Step 1: Fitting Polynomials
In general, when the dimensions of subspaces are mixed, the set of all K-th degree polynomials that vanish on A becomes more complicated.
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 12
![Page 13: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/13.jpg)
Polynomials may be dependent!
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 13
![Page 14: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/14.jpg)
With the closed-form solution, even when the sample data are noisy, if K and subspace dimensions are known, a complete list of linearly independent vanishing polynomials can be recovered from the (null space of) embedded data matrix!
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 14
![Page 15: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/15.jpg)
Step 2: Polynomial Differentiation
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 15
![Page 16: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/16.jpg)
Step 3: Sample Point Selection Given n sample points from K
subspaces, how to choose one point per subspace to evaluate the orthonormal basis for each subspace?
What is the notion of optimality in choosing the best sample when a set of vanishing polynomials is given (for any algebraic set)?
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 16
![Page 17: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/17.jpg)
In the case of segmenting hyperplanes?
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 17
![Page 18: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/18.jpg)
Draw a random line that does not pass the origin
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 18
![Page 19: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/19.jpg)
Lemma 3.9: For general arrangements We shall choose samples as close to the
zero set as possible (in the presence of noise)1. One shall avoid choosing points based on
P(x), as it is merely an algebraic error, not the geometric distance.
2. One shall discourage choosing points close to the intersection of two ore more subspaces, even when P(x)=0.
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 19
![Page 20: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/20.jpg)
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 20
![Page 21: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/21.jpg)
Estimate the Rest (K-1) Subspaces Polynomial division
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 21
![Page 22: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/22.jpg)
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 22
![Page 23: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/23.jpg)
GPCA without knowing K or d’s Determining K and d’s is
straightforward when subspaces are of equal dimension1. If d is known, project samples to (d+1)-
dim space. The problem becomes hyperplane segmentation.
2. If K is known, project samples to l-dim spaces, while l=1, 2, …, compute k-th order Veronese map until it drops rank.
3. If both K and d are unknown, try all the combinations
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 23
![Page 24: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/24.jpg)
GPCA without knowing K or d’s Determine arrangements of different
dimensions1. If data are noise-free, check the Hilbert
function table.
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 24
![Page 25: EE 290A: Generalized Principal Component Analysis Lecture 5: Generalized Principal Component Analysis Sastry & Yang © Spring, 2011EE 290A, University of](https://reader036.vdocuments.site/reader036/viewer/2022062516/56649d2d5503460f94a04848/html5/thumbnails/25.jpg)
2. When the data are noisy, apply GPCA recursively
Sastry & Yang © Spring, 2011
EE 290A, University of California, Berkeley 25
Please read Section 3.5 for the definition of Effective Dimension