Xianbiao Shu1, Fatih Porikli2, Narendra Ahuja1
1xshu2,[email protected] [email protected]
1 2
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Introduction
• Low-rank recovery on Large-Scale Data and its Vision Applications
• Problem: to recover low-rank matrix from its corrupted observation
CVPR2014 2
Image UnderstandingClustering
classificationRecognition
Video Surveillancedenoising, compression background subtraction tracking, saliency alarm
Imagingcompressive
sensing[Shu11] dynamical MRI
Camera Registrationcamera calibration video stabilization
3D reconstruction[Mobahi11]
20 Miss Korean Contestants only 6 principal components [Huang14]
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Overview of Existing Methods
• Robust Principal Component Analysis (RPCA) [Candes11]
– State-of-the-art: convex, global minima guaranteed.
– Cubic complexity: , due to multiple rounds of SVD
– Running time: >300s on a small video clip (size:160x128, 1060 frames)
• Its accelerated methods
– Partial RPCA [Lin09]: only computes major singular values determine ?
– RP_RPCA [Mu11]: random projection , and minimize unstable
– GoDec [Zhou11]: uses bilateral random projection slow convergence
• Matrix factorization methods
– RMF[Ke05], LMaFit [Shen11] require exact rank estimate
• Efficient, stable, automatic(without requiring rank estimate) method?
CVPR2014 3
n
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Robust Orthonormal Subspace Learning(ROSL)
• Orthonormal Subspace Decomposition ,
– Rank initial subspace dimension
• New rank measures: given ,
number of nonzero rows
sum of magnitude of rows
• Problem formulation
Fast sparse coding at quadratic complexity .
Subspace dimension shrinks from to no requiring rank estimate
Non-convex optimization,
CVPR2014 4
rn k
m
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
• ROSL replaces nuclear norm in RPCA by a new rank measure:
row-1 group sparsity under orthonormal subspace.
• Thus, ROSL shares the same global minima as the performance-
guaranteed RPCA.
Given a matrix A , define conventional and new rank measures
respectively as nuclear norm and row-1 group sparsity under
orthonormal subspace:
Then
holds.
Performance of ROSL
CVPR2014 5
Proposition
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Efficient Algorithm
• Lagrange Function
• Alternating Direction Method of Multipliers (ADMM)
– Subspace learning:
Group sparsity shrinkage sequentially updates
automatically shrinks subspace dim from to rank
– Solve error component:
– Update Lagrangian multiplier
• Overall complexity is quadratic
CVPR2014 6Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
ROSL+
• ROSL+: linear-complexity algorithm by random sampling (RS)
• Nystrom method[Talwalkar10]:
• Three major steps:
– Obtain XTL by random sampling h rows and l columns
– Solve and by applying ROSL on
+
– Solve coefficients by minimizing
||
• Final low-rank recovery
CVPR2014 7
AT =[ATL , ATR]
AL =[ATL ; ABL]
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Experimental Results
• Synthetic data (m=n, r=10, k=30, h=l=100), generated by
– Multiplying a matrix and a matrix, which obeys N(0,1).
– then adding a sparse error (sparsity:10%), drawn from Unif[-50, 50].
CVPR2014 9
rm k
m
MAE: Mean of Absolute Error
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Experimental Results
• ROSL at varying initial subspace dimension and weight
• ROSL+ at varying random sampling density (h = l)
CVPR2014 10
r=10, k=30
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Experimental Results
• Background subtraction in surveillance video (160x128,1060frames)
CVPR2014 11
Original RPCA(time:334s) ROSL(time:34.6s) ROSL+(time:3.61s)
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Experimental Results
• Illumination removal in Yale-B face dataset (168x192, 55 frames)
CVPR2014 12
Original RPCA (time:12.16s) ROSL(time: 5.85s)
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Reference• [Huang14]: J. Huang, http://jbhuang0604.blogspot.com/2013/04/miss-korea-2013-contestants-face.html
http://misskorea.mpluskorea.com/missdaegu2013_poll
• [Shu11]: X. Shu and N. Ahuja. Imaging via three-dimensional compressive sampling (3DCS). In ICCV, 2011.
• [Mobahi11]: H. Mobahi, Z. Zhou, A. Yang, and Y. Ma. Holistic Reconstruction of Urban Structures from Low-rank Textures In ICCV_3dRR,
2011.
• [Candes11]: E. Candes, X. Li, Y. Ma, and J. Wright. Robust principal component analysis? Journal of the ACM, 58(3):article 11, 2011.
• [Lin10]: Z. Lin, M. Chen, and Y. Ma. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices.
Technical Report UILU-ENG-09-2214, UIUC, 2010
• [Mu11]: Y. Mu, J. Dong, X. Yuan, and S. Yan. Accelerated low-rank visual recovery by random projection. In CVPR, 2011.
• [Zhou11]: T. Zhou and D. Tao. GoDec: randomized low-rank andsparse matrix decomposition in noisy case. In ICML, 2011.
• [Ke05]: Q. Ke and T. Kanade. Robust l1 norm factorization in the presence of outliers and missing data by alternative convex programming.
In CVPR, volume 1, pages 739–746, 2005.
• [Shen11]: Y. Shen, Z.Wen, and Y. Zhang. Augmented lagrangian alternatingdirection method for matrix separation based on low-rank
factorization. Technical Report TR11-02, Rice University, 2011.
• [Talwalkar10]: A. Talwalkar and A. Rostamizadeh. Matrix coherence and the nystrom method. In Proceedings of the Twenty-Sixth
Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-10), 2010.
CVPR2014 13Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Summary
• Robust Orthonormal Subspace Learning(ROSL)
– A new rank measure using row-1 group sparsity under orthogonal subspace,
which is lower bounded by nuclear norm.
– A fast low-rank recovery method (ROSL) with the same global minima as RPCA.
– An efficient algorithm is given to solve ROSL with stable convergence behavior
– An accelerated version (ROSL+) with linear complexity by random sampling
– Experimental results show that our ROSL/ROSL+ are much faster than the
performance-guaranteed method (RPCA) at the same level of accuracy.
CVPR2014 14
rn k
m
Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Summary
Code is available
https://sites.google.com/site/xianbiaoshu/
Xianbiao Shu Fatih Porikli Narendra Ahuja
CVPR2014 15Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
Thanks
And
Questions?
CVPR2014 16Robust Orthonormal Subspace Learning: Efficient Recovery of Corrupted Low-rank Matrices
