learning sparse representations to restore, classify, and sense images and videos
DESCRIPTION
Learning sparse representations to restore, classify, and sense images and videos. Guillermo Sapiro University of Minnesota. Supported by NSF, NGA, NIH, ONR, DARPA, ARO, McKnight Foundation. Ramirez. Martin Duarte. Lecumberry. Rodriguez. Overview. Introduction - PowerPoint PPT PresentationTRANSCRIPT
-
Learning sparse representations to restore, classify, and sense images and videosGuillermo Sapiro
University of MinnesotaSupported by NSF, NGA, NIH, ONR, DARPA, ARO, McKnight Foundation
-
Learning Sparsity*Martin DuarteRodriguezRamirez
Lecumberry
Learning Sparsity
-
Learning Sparsity* OverviewIntroductionDenoising, Demosaicing, InpaintingMairal, Elad, Sapiro, IEEE-TIP, January 2008
Learn multiscale dictionaries Mairal, Elad, Sapiro, SIAM-MMS, April 2008
Sparsity + Self-similarityMairal, Bach, Ponce, Sapiro, Zisserman, pre-print.
Incoherent dictionaries and universal codingRamirez, Lecumberry, Sapiro, June 2009, pre-print
Learning to classifyMairal, Bach, Ponce, Sapiro, Zisserman, CVPR 2008, NIPS 2008Rodriguez and Sapiro, pre-print, 2008.
Learning to sense sparse signalsDuarte and Sapiro, pre-print, May 2008, IEEE-TIP to appear
Learning Sparsity
-
Learning Sparsity*Introduction I: Sparse and Redundant Representations
Webster Dictionary: Of few and scattered elements
Learning Sparsity
-
Learning Sparsity* Restoration by Energy Minimization y : Given measurements x : Unknown to be recoveredRestoration/representation algorithms are often related to the minimization of an energy function of the formBayesian type of approach
What is the prior? What is the image model?
Learning Sparsity
-
Learning Sparsity* The Sparseland Model for Images M
Learning Sparsity
-
Learning Sparsity* What Should the Dictionary D Be?
Learning Sparsity
-
Learning Sparsity*Introduction II: Dictionary Learning
Learning Sparsity
-
Learning Sparsity* Measure of Quality for DField & Olshausen (96)Engan et. al. (99)Lewicki & Sejnowski (00)Cotter et. al. (03)Gribonval et. al. (04)Aharon, Elad, & Bruckstein (04) Aharon, Elad, & Bruckstein (05)Ng et al. (07)Mairal, Sapiro, Elad (08)
Learning Sparsity
-
Learning Sparsity* The KSVD Algorithm General Aharon, Elad, & Bruckstein (`04)
Learning Sparsity
-
Learning Sparsity*Show me the pictures
Learning Sparsity
-
Learning Sparsity* Change the Metric in the OMP
Learning Sparsity
-
Learning Sparsity* Non-uniform noise
Learning Sparsity
-
Learning Sparsity* Example: Non-uniform noise
Learning Sparsity
-
Learning Sparsity* Example: Inpainting
Learning Sparsity
-
Learning Sparsity* Example: Demoisaic
Learning Sparsity
-
Learning Sparsity* Example: Inpainting
Learning Sparsity
-
Learning Sparsity*Not enough fun yet?: Multiscale Dictionaries
Learning Sparsity
-
Learning Sparsity*Learned multiscale dictionary
Learning Sparsity
-
Learning Sparsity*
Learning Sparsity
-
Learning Sparsity*Color multiscale dictionaries
Learning Sparsity
-
Learning Sparsity*Example
Learning Sparsity
-
Learning Sparsity*Video inpainting
Learning Sparsity
-
Extending the Models Learning Sparsity*
Learning Sparsity
-
Universal Coding and Incoherent Dictionaries
ConsistentImproved generalization propertiesImproved active set computationImproved coding speedImproved reconstructionSee poster by Ramirez and Lecumberry
Learning Sparsity*
Learning Sparsity
-
Sparsity + Self-similarity=Group SparsityCombine the two of the most successful models for images
Mairal, Bach. Ponce, Sapiro, Zisserman, pre-print, 2009 Learning Sparsity*
Learning Sparsity
-
Learning to Classify
-
Learning Sparsity*Global Dictionary
Learning Sparsity
-
Learning Sparsity*Barbara
Learning Sparsity
-
Learning Sparsity*Boat
Learning Sparsity
-
Learning Sparsity*Digits
Learning Sparsity
-
Which dictionary? How to learn them?Multiple reconstructive dictionary? (Payre)
Single reconstructive dictionary? (Ng et al, LeCunn et al.)
Dictionaries for classification!
See also Winn et al., Holub et al., Lasserre et al., Hinton et al. for joint discriminative/generative probabilistic approaches
Learning Sparsity*
Learning Sparsity
-
Learning Sparsity*Learning multiple reconstructive and discriminative dictionariesWith J. Mairal, F. Bach, J. Ponce, and A. Zisserman, CVPR 08, NIPS 08
Learning Sparsity
-
Learning Sparsity*Texture classification
Learning Sparsity
-
Semi-supervised detection learningMIT -- Learning Sparsity*
Learning Sparsity
-
Learning Sparsity*Learning a Single Discriminative and Reconstructive Dictionary
Exploit the representation coefficients for classificationInclude this in the optimizationClass supervised simultaneous OMPWith F. Rodriguez
Learning Sparsity
-
Learning Sparsity*Digits images: Robust to noise and occlusions
Learning Sparsity
-
Learning Sparsity*Supervised Dictionary LearningWith J. Mairal, F. Bach, J. Ponce, and A. Zisserman, NIPS 08
Learning Sparsity
-
Learning to Sense Sparse Images
-
MotivationCompressed sensing (Candes &Tao, Donoho, et al.)SparsityRandom samplingUniversalityStabilityShall the sensing be adapted to the data type?Yes! (Elad, Peyre, Weiss et al., Applebaum et al, this talk).Shall the sensing and dictionary be learned simultaneously?
Learning Sparsity*
Learning Sparsity
-
Some formulas.
Learning Sparsity*+ RIP (Identity Gramm Matrix)
Learning Sparsity
-
Design the dictionary and sensing togetherLearning Sparsity*
Learning Sparsity
-
Just Believe the Pictures Learning Sparsity*
Learning Sparsity
-
Just Believe the PicturesLearning Sparsity*
Learning Sparsity
-
Just Believe the PicturesLearning Sparsity*
Learning Sparsity
-
Learning Sparsity*ConclusionsState-of-the-art denoising results for still (shared with Dabov et al.) and videoGeneralVectorial and multiscale learned dictionariesDictionaries with internal structureDictionary learning for classificationSee also Szlam and Sapiro, ICML 2009See also Carin et al, ICML 2009Dictionary learning for sensing
A lot of work still to be done!
Learning Sparsity
-
Please do not use the wrong dictionaries12 M pixel image7 million patchesLARS+online learning: ~8 minutes
Mairal, Bach, Ponce, Sapiro, ICML 2009
Learning Sparsity*
Learning Sparsity
-
Learning Sparsity*
Learning Sparsity