to restore, classify, and sense images and videospeople.ee.duke.edu/~lcarin/sapiro.pdfto restore,...

Learning sparse representationsLearning sparse representationsto restore, classify, and sense images and videos

Guillermo Sapirop

University of Minnesota

Supported by NGA, ONR, DARPA, ARO, NSF, NIH

Compressive LecturingCompressive Lecturing

Learning Sparsity 2

Ramirez

Martin Duarte

Lecumberry

Rodriguez

Learning Sparsity 3

For the past ~2 years …• Introduce and Extend the K-SVD

– Denoising– Demosaicing– Inpainting– Mairal, Elad, Sapiro, IEEE-TIP, January 2008

• Learn multiscale dictionaries– Mairal, Elad, Sapiro, SIAM-MMS, April 2008

• Incoherent dictionaries– Ramirez, Lecumberry, Sapiro, January 2009, pre-print

• Learning to classify– Mairal, Bach, Ponce, Sapiro, Zisserman, CVPR 2008, NIPS 2008Mairal, Bach, Ponce, Sapiro, Zisserman, CVPR 2008, NIPS 2008– Rodriguez and Sapiro, pre-print, 2008.

• Learning to sense sparse signals– Duarte and Sapiro pre-print May 2008 IEEE-TIP to appear

Learning Sparsity 4

Duarte and Sapiro, pre print, May 2008, IEEE TIP to appear

The Sparseland Model for Images

M Every column in D (dictionary) is M Ka prototype signal (Atom).

NN The vector α

contains very few (say L) non-zeros.

=x

N

DA fixed Dictionary A sparse

& random vector

xD α

Learning Sparsity 5

What Should the Dictionary D Be?

α=≤α−α=α ˆx̂L.t.sy21

minargˆ 00

22

DDα 2

D should be chosen such that it sparsifies the representations (for a given task!)

Learn D :

Multiscale LearningOne approach to choose D is from a known set of transforms (Steerable

Color Image Examples

Task / sensing adapted

Internal structure

wavelet, Curvelet, Contourlets, Bandlets, …)

Learning Sparsity 6

Internal structure

Dictionaries for Reconstruction

Learning Sparsity 7

Color multiscale dictionaries

Learning Sparsity 8

Example: Non-uniform noise

Learning Sparsity 9

Example: Inpainting

Learning Sparsity 10

Learning multiple reconstructive and discriminative dictionaries

With J. Mairal, F. Bach, J. Ponce, and A. Zisserman, CVPR ’08, NIPS ‘08

Learning Sparsity 11

, , , , ,

Semi-supervised detection learningp g

MIT -- Learning Sparsity 12

Supervised Dictionary Learning

With J. Mairal, F. Bach, J. Ponce, and A. Zisserman, NIPS ‘08

Learning Sparsity 13

, , , ,

Learning Incoherent Dictionaries

• Optimization depends on the incoherence• Improved generalization properties• Improved classificationImproved classification

Learning Sparsity 14

Results

Learning Sparsity 15

Results

Learning Sparsity 16

Learning for Compressed Sensing

+ “RIP (Identity Gramm Matrix)”

Learning Sparsity 17

Design the dictionary and sensing togethertogether

Learning Sparsity 18

Just Believe the PicturesJust Believe the Pictures

Learning Sparsity 19

Just Believe the PicturesJust Believe the Pictures

Learning Sparsity 20

Just Believe the PicturesJust Believe the Pictures

Learning Sparsity 21

Conclusions• State-of-the-art denoising results for still

(shared with Dabov et al ) and video(shared with Dabov et al.) and video• Vectorial and multiscale learned

dictionariesdictionaries• Dictionary learning with internal structure• Dictionary learning for classification• Dictionary learning for classification• Dictionary learning for sensing

• Dictionary learning for the task• Optimization is dictionary dependent

Learning Sparsity 22

p y p

Please do not use the wrong dictionaries thanksdictionaries… thanks

• 12 M pixel image• 7 million patches7 million patches• LARS+online

learning:learning: ~8 minutes

Learning Sparsity 23

Learning Sparsity 24