rémi gribonval inria rennes - bretagne atlantique, france · 2013-10-09 · r. gribonval - sparse...
TRANSCRIPT
![Page 1: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/1.jpg)
Inverse problems and sparse models (1/6)
Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France
http://people.irisa.fr/Remi.Gribonvall
![Page 2: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/2.jpg)
2013R. GRIBONVAL - SPARSE METHODS
PDF of the slides
• http://www.irisa.fr/metiss/gribonval/Teaching/
2
![Page 3: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/3.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Overview of the course
• Introduction✓ sparsity & data compression✓ inverse problems in signal and image processing
✦ image deblurring, image inpainting, ✦ channel equalization, signal separation, ✦ tomography, MRI
✓ sparsity & under-determined inverse problems✦ well-posedness
• Complexity & Feasibility✓ NP-completeness of ideal sparse approximation✓ Relaxations✓ L1 is sparsity-inducing and convex
3
![Page 4: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/4.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Overview of the course
• Pursuit Algorithms✓ L1 has performance guarantees✓ L1 is computationally feasible: Basis Pursuit✓ Greedy algorithms: Matching Pursuit & al✓ Complexity of Pursuit Algorithms
• Recovery guarantees✓ Coherence vs Restricted Isometry Constant✓ Worked examples✓ Summary
4
![Page 5: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/5.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Further material on sparsity
5
• Books✓ Signal Processing perspective
✦ S. Mallat, «Wavelet Tour of Signal Processing», 3rd edition, 2008✦ M. Elad, «Sparse and Redundant Representations: From Theory to
Applications in Signal and Image Processing», 2009.✓ Mathematical perspective
✦ S. Foucart, H. Rauhut, «A Mathematical Introduction to Compressed Sensing», Springer, in preparation.
• Review paper: ✦ Bruckstein, Donoho, Elad, SIAM Reviews, 2009
![Page 6: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/6.jpg)
2013 - R. GRIBONVAL - SPARSE METHODS
Sparse models & data compression
![Page 7: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/7.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Large-scale data
• Fact : digital data = large volumes
✓ 1 second stereo audio, CD quality = 1,4 Mbit✓ 1 uncompressed 10 Mpixels picture = 240 Mbit
• Need : «concise» data representations
✓ storage & transmission (volume / bandwidth) ... ✓ manipulation & processing (algorithmic
complexity)
7
![Page 8: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/8.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Notion of sparse representation
• Audio : time-frequency representations (MP3)
• Images : wavelet transform (JPEG2000)
Black = zero
Gray = zero
8
ANALYSIS
ANALYSIS
SYNTHESIS
SYNTHESIS
![Page 9: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/9.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Evidence of sparsity
• Histogram of MDCT coefficients of a musical sound
•
9
![Page 10: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/10.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Mathematical expression of the sparsity assumption
• Signal / image = high dimensional vector
• Definition: ✓ Atoms: basis vectors
✦ ex: time-frequency atoms, wavelets✓ Dictionary:
✦ collection of atoms
✦ matrix which columns are the atoms
• Sparse signal model = combination of few atoms
10
y 2 RN
y ⇡X
k
xk'k = �x
'k 2 RN
{'k}1kK
� = ['k]1kK
![Page 11: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/11.jpg)
2013R. GRIBONVAL - SPARSE METHODS
• Full vector • Sparse vector
Sparsity & compression
11
y xN
nonzero entries = k floats
k ⌧ NN entries
= N floats
+ k positions among N
= bitslog2
✓N
k
◆⇡ k log2
N
k
⇡ �·
Key practical issues: choose dictionary
![Page 12: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/12.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Sparsity: definition
• A vector is ✓ sparse if it has (many) zero coefficients✓ k-sparse if it has at most k nonzero coefficients
• Symbolic representation as column vector
• Support = indices of nonzero components
• Sparsity measured with L0 pseudo-norm
• In french: ✦ sparse -> «creux», «parcimonieux»✦ sparsity, sparseness -> «parcimonie», «sparsité»
12
Not sparse
3-sparse⇧x⇧0 := ⇥{n, xn �= 0} =
�
n
|xn|0
a0 = 1(a > 0); 00 = 0Convention here
![Page 13: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/13.jpg)
2013 - R. GRIBONVAL - SPARSE METHODS
Inverse problems in signal and image processing
![Page 14: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/14.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Deconvolution problem2D Example : deblurring problem
14
• Given data: ✓ blurred image
✓ information on blurring process
• Desired estimate:✓ deblurred image
?
y[i, j]
x[i, j]
![Page 15: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/15.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Blurring process = 2D Convolution
• Definition
• Interpretation : local average
yx h i
j
Reproduced from http://www.robots.ox.ac.uk/~improofs/super-resolution/super-res1.html
15
y[i, j] = (h � x)[i, j] :==X
k,�
h[k, ⇥]x[i� k, j � ⇥]
![Page 16: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/16.jpg)
2013R. GRIBONVAL - SPARSE METHODS
• Optical blur
• Motion blur
Examples of 2D convolution
h =point spread function (PSF)
Reproduced from http://www.robots.ox.ac.uk/~improofs/super-resolution/super-res1.html
16
![Page 17: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/17.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Example of point spread function
Reproduced from http://hea-www.harvard.edu/HRC/calib/hrci_qe.html
17
h[i, j]
ij
![Page 18: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/18.jpg)
2013R. GRIBONVAL - SPARSE METHODS
1D deconvolution problems
• General form
• Telecom: channel equalization✓ h = channel impulse response
• Audio: de-reverberation (reflections on walls)✓ h = room impulse response
€
τ1,h(τ1)€
τ 2,h(τ 2)
€
τ 3,h(τ 3)
18
y(t) = (h ⇥ x)(t) :=Z
h(�)x(t� �)d�
![Page 19: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/19.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Example of room impulse response
Reproduced from http://www.am3d.com/technology/acoustical
19
![Page 20: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/20.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Deconvolution Problem
20
• Given✓ measured data y✓ known filter h
• Find unknown x such that
y = h � x
![Page 21: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/21.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Naive deconvolution in the Fourier domain
• Convolution and Fourier / inverse Fourier
• H(f) = transfer function of filter h
21
F{·}Y (f) = H(f)X(f)y(t) = (h � x)(t)
F�1{·}X̂(f) =
Y (f)H(f)
= X(f)x̂(t) = x(t)
![Page 22: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/22.jpg)
2013R. GRIBONVAL - SPARSE METHODS
• Presence of noise
• Smooth filter: ✓ fast decay of H(f)✓ small values in H(f)✓ division by small values = strong amplification of noise
• Consequence = missing frequency information✓ N frequency components to estimate
✓ m < N reliable frequency components
Issues with naive deconvolution
22
Y (f) = H(f)X(f) + N(f)y(t) = (h � x)(t) + n(t)
X̂(f) :=Y (f)H(f)
= X(f) +N(f)H(f)
X 2 RN
Y 2 Rm YX
![Page 23: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/23.jpg)
2013 - R. GRIBONVAL - SPARSE METHODS
Inverse problems
![Page 24: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/24.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Linear inverse problems: definition
• Definition: a problem where a high-dimensional vector must be estimated from its low dimensional projection
• Generic form:
✓ m observations / measures✓ N unknowns
24
b = Ay + eobservation/measure
projection matrix
unknown noise
b 2 Rm
y 2 RNA 2 Rm⇥N
![Page 25: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/25.jpg)
2013R. GRIBONVAL - SPARSE METHODS
• Unknown image with N pixels
• Partially observed image: ✓ m < N observed pixels
• Measurement matrix
Example: Inpainting Problem
25
b y
yy 2 RN
b[�p] = y[�p], �p 2 Observed
b = My
![Page 26: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/26.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Classes of linear inverse problems
• Determined: the matrix A is square and invertible✓ Unique solution to ✓ Linear function of observations
• Over-determined: more equations than unknowns✓ Unique solution to :✓ Linear function of observations ✓ with pseudo-inverse
• Under-determined: fewer equations than unknowns✓ Infinitely many solutions to✓ Need to choose one?
26
b = Ay
b = Ay
b = Ay
y = A�1bA
A
A
y = A†b
![Page 27: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/27.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Signal space ~ RN
Observation space ~ Rm m<<N
Linear projection
Nonlinear Approximation =
Sparse recovery
Set of signals of interest
Inverse problems
27
Courtesy: M. Davies, U. Edinburgh
b
x
A
![Page 28: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/28.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Example : audio source separation
• « Softly as in a morning sunrise »
28
![Page 29: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/29.jpg)
2013R. GRIBONVAL - SPARSE METHODS
• Mixing model : linear instantaneous mixture
• Source model : if disjoint time-supports …
Blind Source Separation
... then clustering to :1- identify (columns of) the mixing matrix2- recover sources
s1(t)
s3(t)s2(t)
29
yright(t)
yleft(t)
yleft(t)
yright(t)
![Page 30: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/30.jpg)
2013R. GRIBONVAL - SPARSE METHODS
• Mixing model : linear instantaneous mixture
• In practice ...
•
Blind Source Separation
s1(t)
s3(t)s2(t)
30
yright(t)
yleft(t)
yleft(t)
yright(t)
![Page 31: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/31.jpg)
2013R. GRIBONVAL - SPARSE METHODS
• Mixing model in the time-frequency domain
• And “miraculously” ...
•
Time-Frequency Masking
... time-frequency representations of audio signals are (often) almost disjoint.
S(�, f)
31
Yright(�, f)
Yleft(�, f)
Yleft(�, f)
Yright(�, f)
![Page 32: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/32.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Inverse problems
• Inverse problem : exploit indirect or incomplete obervation to recontruct some data
• Sparsity : represent / approximate high-dimensional & complex data using few parameters
32
Dat
a
Rep
rese
ntat
ion
Reduce the dimension
Dat
a
Obs
erva
tions
Reconstruct
few nonzero components
fewer equations than unknowns
y ⇡ �x
z = My
![Page 33: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/33.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Forward linear model
Signal Processing Vocabulary
Known linear system:dictionary, mixing matrix, sensing system...
Observed data:signal, image, mixture of sources,...
b � Ax
Unknownrepresentation, sources, ...
33
A DecompositionReconstruction
Separation
x
b
![Page 34: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/34.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Unknownrepresentation, sources, ...Regression coeffs
Forward linear model
Machine Learning Vocabulary
Known linear system:dictionary, mixing matrix, sensing system...
Observed data:signal, image, mixture of sources,...
b � Ax
34
A DecompositionReconstruction
SeparationDesign matrix
Observation
X
y
X
b
x
y = Xw
w
![Page 35: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/35.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Unknownrepresentation, sources, ...Regression coeffs
Forward linear model
Statistics Vocabulary
Known linear system:dictionary, mixing matrix, sensing system...
Observed data:signal, image, mixture of sources,...
b � Ax
35
A DecompositionReconstruction
Separation
y = X�
Design matrix
Observation
X
y
X
�
b
x
![Page 36: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/36.jpg)
2013 - R. GRIBONVAL - SPARSE METHODS
Inverse problems & Sparsity
![Page 37: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/37.jpg)
2013R. GRIBONVAL - SPARSE METHODS
Inverse Problems & Sparsity:Mathematical foundations
• Bottleneck 1990-2000 : ✓ Ill-posedness when fewer equations than unknowns
• Novelty 2001-2006 : ✓ Well-posedness = uniqueness of sparse solution:
✦ if are “sufficiently sparse”,
✦ then
✓ Recovery of with practical pursuit algorithms ✦ Thresholding, Matching Pursuits, Minimisation of Lp norms p<=1,...
37
x0, x1
Ax0 = Ax1 ⇥� x0 = x1
Ax0 = Ax1 � x0 = x1
x0
![Page 38: Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France · 2013-10-09 · R. GRIBONVAL - SPARSE METHODS 2013 Further material on sparsity 5 •Books Signal Processing perspective](https://reader034.vdocuments.site/reader034/viewer/2022050103/5f41f5464fa46e0dd9091444/html5/thumbnails/38.jpg)
2013R. GRIBONVAL - SPARSE METHODS
x
Sparsity and subset selection
x• Under-determined system✓ Infinitely many solutions
• If vector is sparse: ✓ If support is known (and columns independent)
✦ nonzero values characterized by (over)determined linear problem✓ If support is unknown
✦ Main issue = finding the support! ✦ This is the subset selection problem
• Objectives of the course✦ Well-posedness of subset selection✦ Efficient subset selection algorithms = pursuit algorithms✦ Stability guarantees of pursuits
b ~ A
b ~
38