jon dattorro convexoptimization.com. prototypical cardinality problem perspectives: combinatorial...
TRANSCRIPT
Jon Dattorro
convexoptimization.com
prototypical cardinality problemprototypical cardinality problem
kx
x
bAx
x
card
0
subject to
find
Perspectives:Combinatorial
Geometric
2
Euclidean bodiesEuclidean bodiesPermutation Polyhedron
T{ , , }n nX X X X R 1 1 1 1 0P• n! permutation matrices are vertices in (n-1)2 dimensions.• permutaton matrices are minimum cardinality doubly stochastic matrices.
Hyperplane
}1{ T 1R xx nH
H
3
Geometrical perspective
Compressed SensingCompressed Sensing
bAx
x
subject to
||||minimize 1
1-norm ball: 2n vertices, 2n facets
4 Candes/Donoho (2004)
Candes demo %Emmanuel Candes, California Institute of Technology, June 6 2007, IMA Summerschool.
clear all, close all n = 512; % Size of signal m = 64; % Number of samples (undersample by a factor 8)
k = 0:n-1; t = 0:n-1; F = exp(-i*2*pi*k'*t/n)/sqrt(n); % Fourier matrix freq = randsample(n,m); A = [real(F(freq,:)); imag(F(freq,:))]; % Incomplete Fourier matrix
S = 28; support = randsample(n,S); x0 = zeros(n,1); x0(support) = randn(S,1); b = A*x0;
% Solve l1 using CVX cvx_quiet(true); cvx_begin variable x(n); minimize(norm(x,1)); A*x == b; cvx_end
norm(x - x0)/norm(x0) figure, plot(1:n,x0,'b*',1:n,x,'ro'), legend('original','decoded')
5wikimization.org
6
Candes demo
0
find
subject to
x
Fx Fx
binary mask
Fourier matrix
is sparse
F
x
0
64 2
512
28
m n
m
A F
b Fx
m
n
k
R
R
k-sparse sampling theoremk-sparse sampling theorem
nmA R• Donoho/Tanner (2005)
7
two geometrical two geometrical interpretationsinterpretations
0
subject to
find
xbAx
x
}0{ xAxK 8
motivation to study conesmotivation to study conesconvex cones generalize orthogonal
subspaces
Projection on K determinable from projection on -K* and vice versa. (Moreau)
Dual cone:
nKK R *
}0{ T* KxyxyK
9
application - LP application - LP presolverpresolverDelete rows and columns of matrix Acolumns: smallest face F of cone K containing
b
A holds generators for K
If feasible, throw A(: , i) away
}0{ xAxK
10
}{)( * bKaKabKF
1),(:
0
0tosubject
find
T
T
T
ziA
zA
zb
z
application - application - CartographyCartography
11
list reconstruction from distance list reconstruction from distance DD
metric multidimensional scalingprincipal component analysisKarhunen-Loeve transform
cartography: projection on semidefinite cone
12
a.k.a
13
0tosubject
||)(||minimize
nT
n
FnT
n
VDV
VHDVhD
S
projection on semidefinite cone because
nT
n VV hSS subspace of symmetric matrices is isomorphic with subspace of symmetric hollow matrices
is convex problem (Eckart & Young) (§7.1.4 CO&EDG)
optimal list X from (§5.12 CO&EDG)
14
nT
n VDV
nT
n
nT
n
FnT
n
rank
0tosubject
||)(||minimize
VDV
VDV
VHDVhD
S
(EY)
ordinal reconstructionordinal reconstruction
15
M
D
KD
VDV
VDV
VODVh
vect
rank
0tosubject
||)(||minimize
nT
n
nT
n
FnT
n
S
• nonconvex• strategy: break into two problems: (EY) and convex problem
• fast projection on monotone nonnegative cone KM+ (Nemeth, 2009) •
MK
D
tosubject
||vect||minimize
25020R
Cardinality heuristics
16
y
*minimize vec ,
subject to vec
y
E t
0
0minimize || ||
subject to vecE t
4M 4MR
0
Rank heuristicstrace is convex envelope of rank on PSD
matrices
rank function is quasiconcave
17
Idea behind convex iteration
18
IGG ,tr (vector inner product)
Convex Iteration
19
application -
20
(Recht, Fazel, Parrilo, 2007)
(Rice University 2005)
one-pixel camera - MIT
21
22
one-pixel camera - MIT
application - MRI phantom
23
Candes, Romberg, Tao 2004
• Led directly to sparse sampling theorem
MATLAB>> phantom(256)
24
application - MRI phantom• MRI raw data called k-space
• Raw data in Fourier domain• aliasing at 4% subsampling
1vect f
25
application - MRI phantom
P16 162 × 2C
(projection matrix)
y is direction vector from convex iteration
• hard to compute
26
application - MRI phantom
27
application - MRI phantom
reconstruction error: -103dB