toward 0-norm reconstruction, and a nullspace technique for compressive sampling christine law gary...
TRANSCRIPT
Toward 0-norm Toward 0-norm Reconstruction, and A Reconstruction, and A Nullspace Technique Nullspace Technique
for Compressive for Compressive SamplingSamplingChristine LawChristine LawGary GloverGary Glover
Dept. of EE, Dept. of Dept. of EE, Dept. of RadiologyRadiology
Stanford UniversityStanford University
OutlineOutline
0-norm Magnetic Resonance 0-norm Magnetic Resonance Imaging (MRI) reconstruction.Imaging (MRI) reconstruction. HomotopicHomotopic Convex iterationConvex iteration
Signal separation exampleSignal separation example
1-norm deconvolution in fMRI. 1-norm deconvolution in fMRI. Improvement in cardinality Improvement in cardinality
constraint problem with nullspace constraint problem with nullspace technique.technique.
Shannon Shannon vsvs. Sparse . Sparse SamplingSampling
Nyquist/Shannon : sampling rate ≥ 2*max freqNyquist/Shannon : sampling rate ≥ 2*max freq
Sparse Sampling Theorem: (Candes/Donoho Sparse Sampling Theorem: (Candes/Donoho 2004)2004)
Suppose Suppose xx in in RRnn is is kk-sparse and we are given -sparse and we are given mm Fourier coefficients with frequencies selected Fourier coefficients with frequencies selected uniformly at random. If uniformly at random. If
mm ≥ ≥ kk log log22 (1 + (1 + n n / / kk))
then then
reconstructs reconstructs xx exactly with overwhelming exactly with overwhelming probability. probability.
1arg min s.t. =x x x y
n
k = 2
Reconstruction by Reconstruction by OptimizationOptimization
Compressed Sensing theory (2004 Donoho, Candes): under certain conditions,
Candes et al. IEEE Trans. Information Theory 2006 52(2):489 Donoho. IEEE Trans. Information Theory 2006 52(4):1289
1arg min s.t. =x x x y
0arg min s.t. =x x x y
y are measurements (rats)
x are sensors (wine)
0-norm reconstruction0-norm reconstruction
Try to solve 0-norm directly.Try to solve 0-norm directly.
For For pp-norm, where 0 < -norm, where 0 < pp < 1 < 1
Chartrand (2006) demonstrated fewer samples Chartrand (2006) demonstrated fewer samples y y required than 1-norm formulation.required than 1-norm formulation.
Chartrand. IEEE Signal Processing Letters. 2007: 14(10) 707-710.Chartrand. IEEE Signal Processing Letters. 2007: 14(10) 707-710.
0arg min s.t. =u u u y
Trzasko (2007): Rewrite the Trzasko (2007): Rewrite the problem problem
0, .min lim s.t
i iu u u y
00, lim
i iu u
Trzasko Trzasko et alet al. IEEE SP 14th workshop on statistical signal processing. 2007. 176-180.. IEEE SP 14th workshop on statistical signal processing. 2007. 176-180.
0arg min s.t. =u u u y
where where is tanh, laplace, log, is tanh, laplace, log, etcetc. . such thatsuch that
Homotopic function in Homotopic function in 1D1D
Laplace function: 1 xe
Start as 1-norm problem, then reduce slowly and approach 0-norm function.
1000
1
Homotopic methodHomotopic method
2
2min ,
2iiE
uu u y
T H,
,
Find minimum of by zeroing Lagrangian gradient:
= 0
isdiag
u
E
E u u u y
d d uu
u
DemonstrationDemonstration when is big (1when is big (1stst iteration), solving 1-norm problem. iteration), solving 1-norm problem. reduce reduce to approach 0-norm solution. to approach 0-norm solution.
061%x
02%x
original x∆
: image to solve
: k-space measurements (undersampled)
: Total Variation (TV) operator
: Fourier matrix (masked)
u
y
Example 1Example 1
F
F
original phantom - reconstructionerror 20 log
original phantom
original subsampled
Zero-filled Reconstruction Fourier sample mask
HomotopicHomotopic result: use 4% Fourier data error: -66.2 dB 85 seconds
1-norm result: use 4% Fourier data error: -11.4 dB 542 seconds
1-normrecon
homotopicrecon
Example 2Example 2
AngiographyAngiography 360x360, 27.5% radial samples360x360, 27.5% radial samples
original
reconstructionreconstruction1-norm method: 1-norm method: error: -24.7 dB, error: -24.7 dB, 1151 seconds1151 seconds
360x360360x360
reconstruction reconstruction homotopichomotopic method: method:
error: -26.5 dB, error: -26.5 dB, 101 seconds101 seconds
original
27.5% samples27.5% samples
2
2
0=
min ,2
arg min s.t. u y
E uu
u u
u y
0,1i
Chretien, An Alternating l1 approach to the compressed sensing problem, arXiv.org .Dattorro, Convex Optimization & Euclidean Distance Geometry, Meboo.
Convex Iteration
Convex IterationConvex Iteration2
2min ,
2E u
uu y
T H
Find minimum of by zeroing Lagrangian gradient:
sgn - 0
sgn
u
E
E u u y
uu
u
Convex Iteration demoConvex Iteration demo
zero-filled reconstruction Fourier sample mask
use 4% Fourier data error: -104 dB 96 seconds
reconstruction
Signal Separation by Convex Iteration
-1 -1D
D
Du u u
is Total Variation operator
D is Discrete Cosine Transform matrix
D
T
is inverse measurement in domain
real
: Gram-Schmidt orthogonalization of Fourier matrix
: binary sampling mask rank = rank
: ( )
=
y y P u u
P M
M P M
F
: F F w
F :
Dwant to find ,u u
D1 1D
-1 -1
D
minimize,
s.t. Dy P
1-norm formulation
as convex iteration
T TD
D
-1 -1
D
minimize ,,
s.t. Dy P
0 50 100 150 200 250 300-60
-40
-20
0
20
40
60
Signal Construction
0 50 100 150 200 250 300-40
-20
0
20
40
60
0 50 100 150 200 250 300-6
-4
-2
0
2
4
6
=
+
Cardinality(steps) = 7Cardinality(cosine) = 4
0 50 100 150 200 250 300-80
-60
-40
-20
0
20
40
60
0 50 100 150 200 250 300-50
-40
-30
-20
-10
0
10
-1
-1D DD
u
u
u
u∆
uD
D
Donoho, Tanner, 2005. Baron, Wakin et al., 2005
Minimum Sampling RateMinimum Sampling Rate
k/n
m/k
m measurementsk cardinalityn record length
0 50 100 150 200 250 3000
5
10
15
20
25
30
35
m = 28 measurementsm/n = 0.1 subsamplem/k = 2.5 sample ratek/n = 0.04 sparsity
0 50 100 150 200 250 300-60
-40
-20
0
20
40
60
Cardinality(steps) = 7Cardinality(cosine) = 4
M uF
uSignal Separationby Convex Iteration
0 50 100 150 200 250 300-50
0
50
100error = -23.189286 dB
orig signal
L1 est signal
0 50 100 150 200 250 300-50
0
50
100error = -241.880696 dB
orig signal
est signal
0 50 100 150 200 250 300-50
0
50
100error = -14.488360 dB
orig signal
est signal using TV alone
0 50 100 150 200 250 300-50
0
50
100error = -6.648319 dB
orig signal
est signal using DCT alone
1-norm
Convex iteration
TV only
DCT only
-241dB reconstruction error
0 50 100 150 200 250 300-40
-20
0
20
40
60
piecewise
est piecewise
0 50 100 150 200 250 300-6
-4
-2
0
2
4
6
cosine
est cosine
u∆
uD
functional Magnetic Resonance functional Magnetic Resonance Imaging (fMRI)Imaging (fMRI)
Haemodynamic Response Function Haemodynamic Response Function
(HRF)(HRF)
How to conduct How to conduct fMRI?fMRI?
Huettel, Song, Gregory. Functional Magnetic Resonance Imaging. Sinauer.
Stimulus Timing
What does fMRI What does fMRI measure?measure?
Neural activity Signalling Vascular response
Vascular tone (reactivity)Autoregulation
Metabolic signalling
BOLD signal
glia
arteriole
venule
B0 field
Synaptic signalling
Blood flow,oxygenationand volume
dendriteEnd bouton
fMRI signal originfMRI signal originrest
Oxygenated Hb
Deoxygenated Hb
task
Oxygenated Hb
Deoxygenated Hb
Haemodynamic Response Function Haemodynamic Response Function
(HRF)(HRF)
Stimulus Timing Canonical HRF Predicted Data
=
=
Time
Time
Which part of the brain is Which part of the brain is activated?activated?
http://www.fmrib.ox.ac.uk/
Time Time
Actual measurement Prediction
Sig
nal
Inte
nsi
ty
Deconvolve HRF Deconvolve HRF hh
10
0
-10
1
0100 200 300 400 500
0
0.5
1
100 200 300 400 500
0
1
2
100 200 300 400 500-10
0
10
100 200 300 400 5000
0.5
1
100 200 300 400 500
0
1
2
100 200 300 400 500-10
0
10
100 200 300 400 5000
0.5
1
100 200 300 400 500
0
1
2
100 200 300 400 500-10
0
10
100 200 300 400 5000
0.5
1
100 200 300 400 500
0
1
2
100 200 300 400 500-10
0
10
y
DhD: convolution matrix
D(: , 1)
1 2
3
1
T
minimize
s.t.
(1) ( ) 0
0
hWh y Dh
h
h h n
E h
W: CoifletE: monotone cone
1 32
canonical HRFdeconvolved HRF
(d)
WhDiscrete wavelet transform
Deconvolve HRF Deconvolve HRF hh
10
0
-10
1
0100 200 300 400 500
0
0.5
1
100 200 300 400 500
0
1
2
100 200 300 400 500-10
0
10
100 200 300 400 5000
0.5
1
100 200 300 400 500
0
1
2
100 200 300 400 500-10
0
10100 200 300 400 500
0
0.5
1
100 200 300 400 500
0
1
2
100 200 300 400 500-10
0
10
y
stimulustiming
1 2
3
1
T
minimize
subject to
(1) ( ) 0
0
hWh y Dh
h
h h n
E h
W: Coiflet waveletE: monotone coneD: convolution matrix measurement
smoothness
100 200 300 400 5000
0.5
1
100 200 300 400 500
0
1
2
100 200 300 400 500-10
0
10
Dh
D( : ,
1)
Deconvolution resultsDeconvolution results
10 canonical canonical20 log /h h h
0 5 10 15 20 25 30 35-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
impulse (evenly spaced) = -10.6dBimpulse (jittered) = -6.2dBBlock = -11.8dBCanonical HRF
Low noise simulation: SNR= 6dB
0 5 10 15 20 25 30 35-1.5
-1
-0.5
0
0.5
1
1.5
impulse (evenly spaced) = -6.7dBimpulse (jittered) = -4.4dBBlock = -3.5dBCanonical HRF
High noise simulation: SNR = -10dB
in vivoin vivo deconvolution deconvolution resultsresults
0 5 10 15 20 25 30
-0.5
0
0.5
1
frame
RO
I val
ue
file: C:\Documents and Settings\THE SHEEP\My Documents\rawdata\Nov11/measure_hrf/hrf roi loc: 38 48 13 22 5:
frame
DICOMname = ../../../rawdata/Nov11/002/I ccfname = ../../../rawdata/Nov11/correl/hrf
p<0.350000 t-statistic = 0.5057 cc-statistic= 50.5 total # activated pixels = 162
0
DICOMname = ../../../rawdata/Nov11/002/I ccfname = ../../../rawdata/Nov11/correl/hrf
p<0.350000 t-statistic = 0.5057 cc-statistic= 50.5 total # activated pixels = 162
0
0 5 10 15 20 25 30
-0.5
0
0.5
1
frame
RO
I va
lue
DICOMname = ../../../rawdata/Nov10/002/I ccfname = ../../../rawdata/Nov10/correl/block
p<0.050000 t-statistic = 2.0725 cc-statistic= 145.0 total # activated pixels = 93
0 2 4 6
…
HRF calibration
…
HRF deconvolution
…
HRF deconvolution
Time (s) Time (s)
Cardinality Constraint Problem
3
5
Ab3
1
x5
1
A = 0.29 0.47 0.62 0.16 0.24
0.75 0.51 0.89 0.52 0.42
0.82 0.41 0.20 0.86 0.79
0.29 0.62 0.16
0.75 0.89 0.52
0.47 0.24
0.51 0.42
0.41 0.70. 982 0.20 0.86
b = A(:,2) * rand(1) + A(:,5) * rand(1)
Find Find xx with desired with desired cardinalitycardinality
e.g. e.g. kk = 2, = 2, wantwant 2
5
0
0
0
x
x
x
0want 2x
4
2
5
0
0
x
x
x
x = A\b 1arg min
s.t.
x x
Ax b
2
4
5
0
0
x
x
x
=
†A b 2
arg min
s.t.
x x
Ax b
1
2
3
4
5
x
x
x
x
x
m=3
n=5
Ab3
1
x5
1
Check every pair for k = 2Possible # solution:5
2
In general, !
! ( )!
n n
k k n k
From the Range perspective From the Range perspective ……
3
5
A
2
5
= 0
Nullspace PerspectivePerspective
T1ZT2ZT3ZT4ZT5Z
Z
Tpx Z xi ii Tp 0Z x ii
p
p p( )
x Z x
Ax A Z x Ax b
px = A\bParticular soln.
General soln.
Z = 0.57 0.34 -0.03 -0.26 -0.16 0.02 -0.68 -0.63 0.22 0.47
xp = 0.34 1.11 -0.30 0 0
-4 -2 0 2-1
0
1
2
3
4
5
6
1
2
1 2
1 2
1 2
1 2
1 2
0.57 + 0.34 + 0.34 0
-0.03 - 0.26 1.11 0
-0.16 + 0.02 0.30 0
-0.68 - 0.63 0 0
0.22 + 0.47 0 0
T
1...5
p
i
x Z xi ii
How to find intersection of How to find intersection of lines?lines?
Sum of normalized wedges
1T1
p
T
1n
Z
Z x
Z
(0, 0)
x = 0.34 1.11 -0.30 0 0
2
5
0
0
0
x
x
x
(-1.16, 4.51)
x = 0 0.68 0 0 0.51
Z = -0.59 -0.36 0.06 -0.55 0.15 0.36 0.74 -0.08 -0.27 0.66
Znew = -0.41 -0.18 -0.80 -0.30 0.48 0.19 -0.26 -0.07 1.00 0.37
-5 0 5-4
0
4
Znew = Z * randn(2);
-2 -1 00
2
4
6
8
SummarySummary
Ways to find 0-norm solutions other Ways to find 0-norm solutions other than than
1-norm (homotopic, convex iteration)1-norm (homotopic, convex iteration) fewer measurementsfewer measurements fasterfaster
In cardinality constraint problem, In cardinality constraint problem, convex iteration and nullspace convex iteration and nullspace technique success more often than 1-technique success more often than 1-norm.norm.