2

Sparse Image Reconstruction

2.1. Introduction

In nano MRI, the readback image is a blurred and noisyversion of the pristine image. The latter term denotes theobject (molecule) of interest. Image reconstruction techniqueshave been applied in a variety of fields: for example, MRI,astronomy and image deblurring. The particularity of imagereconstruction in nano MRI is that the pristine image issparse, something that also holds true for astronomy. Byusing an algorithm that takes into account the sparsity of thepristine image, we can obtain a better reconstructed image.Making use of a constraint reduces the search space of thereconstructed image: it is therefore advantageous to use allpossible constraints.

The pristine image will be assumed to be sparse. ForMRFM, in particular, the non-zero image values are assumedto be non-negative valued for a reason mentioned later insection 2.3. It might be possible to impose a stricter conditionon the non-zero values: for example, they may come from afinite set. This stricter condition will not be explored in thiswork. Additive white Gaussian noise will be assumed for theobservation noise.

Molecular Imaging in Nano MRI© ISTE Ltd 2014. Published by ISTE Ltd and John Wiley & Sons, Inc.

, Michael Ting.

8 Molecular Imaging in Nano MRI

There are a number of methods that address sparse imagereconstruction. These methods can be loosely classified aseither non-Bayesian or Bayesian. The former consists ofmethods that do not provide a probabilistic interpretation ofconstraints, if they do exist, on the image, noise level, etc. Thelatter consists of methods that establish probability prior(s)on the unknown parameters. Many algorithms can beformulated from a non-Bayesian perspective as well as from aBayesian perspective; the two interpretations complementeach other and shed light on the algorithm’s behavior.

2.2. Problem formulation

By enumerating a 2D or 3D array in an order (e.g.,column- or row-major order), we can equivalently represent a2D or 3D image by a vector. Without loss of generality,unnecessarily, denote by x ∈ R

M the pristine image and byy ∈ R

N the observation. Let yi ∼ N (hTi x, σ

2), where N (μ,Σ)denotes the Gaussian distribution with mean μ andcovariance matrix Σ. The observation model can be writtenas

y = Hx+ w, w ∼ N (0, σ2I), [2.1]

where H (h1| . . . |hN )T ∈ RN×M represents the system psf.

If H were orthonormal, then [2.1] would degenerate into adenoising problem. Define the signal-to-noise ratio (SNR) as:SNR 10 log10( Hx 2

2/σ2).

The basic version of the sparse image reconstructionproblem is: given H, σ2 and y, estimate x knowing that it issparse. With nano MRI, the image sparsity is in the standardbasis of R

M . In other applications where sparsity exists inanother domain, e.g. some wavelet basis, it is possible totransform the observation model to [2.1]. More advancedversions of the problem include the case when: (1) σ2 is

Sparse Image Reconstruction 9

unknown; (2) H is approximately known and σ2 is unknown;or (3) both H and σ2 are unknown.

The distribution of non-zero values of x is an importantaspect in the problem formulation. An image reconstructionalgorithm that uses the correct non-zero distribution wouldbe expected to perform better than one that does not. Ingeneral, the non-zero xi can assume any non-zero value. Inthe case when the non-zero xi comes from a subset of R\{0},an algorithm that does not make use of this informationwould be expected to have poorer performance than thealgorithm that does. In short, the reconstruction algorithmshould make use of hypotheses that match the problem, nomore and no less. This, however, may not always be known a

priori, so a mismatch can occur.

Another salient aspect of the problem is the nature of thematrix H. For nano MRI, an increased number of observationscan be expected to increase cost or processing time, so ideallyM would be small. On the other hand, in order to achieve goodresolution, N can be expected to be large. If M < N , [2.1] withσ2 = 0 would be an underdetermined system.

2.3. Validity of the observation model in MRFM

In experiments that use the iOSCAR detection protocol,the measurements yi are taken according to the schematicillustrated in Figure 2.1. As can be seen, yi is a filteredenergy statistic. The clock signal used in the generation ofthe in-phase and quadrature-phase signals sI(t) and sQ(t),respectively, comes from the pulses of the rf field B1(t). Thesources of noise in the measurements include: phase lock loop(PLL) noise, interferometer noise and system thermal noise.The major noise contributor is the PLL, and its phase noisecan be characterized as approximately narrowband Gaussianaround the bandwidth of interest.

10 Molecular Imaging in Nano MRI

LPF

LPF

+−

interferometer

measurementPLL

phaseshift

CLK

(·)2

fs

fs yi

s(t)

sQ(t)

sI(t)

-90◦

(·)2

Figure 2.1. Schematic of energy-based measurements for image

reconstruction under iOSCAR

Consider a simplified analysis of the noise at the output ofFigure 2.1. The assumptions that will be made are: (1) thenoise variance in the samples of the in-phase andquadrature-phase signals is unity, (2) there are no randomspin flips and (3) ignore the effect of the lowpass filter (LPF),which introduces correlation across the samples of eachbranch. Let Gi, 1 ≤ i ≤ M denote the independent andidentically distributed (i.i.d.) Gaussian random variables(r.v’s) Gi ∼ N (0, 1). The quadrature-phase lower branch afterM samples equals M

i=1G2i ∼ χ2

M . Let δi be the responseinduced by the iOSCAR protocol: if no spin is present, δi ≡ 0,whereas in the presence of a spin, δi is a telegraph signal. LetJi ∼ N (0, 1) be the noise in the in-phase upper branch, whereGi, Ji are independent. The upper branch after M samplesequals M

i=1(Ji + δi)2 ∼ χ2

M (λM ), which is a non-centralchi-squared r.v. with non-centrality parameter λM

Mi=1 δ

2i .

Sparse Image Reconstruction 11

As M → ∞, χ2M →d N (M, 2M), whereas a normal

approximation for χ2M (λ) as M → ∞ is

N (M + λM , 2(M + 2λM )). The difference between the upperand lower branches then is approximately N (λM , 4(M + λM ))as M → ∞. In the low SNR regime, M λM , andvar(yi) ≈ 4M . This is consistent with [2.1]. In the high SNRregime, however, var(yi) would not be approximately equalfor all i , so [2.1] would not be a good representation.

2.4. Literature review

A complete review of sparse estimation algorithms isbeyond the scope of this text. Instead, a sampling of somerelevant works is presented in order to highlight usefullearning and insight. The sparse image reconstructionproblem is related to three other extensively studiedproblems: (1) sparse denoising, (2) variable selection in linearregression and (3) CS.

2.4.1. Sparse denoising

As previously mentioned, a special case occurs when H isorthonormal. In this case, the sparse reconstruction problemsimplifies into the sparse denoising problem. The latter hasappeared in the context of wavelet regression, where onewould like to estimate an unknown function innoise [JOH 98, CLY 99]. The empirical Bayes methodin [JOH 98] used the sparse prior

xi|w, ν2 i.i.d.∼ (1− w)δ(xi) + wφ(xi; 0, ν2). [2.2]

A later publication by the same authors replaced thenormal distribution in [2.2] with a Laplaciandistribution [JOH 04], leading to

xi|w, a i.i.d.∼ (1− w)δ(xi) + wγL(xi; a), [2.3]

12 Molecular Imaging in Nano MRI

where γL(x; a) = (1/2)ae−a|x| is the Laplacian probabilitydensity function (p.d.f.) with shape parameter a. An empiricalBayes method using [2.3] achieves performance that waswithin a constant of the asymptotic minimax error undercertain conditions [JOH 04]. Henceforth, refer to prior [2.3]as the Laplacian with atom at zero (LAZE) prior. Other p.d.f ’sbesides the Laplacian can be used with the same asymptoticresult, so long as the following properties hold true: (1) thep.d.f. is heavy tailed, (2) it is unimodal and symmetric and (3)it satisfies some regularity conditions.

2.4.2. Variable selection

Variable selection is an old topic in statistics, where weneed to choose the independent variables (covariates) thatare used to explain the observation. A plethora of methodsexist, from subset selection, ridge regression, the lasso andthe elastic net.

2.4.3. Compressed sensing

CS is a newer topic that is very closely related to variableselection. The former has its genesis in the representation ofa signal in an overcomplete basis. When no noise is present(σ2 = 0), problem [2.1] reduces to finding the sparsestrepresentation of y in terms of the columns of H

P0 : minimize x 0 such that y = Hx, [2.4]

where · 0 denotes the vector l0 “norm”, defined as x 0

#{i : xi = 0}. The solution to P0 requires an enumerativeapproach that is exponential in M . A convex relaxation of P0that has lower computational complexity is

P1 : minimize x 1 such that y = Hx, [2.5]

Sparse Image Reconstruction 13

where the vector lp norm for p ≥ 1 is defined asx p ( i |xi|p)1/p. Under certain conditions, the solution of

P1 is equivalent to that of P0.

In traditional CS, the columns of H have low mutualincoherence. This is not necessarily the case for nano MRI: itdepends on the psf. In the realistic case where noise ispresent (σ2 > 0), the equivalence guarantee of P1 to P0 nolonger holds. Under certain conditions, however, P1’s solutionis close to P0’s solution [TRO 07].

Several algorithms that are used in compressed sensinginclude orthogonal matching pursuit (OMP) [TRO 07,CAI 11], stagewise OMP [DON 06], and sparse Bayesianlearning (SBL) [WIP 04].

2.5. Reconstruction performance criteria

Given the pristine image x and its reconstruction x̂, thereconstruction error can be defined in various ways [TIN 09].Although the conventional way of assessing reconstructionerror is via the l2 norm, this choice may not provide a goodassessment of how “close” x̂ is to x. As a result, the “detectionerror” criterion is used to assess whether x̂ has the same zeroand non-zero locations as x. Finally, the sparsity of x̂ isassessed with respect to x. The three reconstruction criteriaused for the simulation study are listed in Table 2.1. Thevalid range of each criterion is given in the third column,assuming that x = 0.

To give a sense of what values can be expected of areasonably performant reconstructor, the error criteria areevaluated in the case of the trivial zero reconstructor, wherex̂ = 0, and in the case of the perfect reconstructor, wherex̂ = x. These results are given in Table 2.2.

14 Molecular Imaging in Nano MRI

Description Definition Valid

range

Normalized l2 error norm x− x̂ 2/ x 2 [0,∞)a

Normalized detection error M

i=1 xor(I(xi = 0), I(x̂i = 0))/ x 0 [0,M ]a

Normalized l0 norm x̂ 0/ x 0 [0,M ]a

aAssuming that x is non-trivial, i.e. x = 0.

Table 2.1. Criteria used to evaluate the reconstruction performance.

Description Zero reconstructor Perfect reconstructor

Normalized l2 error norm 1 0

Normalized detection error 1 0

Normalized l0 norm 0 1

Table 2.2. Criteria values for the zero and perfect reconstructor

Thus, while the normalized l2 error norm might have avalid range of [0,∞), a good reconstructor should attain avalue in the more restrictive range [0, 1]. A similar reasoningcan be applied to the normalized detection error. The idealnormalized l0 norm, unity, is not an endpoint of the validrange.