Hybrid Compressed Sensing of ImagesAbdolreza Abdolhosseini Moghadam and Hayder Radha

Dep. of Electrical and Computer Engineering, Michigan State University, Email: {abdolhos,radha}@msu.edu

Abstract—We consider the problem of recovering a sig-nal/image (x) with a k-sparse representation, from hybrid (com-plex and real), noiseless linear samples (y) using a mixture ofcomplex-valued sparse and real-valued dense projections withina single matrix. The proposed Hybrid Compressed Sensing (HCS)employs the complex-sparse part of the projection matrix to di-vide the n-dimensional signal (x) into subsets. In turn, each subsetof the signal (coefficients) is mapped onto a complex sample ofthe measurement vector (y). Under a worst-case scenario of suchsparsity-induced mapping, when the number of complex sparsemeasurements is sufficiently large then this mapping leads to theisolation of a significant fraction of the k non-zero coefficientsinto different complex measurement samples from y. Using asimple property of complex numbers (namely complex phases)one can identify the isolated non-zeros of x. After reducing theeffect of the identified non-zero coefficients from the compressivesamples, we utilize the real-valued dense submatrix to form afull rank system of equations to recover the signal values inthe remaining indices (that are not recovered by the sparsecomplex projection part). We show that the proposed hybridapproach can recover a k-sparse signal (with high probability)while requiring only m ≈ 3k 3

√n/2k real measurements (where

each complex sample is counted as two real measurements). Wealso derive expressions for the optimal mix of complex-sparse andreal-dense rows within an HCS projection matrix. Further, in apractical range of sparsity ratio (k/n) suitable for images, thehybrid approach outperforms even the most complex compressedsensing frameworks (namely basis pursuit with dense Gaussianmatrices). The theoretical complexity of HCS is less than thecomplexity of solving a full-rank system of m linear equations.In practice, the complexity can be lower than this bound.

Index Terms—Compressed sensing, sparse projections, itera-tive decoding algorithms

I. INTRODUCTION

Over the past few years, Compressed Sensing [1], [2] hasbeen introduced as an alternative, efficient method for sam-pling signals which have sparse representations with respectto some bases. Consequently, some compressive samplingschemes for sensing images [12], [13] have been proposed,exploiting the fact that images have sparse or compressiblerepresentations in some known bases (such as DCT, Con-tourlets and so on [14]-[16]). Consider a n-dimensional, real-valued signal (e.g. an n pixel image) x which has a k-sparserepresentation θ in the transform domain ψ, that is:

x ∈ Rn, x = ψθ, k = ‖θ‖0 := {i : θi �= 0} (1)

The main idea of Compressed Sensing (CS) [1], [2] isthat instead of sensing n samples xi, the encoder projectsx onto m compressive samples y according to a projection

MMSP’10, October 4-6, 2010, Saint-Malo, France. 978-1-4244-8112-5/10/$26.00 2010 IEEE

matrix P , i.e. y = Px, where k < m < n. Accordingly, thedecoder recovers the sparse representation θ by utilizing thecompressive samples y and the effective projection matrix Pψand thereafter reconstructs x. Generally the under-determinedsystem of equations y = Px = Pψθ has an infinite set ofsolutions. However, if certain properties outlined in [1], [2] aremet then it is possible to find θ (and consequently x = ψθ)uniquely using the following optimization problem:

θ = argmin ‖θ̂‖0 = argmin ‖θ̂‖1 : y = Pψθ̂ (2)

Under early CS solutions, the complexity of the solver hasan inverse relationship to the number of compressive samplesrequired. For instance, greedy algorithms [3]-[5] have lowcomplexities but require relatively large numbers of samples;on the other hand, convex relaxation (�1 optimization) algo-rithms [6] recover the signal (or transform coefficients of animage) with fewer samples but they are extremely complex(Ω(n3)) relative to greedy algorithms. Furthermore, in caseof images, these CS methods suffer from what is known asphase transition [19]. In words, early CS solutions recover asparse/compressible signal when the sparsity ratio (k/n) is insome specific range. Outside this range, the probability of thesignal recovery (under that specific solver) drops significantly.Typically for early CS solutions, this ratio is small. Although,images may have sparse/compressible representations in somebases, in practice, the sparsity ratio of an image is (usually)outside the optimal range of these CS solutions. Hence, forimages, a CS framework capable of operating under moderatesparsity is desirable. This led to many efforts with promisingresults that demonstrated improvements over traditional CSapproaches [3], [7]-[11].

More recently, there has been some interest in pursuingsparse projections that lead to low complexity and low mea-surement requirements [8]-[11]. Overall, state of the art sparseapproaches have shown to achieve recovery (with high proba-bility) using m = O (k log(n/k)) measurements and with timecomplexity O (n log(n/k)) [10]. Despite the clear benefits ofsparse projections, one needs to use rather strict structures(e.g., expander graphs) to achieve these bounds simultaneously[7]-[10]. Even then, low values for the constant (c) of themeasurement bound m = O (k log(n/k)) = ck log(n/k)cannot be guaranteed. In this context, using traditional denseGaussian matrices requires minimal measurements, however,at the expense of a high complexity �1 minimization decodingalgorithm.

In this paper, we present a Hybrid Compressed Sensing(HCS) framework, which employs two classes of (sparse anddense) sensing projections to solve the problem of compressive

978-1-4244-8112-5/10/$26.00 ©2010 IEEE 99

sampling in a novel way. More specifically, the HCS projec-tion matrix P is composed of two sub-matrices: a sparse,complex-valued sub-matrix P (c) and a random dense, real-valued sub-matrix P (r). As demonstrated below, the sparsecomplex-valued projection provides a natural mechanism forlowering the solver complexity. Meanwhile, the dense real-valued projection can facilitate the lowering of the samplingrequirement for the recovery of the signal. Hence, in thecontext of the proposed hybrid sparse-dense framework, weapproach compressed sensing as an optimization problem byidentifying the best mix of sparse-dense random projectionsthat minimize the total number of samples required for therecovery of a k-sparse signal x.

The sparse complex-valued sub-matrix P (c) of HCS canemploy any underlying structure for its non-zero entries,including structures that are based on combinatorial designsor expander graphs [8]-[10]. Here, we focus on a worst casescenario (in terms of required measurements) by employinga simple structure that divides the signal coefficients intoa number of non-overlapping sub-signals (i.e. partitions). InSection II, we show, that when the number of sub-signals (orequivalently the number of rows of P (c)) is sufficiently large,then this partition (of coefficients) leads to the isolation ofmost of the k-non-zero transform coefficients into differentcompressive samples. Hence, one can solve for the non-zero coefficients of x in a “divide-and-conquer” strategy.After recovering these isolated non-zero transform coefficients(utilizing a simple property of complex numbers), one mightuse basic linear algebra methods to recover the remainingtransform coefficients utilizing the random dense sub-matrixP (r). In Section III, we show that (with high probability) HCSrequires m ≈ (3 + δ)k 3

√n/k compressive samples to recover

a k-sparse signal of length n where δ � 1. Although HCS isnot optimal (in terms of sample requirements for the perfectrecovery), it can reconstruct a signal from fewer samplescompared to the most complex CS framework (i.e., basispursuit with dense Gaussian matrices) in a practical range ofthe sparsity ratio suitable for images. Moreover the complexityof HCS is low due to the solver simplicity. Specifically, thecomplexity of the HCS solver (in the worst case) is lessthan the complexity of solving a (full-rank) system of mlinear equations. Section IV presents our simulation resultsand Section V concludes this paper.

II. HYBRID SPARSE-DENSE PROJECTIONS

For the simplicity of notations and without loss of general-ity, throughout this paper, we assume that the signal/image xis sparse itself, that is the sparsifying domain ψ in (1) is theidentity matrix (hence θ = x). Similarly, it would be beneficialif consider the vectorized form of an image. By vectorizedform, we mean that we stack the columns of the image on topof each others, that is:

l = (j − 1)n1 + i : xl := xi,j (3)

where xi,j is the pixel located at the Cartesian location(i, j). A sparse-projection based method naturally projects a

subset xMiof the signal x (xMi

⊂ {x1, . . . , xn}) into anobservations yi, where yi ∈ {y1, . . . , ym} and Mi is theset of column indices for the non-zero entries of the ith

row of a sparse projection matrix P . For example, if theith row of a sparse projection matrix P has zero entrieseverywhere except for non-zero entries in some locations(column indices), say Mi = {3, 8, 31}, then we have acorresponding subset xMi

= {x3, x8, x31}. Consequently, theith compressive sample yi is a function of the subset xMi

(yi = gi(xMi)), and yi does not depend on the values of

other elements of the signal x. Overall, the natural division ofthe signal x (into some subsets) by a sparse projection leadsto low-complexity recovery algorithms since one can beginby recovering one or few subsets of x from a correspondingsubset of observations, and then progressively recover moresubsets of x. Due to their attributes, sparse projections havebeen receiving more attention recently [7]-[11]. One of the keycontributions of this work is extending the utility of sparseprojections by 1) using complex-valued entries with simplestructures instead of binary-valued projections with rathercomplicated and restrictive structures and 2) complimentingthem with important aspects of dense sensing as explainedfurther below. More importantly, as opposed to most existingCS solutions, which rely on solely sparse or merely densematrices, our proposed method employs the features of thesetwo classes of matrices simultaneously to design a simple yetefficient solution for CS.

The following notation will be used in this paper. Let A bea b × c matrix and let i ⊆ [b] = {1, . . . , b} and j ⊆ [c]. AT

represents the transpose of A. By Ab, and A,c we respectivelymean submatrices of A given by restricting A on rows andcolumns indexed by those members of b and c. Moreover letD be a d×1 vector and let e ⊆ [d]. Then D̂e is a d×1 vectorgiven by keeping the entries of D indexed by e and settingthe rest (indices of [d]\e) to zero.

A. HCS Projection Matrices

The projection matrix (P ) of HCS for a length n signalis composed of two sub-matrices (P (c)) and (P (r)): PT =

[(P (c)

)T (P (r)

)T], where P (r) is a mr × n real-valued,

dense random matrix and P (c) is a mc × n sparse matrixwith random complex entries (in section III, we find theoptimal values for mc and mr under an extreme case tobe explained). If we count each complex valued compressivesample as two real samples, then the projection matrix issensing 2mc + mr measurements. For simplicity of analysisand without loss of generality assume mc counts n. Now letM = {M1, . . . ,Mmc

} be a set of equal size, random subsetsover [n] = 1, 2, . . . , n such that M forms a partition. Thus:∀i, j ∈ [n], i �= j : |Mi| = w,Mi ∩ Mj = ∅,∪mc

i=1Mi = [n].Then the ith row of P (c) is non-zero, only in the columnindices of Mi. Also let f (i) be a 1 × w row vector withrandom complex entries having distinct phases in the rangeof [0, π). Then we insert the entries of f (i) in the ith row

We say that the ith measurement sample spans the indices of Mi.

100

of P (c) in the column indices of Mi. This can be writtenas: ∀i ∈ [mc] : P

(c)i,Mi

= f (i) and P(c)i,[n]\Mi

= 0. Thus the ith

sample (where i ∈ [mc]) is a function of the signal coefficientsonly in the indices of Mi: yi = f (i)xMi

. Appending mr

real-valued, dense random rows to this matrix yields the finalprojection matrix.

B. HCS Solver

We first highlight an important (yet arguably clear) propo-sition, which is critical for the successful working of theHCS solver: if a subset xMi

of the coefficients of x has atmost one non-zero coefficient, then only one complex-valuedcompressive sample yi is sufficient for the recovery of thatsubset.

Proposition 1. Let Mi ⊂ [n], |Mi| = w be the (ordered)set of indices of non-zero coefficients for the ith row of asparse matrix P (c). And let P

(c)i,Mi

= f (i) = [ ejφ(i)1 ... ejφ

(i)w ]

be the corresponding vector of complex exponentials (on theunit circle) such that the random phases of f (i) are distinctin the range of [0, π), ∀l, t ∈ [w], l �= t : φ

(i)l ∈ [0, π), φ

(i)t �=

φ(i)l . If xMi

is real-valued and non-zero in at most one index(‖xMi

‖0 ≤ 1) then having yi = P(c)i, x = f (i)xMi

and f (i),one can recover xMi

deterministically. Moreover having yiand f (i), we can detect the event that xMi

is non-zero inat least two indices. Specifically if yi = 0 then xMi

= 0.If the phase of yi equals to the phase of lth entry of f (i)

(∠yi = ∠f (i)l = φ

(i)l ) then xMi

is non-zero only in the indexof Mi,l and has the value of xMi,l

= yi/f(i)l = ‖yi‖. If the

phase of yi is not among the phases of f (i) ({φ(i)1 , . . . , φ

(i)w }),

then xMiis non-zero in at least two indices. The proof is

presented in the Appendix.The HCS solver consists of two major stages: 1) finding and

recovering subsets of the signal coefficients spanning at mostone non-zero coefficient and 2) forming a full-rank systemof linear equations and solving it. For the first mc complexvalued compressive samples (yi, i ∈ [mc]), the solver looksfor the phase of each compressive sample (∠yi) among theavailable phases in the associated random vector. As beforelet f (i) = P

(c)i,Mi

, w = |Mi| and ∠f (i) = [∠f (i)1 . . . ∠f (i)

w ].Assume the phase of sample (yi) equals to the phase of tth

entry of f (i). Then by Proposition 1, xMiis zero in all indices

except in the index of Mi,t. This can be written as:

∠yi = ∠f (i)t ⇔ xl =

{‖yi‖ l = Mi,t

0 l ∈ Mi\t

}(4)

Moreover, if the compressive sample yi is zero, then xMiis

a zero vector. Otherwise (if the sample is neither zero norits phase is found in ∠f (i)), this subset of the coefficients(xMi

) spans at least two non-zeros and Proposition 1 is notapplicable to this subset. Define I ′ as the set of sample indices,such that each of these samples is either zero or the phase ofthat sample can be found among the available phases in therespective random vector:

I ′ ={i ∈ [mc] : yi = 0 or ∠yi ∈ ∠f (i)

}(5)

Thus the first stage of the HCS solver determines signal valueson indices of I = ∪i∈I′Mi. Consequently P x̂I is the effectof identified coefficients on the compressive samples y; andby subtracting P x̂I from the compressive samples y we arenullifying such effect: y − P x̂I = P x̂Ic where Ic is theset complement of I (I ∪ Ic = [n]). If we consider the nsignal coefficients as unknowns and compressive samples asequations, the problem of y = Px can be viewed as a systemof n unknowns and m = 2mc + mr equations where |I|coefficients have already been recovered (in the first stage ofthe solver). Having n − |I| further (independent) equationsone can recover the remaining unknowns. The random densesubmatrix (P (r)) and complex samples spanning at least twonon-zeros provide us such equations.

III. ANALYSIS OF THE HCS SOLVER

In this section, we derive the sample requirements of HCS(under perfect recovery) and also the complexity of suchprocedure. To that end, we first find the average number ofsamples required for the perfect recovery and then using thetools from the concentration of measurement [17] we showthat our derived bound is tight.

Recall that complex compressive samples span disjointsubset of coefficients. Hence it is straightforward to see thatthe distribution of number of non-zeros spanned by each sam-ple is multinomial. The problem of finding this multinomialdistribution can be casted as the classical “balls into bins”problem [17]. Hence probability of the event that a complexcompressive sample (say yi, i ∈ [mc]) spans j non-zero(s) is:

Pr (‖xMi‖0 = j) =

(k

j

)(1− 1

mc

)k−j (1

mc

)j

(6)

Consequently, the expected number of samples spanning atmost one non-zero coefficient (|I ′|) for large values of k is:

E(|I ′|) = mc Pr (‖xMi‖0 ≤ 1) ≈ mc

(1 +

k

mc

)e−

kmc (7)

Since each complex sample spans w = n/mc coefficients, thenumber of identified coefficients (I) on average is E(|I|) =wE(|I ′|) = n(1 + k/mc) exp (−k/mc). Note that we havenot utilized mc − I ′ complex samples in the first stage of thealgorithm (since these samples span at least two non-zeros).Counting each complex sample as two equations, we needat least mr ≥ n − E(|I|) − 2 (mc − I ′) further equations toform a full rank system of equations to recover the remainingcoefficients. To identify the required number of measurements,this problem can be casted as the following optimizationproblem:

argminmc,mr

2mc +mr s.t. mr ≥ n−E(|I|)− 2 (mc − I ′) (8)

The following two lemmas outline key results for solvingthis problem and determine the sample requirements of theHCS solver. The first lemma provides expressions for thenumbers of complex samples (mc) and real samples (mr) that

101

20 40 60 80 100 120

�0.4

�0.2

0

0.2

0.4

Target Signal=Doppler, n=120, k=20 Noiseless

20 40 60 80 100 120

�0.4

�0.2

0

0.2

0.4

HCS, m=50, MSE=0.000000, Time=0.006031

20 40 60 80 100 120

�0.5

0

0.5

BP, Gaussian Projection, m=50, MSE=0.015769, Time=0.044851

20 40 60 80 100 120

�0.5

0

0.5

1

OMP, Gaussian Projection, m=50, MSE=0.024785, Time=0.011491

1000 2000 3000 40000

2

4

Target Signal=Blocks, n=4800, k=800 Noiseless

1000 2000 3000 40000

2

4

HCS, m=2400, MSE=0.000000, Time=6.450577

1000 2000 3000 40000

2

4

BP, Gaussian Projection, m=2400, MSE=0.000037, Time=375.307032

1000 2000 3000 4000

�5

0

5

10

OMP, Gaussian Projection, m=2400, MSE=0.028259, Time=416.847072

500 1000 1500

0

0.05

0.1

0.15

0.2

Target Signal=Leopold, n=1800, k=200 Noiseless

500 1000 1500

0

0.05

0.1

0.15

0.2

HCS, m=720, MSE=0.000000, Time=0.224265

500 1000 1500

0

0.05

0.1

BP, Gaussian Projection, m=720, MSE=0.000126, Time=6.620905

500 1000 1500

0

0.05

0.1

OMP, Gaussian Projection, m=720, MSE=0.000266, Time=11.789665

Fig. 1. m, n and k represent number of samples, signal length and signal sparsity respectively.

are required for the recovery of x under HCS. The secondlemma simplifies these expressions and show that the HCSmeasurement bound is tight. A corollary is also stated below.(See the appendix for a sketch of the proof.)

Lemma 1. Consider a k-sparse signal x of length n anddefine l = n/k. Then on average, HCS requires mc ≈k

(3

√l2 − 3

√16729l −

13

)complex-valued samples and mr =

n− nα− 2αmc real-valued samples for the perfect recoveryof x where α = (1 + k/mc) exp(−k/mc).

Lemma 2. If n(2(α− α2)2 ln k

)3 � k4 then with a prob-ability of at least 1 − 2

k , HCS requires m = 2mc + mr

compressive samples for the perfect recovery of x wheremc = k(1 + δ) 3

√n2k , mr = n− nα− 2αmc and 0 < δ � 1.

Corollary 1. If 3√2l2 ≤ 2 + 1/(1− α) then mr ≤ mc.

Although the HCS sampling bound is not optimal for allvalues of k and n, HCS outperforms even the most complexCS solutions over a wide range of typical and practical valuesof k and n (e.g., as long as k/n is not excessively small).Specifically let mopt = Ck log n/k be the sample requirementfor an optimal CS framework where C is a constant onlydepending on the projection matrix and the solver. If 3

√n/k <

1.25C log(n/k), then HCS requires fewer samples for perfectrecovery when compared with the sampling requirement ofthe hypothetical optimal solver. Now let us compute thecomplexity of the HCS solver. Recall that the HCS decodingprocess consists of two stages. In the first stage, for eachcomplex valued compressive sample (yi, i ∈ [mc]), we lookfor the phase of that sample (∠yi) among the available phasesin the respective random vector ∠f (i). Assume the entriesof f (i) = [exp(jφ

(i)1 ) exp(jφ

(i)2 ) . . . exp(jφ

(i)w )] are sorted

based on phases: a, b ∈ [w], a > b ⇔ ∠f (i)a = φ

(i)a > f

(i)b =

∠φ(i)b . Then the complexity of this search for each compressive

sample is only log(w). Since we have to repeat this search forall complex valued compressive samples, thus the complexityof the first stage of the solver is O(mc log(w)).

As before, let |I ′| ≥ 0, denotes the number of compressive

samples spanning at most one non-zero coefficient, then wehave not utilized mc−|I ′| complex sample from the first stageof the solver. The second stage of the HCS solver finds thesolution of a full rank system of mr +mc−|I ′| ≤ 2mc equa-tions. Let g(β) be the complexity of linear solver employed inthe second stage of HCS solver to solve a full rank system of βequations. Then the complexity of the second stage of HCS isO (g(mr +mc − |I ′|)) ≤ O (g(2mc)). Finally the complexityof HCS solver is O (max{mc log(n/mc), g(2mc)}).

IV. SIMULATION RESULTS

We tested the proposed HCS framework on a large numberof standard compressed sensing signals and images. Here,we present the results for images of Monalisa and Lennaand compare our results with two dominant solvers/projectionmatrices (Fig. 2 and Fig. 3). We performed compressivesampling on the whole image of Monalisa, while for Lena, weapplied a block based compressed sensing. More specificallyfor Lena, the target image is formed by keeping the largestk = 8 DCT coefficients in all 8 × 8 blocks and set therest of DCT coefficients to zero. For Monalisa image, a totalof k = 57979 DCT coefficients (with the largest absolutevalue) out of n = 283554 coefficients have been kept andwe set the remaining coefficients to zero. In our simulations,we counted each complex valued compressive sample as tworeal samples. In these figures m, tbp, thcs and tomp representthe number of (real) compressive samples and the decodingtime required for BP, HCS and OMP respectively. In allsimulations, we have assumed that for a given number ofsamples m, the ratio of complex valued compressive samplesand real valued dense samples (for HCS) is approximately one:(mc = �m/3� and mr = m−2mc). It is important to note thatthis sample assignment is not optimal and HCS needs fewersamples for the perfect recovery of the signal (see Lemma 1).However in some real world application, the number of non-zero coefficients might not be known beforehand. AlthoughBP achieved virtually perfect reconstruction, but clearly thecomplexity of BP is much higher compared to HCS. Furthersimulation results in case of 1D signals are presented in Fig.1.

102

Fig. 2. From left to right, the k-sparse image of Monalisa, reconstructed by applying HCS, BP and OMP respectively to the whole image (m = 150746).The decoding times are tbp = 109.5077, thcs = 16.1704 and tomp = 15.1670 seconds.

Fig. 3. From left to right, the k-sparse image of Lenna, reconstructed with m = 28 compressive samples by applying HCS, BP and OMP respectively onblocks of size 8× 8 pixel. The decoding times are tbp = 41.0283, thcs = 4.4735 and tomp = 1.1512 seconds.

V. CONCLUSION

In this paper, we considered the problem of recovering ak-sparse signal x from a limited number of linear samples(y = Px) and proposed Hybrid Compressed Sensing (HCS)to solve this problem. We have shown that HCS recovers xfrom m ≈ (3 + δ)k 3

√n/2k real measurements where δ � 1.

Although this bound is not optimal when compared to thewell-known bound of O (k log(n/k)), in a practical range ofsparsity ratio, it outperforms the most complex CS approachthat utilizes dense Gaussian projections and which is known torequire the minimal number of measurements. To conclude thispaper, we should highlight that the simulation results presentedfor HCS can be improved (arguably significantly). First, in allsimulations, a non-iterative (full rank) linear equation solverhas been deployed in the second phase of HCS. We can reducethe complexity of HCS furthermore by utilizing iterative linearequation solver (such as Jacobi method, Successive OverRelaxation method, etc. [18]) in the second stage of HCS.Moreover, the performance of HCS can be improved further byallowing overlaps and more restrictive structures in the sparsecomplex-valued part P (c) of P . As stated earlier, in this paper,we focused on the worst-case (partition) scenario for P (c).

VI. APPENDIX

Proof of Proposition 1 Clearly if xMiis a zero vector

then yi = 0. Now assume yi = 0. We show that withhigh probability xMi

= 0. Note that if xMiis non-zero

only in index of Mi,l (‖xMi‖0 = 1 and xMi,l

�= 0) thenyi = xMi,l

ejφ(i)l �= 0. Now suppose xMi

is non-zero inat least two indices (‖xMi,l

‖0 ≥ 2). Hence yi is a linearcombination of at least two independent complex numbers.Suppose the values of all entries of xl are bounded by η. Thenthe magnitude of yi is a continuous random variable (at least)in the range of [0, η). Thus the probability of the event thatthe magnitude of yi takes exactly the value of zero, is zero.Hence with high probability: xMi

= 0 ⇐ yi = 0 and thereforexMi

= 0 ⇔ yi = 0. Now assume xMiis non-zero only in the

index of Mi,l. Hence: yi = xMi,lejφ

(i)l . Note that in this case,

the phase of yi (i.e. ∠yi = φ(i)l ) can be found in the ordered

set of available phases in f (i) (∠f (i) = {φ(i)1 , . . . , φ

(i)w }). In

other words, knowledge of the phase ∠yi = φ(i)l translates

into knowledge of the index l associated with the locationof the (single) non-zero coefficient xMi,l

. Now we prove (bycontradiction) that if the phase of yi can be found in ∠f (i) thenwith high probability xMi

is non-zero exactly in one index.

103

Suppose xMiis non-zero in at least two indices and ∠yi can be

found among ∠f (i). By the problem setup, ∠yi is a continuousrandom variable in the range of [0, π). However, we know thatthe probability of the event that a continuous random variable(∠yi) attains the values of a finite set of numbers (∠f (i)), iszero and this is a contradiction. Thus:

∠yi = ∠f (i)l ⇔ xu =

{‖yi‖ u = Mi,l

0 u ∈ Mi\Mi,l

}(9)

Finally if yi �= 0 and ∠yi is not found in the ordered set∠f (i), then xMi

is non-zero at least in two indices. �

Proof of Lemma 1.To that end we solve the following optimization problem:

argminmc,mr

2mc+mr s.t. mr ≥ n−E(|I|)−2 (mc − I ′) (10)

One can use the Lagrangian multiplier to solve (10). Defineα = E(|I ′|)/mc < 1, then the objective function (L) is:

L = 2mc +mr + λ (αn+ 2(1− α)mc +mr − n) (11)

Taking derivatives respect to three parameters λ, mc and mr

gives us three equations:

λ = −1, αn+ 2(1− α)mc +mr = n (12)∂L

∂mc= 0 ⇒ 2m3

c + 2km2c + 2k2mc = nk2 (13)

Define l = n/k. Let us assume l2 � l, then solving (13)yields:

mc = k

(3

√l

2− 3

√16

729l− 1

3

)< k

3

√l

2(14)

Hence we have:

mr = n− nα− 2αmc (15)

Assuming 3√2l2 < 2 + 1/(1− α) implies mr < mc. �

Proof of Lemma 2.We show that deviations from our derived bound is insignifi-cant by following these steps: 1) We use Corollary 5.11 in [17]which in case of our problem, states that the distribution ofnumber of non-zeros spanned by each sample is approximatelyPoisson with mean k/mc. 2) Assuming independent Poissondistributions of non-zero coefficients in compressive samples,we compute the probability that a given sample spans morethan one non-zero and find its variance and mean. Further onecan easily find μ, the average number of samples spanningat least two non-zero coefficients. 3) Assuming independentPoisson distributions of non-zeroes in the compressive sam-ples, let γ be the number of samples spanning at least twonon-zero coefficients, we find the minimum t such that:Pr(γ − μ ≥ t) < 1/k. In words, t (with high probability)is the maximum deviation of γ from μ. Then we show thatunder a realistic presupposition, we have: t � k. 4) ApplyingCorollary 5.11 from [17] to the value of t computed in the step3, we have: in the actual distribution of non-zero coefficientsamong the partitions (i.e. multinomial), the maximum number

of samples spanning at least two non-zeros is less than μ+ twith a probability of higher than 1− 2

k = 1−O(1/k).Let us assume k′ = k′1, . . . , k

′mc

represents the number ofnon-zeros spanned by each complex compressive sample inPoisson case. Let the binary random variable νi be one whenk′i > 1 and zero otherwise. Then Pr(νi = 1) = 1 − α.Hence E(νi) = 1 − α and σ2(νi) = α(1 − α). Defineσ2 =

∑(σ2

i )/mc = σ2i . By Bennet’s inequality [17] we have:

Δ = Pr

(mc∑i=1

(νi − E(νi)) > t

)≤ exp

(−mcσ

2h

(t

mcσ2

))

where h(u) = (1+u) log(1+u)−u > 3u2

6+2u for u ≥ 0. SolvingΔ ≤ 1/k gives t ≈

√2σ2mc ln k. It is straightforward to see

that when n(2σ2 ln k

)3 � k4 then we have t � k and thiscompletes the proof. �

ACKNOWLEDGMENT

This work was supported in part by NSF.

REFERENCES

[1] E. Cands, J. Romberg, and T. Tao, “Robust uncertainty principles: Exactsignal reconstruction from highly incomplete frequency information”.IEEE Trans. on IT, Feb. ’06.

[2] D. Donoho and Y. Tsaig, “Extensions of compressed sensing”. (SignalProcessing, 86(3), pp. 533-548, March 2006).

[3] D. L. Donoho, Y. Tsaig, I. Drori, and J. Starck, “Sparse solutionof underdetermined linear equations by stagewise orthogonal matchingpursuit”. Preprint, 2007.

[4] D. Needell and J. A. Tropp, “CoSaMP: Iterative signal recovery fromincomplete and inaccurate samples”. Preprint, 2008.

[5] Y. C. Pati R. Rezaiifar, and P. S. Krishnaprasad, “Orthogonal MatchingPursuit: Recursive Function Approximation with Applications to WaveletDecomposition”, Proceedings of the 27 th Annual Asilomar Conferenceon Signals, Systems, and Computers, Nov. ’93.

[6] S. Chen, , D. Donoho, and M. Saunders. “Atomic decomposition by basispursuit”. Tech. Report 479, Dept. of Stat., Stanford Univ., May ’95.

[7] R. Berinde and P. Indyk, “Sparse recovery using sparse random matrices”,Preprint, 2008.

[8] P. Indyk and R. Berinde “Sequential Sparse Matching Pursuit”, Allerton,2009.

[9] A. A. Moghadam and H. Radha, “Complex Sparse Projections forCompressed Sensing,” in the Proceedings CISS 10, Princeton.

[10] S. Jafarpour, W. Xu and et. al, “Efficient compressed sensing using high-quality expander graphs”. IEEE Trans. IT, 2009.

[11] Dror Baron, Shriram Sarvoham, Richard G. Baraniuk, “Bayesian com-pressive sensing via belief propagation”. (To appear in IEEE Trans. SignalProcessing, 2009).

[12] M. Duarte, M. Davenport and et. al “Single-pixel imaging via compres-sive sampling.” IEEE SP Magazine, 25(2) pp. 83-91, March 08.

[13] P. Nagesh and B. Li, “Compressive Imaging of Color Images”. IEEEICASSP, Taipei, Taiwan, 09.

[14] M. N. Do and M. Vetterli, “The contourlet transform: an efficientdirectional multiresolution image representation,” IEEE Trans. on IP,14(12):2091-2106.

[15] R. Eslami and H. Radha, “New Image Transforms using Hybrid Waveletsand Directional Filter Banks: Analysis and Design,” ICIP, Sep. 2005.

[16] I. W. Selesnick, R. G. Baraniuk and N. G. Kingsbury, “The Dual-TreeComplex Wavelet Transform,” IEEE SP Magazine, 22(6):123-151, 2005.

[17] “Probability and Computing, Randomized Algorithms and ProbabilisticAnalysis“, Michael Mitzenmacher and Eli Upfal, Cambridge UniversityPress, 2005.

[18] “Iterative Methods for Solving Linear Systems”, Anne Greenbaum,Society for Industrial and Applied Mathematics, 1997.

[19] David L. Donohoa, A. Maleki and A. Montanari, “Message-passingalgorithms for compressed sensing”, preprint, 2009.

104