Compressed Sensing for Networked Information Processing

Reza Malek-Madani, 311/ Computational Analysis

Don Wagner, 311/ Resource Optimization

Tristan Nguyen, 311/ Computational Analysis

28 November 2007

Ubiquitous Compressibility

DoD acquires and uses huge amount of data – In many scenarios, most of the data in a signal can be

discarded with almost no perceptual loss– E.g., lossy compression formats for sounds and images

Key Questions:– Why acquire all the data when most will be discarded? – Can we directly measure the relevant information?

Challenge: Develop mathematical and computational techniques that allow us to directly acquire relevant information from signals and images in compressed form.

Key Words: Adaptivity, Parallelization, Stability, Nonlinearity, Noise

What is Compressed Sensing?

• Underlying Assumption: Most signals are compressible in some representation (i.e., most coefficients are small relative to some basis)

• Compressed Sensing:– “Measure” the signal via a random projection to yield a

compact representation– Reconstruct the signal from its compact representation

Nyquist vs. Compressed Sensing

Nyquist rate samples of wideband signal (sum of 20 wavelets)

N = 1024 samples/second

Reconstruction from compressed sensing

M = 150 random measurements/second

MSE < 2% of signal energy

• Nyquist rate samples of image N = 65536 pixels

• Reconstruction from compressed sensingM = 20000 projections

MSE < 3% of signal energy

Nyquist vs. Compressed Sensing

Compressed Sensing

• Forward Problem: Random projection is the key idea

• Inverse Problem: Reconstruct from ; this is an ill-posed problem [Candes-Romberg-Tao, Donoho, 2004]

measurementssparsesignal

sparsein some

basis

• Reconstruction: find given

• Classical L2 approach:

• L2 algorithm is fast, but unfortunately it is wrong

CS Signal Recovery

• Reconstruction: given(ill-posed inverse problem) find

• L2 fast, wrong:

• L0 correct, slow:

• L1 correct, mild oversampling: [Candes et al, Donoho]

CS Signal Recovery

Linear-programming problem

Theoretical Result (Donoho, 2004)

Theorem: There is a function g, from the interval (0, 1] to itself, with the following characteristics:

Fix ε > 0.

If K/M > g(M/N)(1 + ε) then, with overwhelming probability for large N,

= . If K/M < g(M/N)(1 – ε ), then does not equal .

Applications

• Image Understanding (feature detection)

• Communication – Underwater Communication (RF)– Wireless Communication– Channel Parameter Identification (cognitive radio, radar)

• Distributed Sensing (fusion of partial information)

• ….• …

Mathematical Challenges

• Randomness versus determinism– Can random sensing matrices be replaced by deterministic ones? – What is the impact on the theoretical development? – Can rigorous bounds be developed for the equivalents of K, M and N?

• Faster optimization algorithms – Reconstruction via L1 minimization is relatively slow– Other algorithmic ideas need to be developed

• First-order vs. second-order methods?• Combinatorial vs. linear vs. nonlinear methods?

– Need to create baselines for comparing algorithms in terms of reconstruction speed and accuracy

• Multiple sensors/multiple targets– Can the underlying theory be extended to handle distributed, networked sensors and

multiple targets?– Develop the mathematical tools needed to take advantage of the statistical correlations

among signals to perform multi-signal reconstruction

Mathematical Challenges

• How many measurements are needed?– One can get by with much fewer measurements, but at the

expense of having to solve a tougher (i.e., non-convex) optimization problem. What is the tradeoff?

• Important extensions needed– No development to date of distributed reconstruction algorithms

• Very important for distributed sensor networks– Does the theoretical developments to date adequately address

the issue of noise?• Random vs. pseudo-random vs. deterministic sensing

matrices

Tangential Issues

• Extension of compressed-sensing optimization to affine rank minimization– Potentially very important in data mining

• Numerical partial differential equation solvers – Can de-aliasing techniques benefit from the compressed-

sensing approach?

Budget

• Current seed investment: $250K/year

• Proposed First Year: $1.2M/year– 40% for analytical/theoretical development– 40% for algorithmic/computational development– 20% for application/sensor development

• Outyear growth towards applications

FY FY+1 FY+2 FY+3

ONR $1.2M $1.4M $1.5M $1.5M

Summary

• Compressed Sensing is an important emerging area– Cuts across of sciences and engineering– Pioneering foundations are in place– ONR is well positioned to be a leader