distributed compressive video sensing

Li-Wei Kang and Chun-Shien LuInstitute of Information Science, Academia Sinica

Taipei, Taiwan, ROC{lwkang, lcs}@iis.sinica.edu.tw

April 20092009 IEEE International Conference on Acoustics, Speech, and Signal Processing

(ICASSP2009, Taipei, Taiwan, ROC)

Distributed Compressive Video Sensing

Distributed Compressive Video Sensing April 24, 2009 2

Distributed Source Coding

[bits]XR

[bits]YR

H X

H Y

|H Y X

|H X Y

,X YR R H X Y

Vanishing error probabilityfor long sequences

Vanishing error probabilityfor long sequences

No errorsNo errors

[Slepian and Wolf, 1973]


Distributed Video Coding

“Motion JPEG”

Decoder

“Motion JPEG”

Encoder

X’X

Wyner-ZivInterframe Decoder

Wyner-ZivIntraframe Encoder

Side Information

Y

[Girod, 2006]


Distributed Video Coding

• The statistical dependency between X and YLaplacian distribution

,2

ii YXii eYXp

ii YX

2


Compressive Sensing

• When data is sparse/compressible, one can directly acquire a condensed representation with no/little information loss

• Random projection will work

[Baraniuk, 2008]


Compressive Sensing

• Directly acquire “compressed” data

• Replace samples by more general “measurements”

[Baraniuk, 2008]


Compressive Sensing

• y = Фx = ФΨθ = Aθ

y Ф Ψ θ

x = Ψθ

A = ФΨ

N×1N×N

M×NM×1

[Baraniuk, 2008]


Measurement Matrix

• Scrambled block Hadamard ensemble (SBHE)partial block hadamard transform and random column

permutation

Ф = QMWPN

L. Gan, T. T. Do, and T. D. Tran, “Fast compressive imaging using scrambled hadamard ensemble,” in Proc. of European Signal Processing Conf., Lausanne, Switzerland, August 2008 (EUSIPCO2008).


Signal Reconstruction

• The convex unconstrained optimization problem

• Can be seen as a maximum a posteriori criterion for estimating θ from

y = A θ + n, where n is white Gaussian noise

1

2

2A

2

1min

y


Signal Reconstruction

• Signal recovery from random measurementsGradient projection for sparse reconstruction (GPSR)Two-step iterative shrinkage/thresholding algorithm (TwIST)Orthogonal matching pursuit (OMP)

M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, “Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems,” IEEE J. of Selected Topics in Signal Processing, vol. 1,no. 4, pp. 586-597, Dec. 2007.

J. M. Bioucas-Dias and M. A. T. Figueiredo, “A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Trans. on Image Processing, vol. 16, no. 12, pp. 2992-3004, Dec. 2007.

T. Blumensath and M. E. Davies, “Gradient pursuits,” IEEE Trans. on Signal Processing, vol. 56, June 2008.



• Measurement matrix Ф: scrambled block Hadamard ensemble (SBHE)

• Sparse basis matrix Ψ: DWT• Video signal sensing (encoder): general random

projection• Video signal recovery (decoder)

Key frame: GPSR with default settingsCS frame

side information generation (motion compensated interpolation)GPSR with the proposed initialization and the proposed

termination criteria



CS measurementyt = Ф xt

Each frame xtMeasurement vector (compressed frame) yt

Measurement vector yt

for each non-key frameInitialization by SI generation

Reconstructed previous key frames

GPSR optimization

Stopping criteria (a)-(c)

Non-stop Stop

ttx ~~ Reconstructed non-key frame

tx~

Compressive video sensing

Video signal recovery



• At the decoder, for a CS frame xt = Ψθt

its side information St = ΨθSt can be generated from its previous reconstructed key frames

• Proposed initializationinitial solution at the 0-th iteration:

• α(xt, St): the Laplacian parameter of (xt- St)

,~ 0

Stt ,~ 0tt Sx


Key frame (t - 1)

Non-key frame t

Key frame (t + 1)

yt-1 = Φxt-1

with higher MR

yt = Φxt

with lower MR

yt+1 = Φxt+1

with higher MR

GPSR reconstruction

GPSR reconstruction

Proposed Modified

GPSR reconstruction

Side information (t)

Reconstructed frame (t)

Reconstructed frame (t-1)

Side information generation

Reconstructed frame (t+1)


Distributed Compressive Video Sensing itx~

xt St

α(xt, )

α(xt, St)

,2

1min

1

2

21 tttt AyFt

22 StttF

it

it

it FWFWF ~~~

2211

itx~

α( , St) itx~


Proposed Termination Criterion

• First:

• Second:

• Third:

T

Sx

SxSx

ti

t

ti

tti

t

,~

,~,~

1

1

0~~ 1 i

ti

t FF

Fi

t

it

it

TF

FF

1

1

~

~~


Proposed Termination Criterion

• MR is low (MR ≤ 20%): if the First criterion with Tα = 0.9 is satisfied, the algorithm will stop

• MR is middle (20% < MR ≤ 70%): if the First criterion with Tα = 0.05 or the Second criterion is satisfied, the algorithm will stop

• MR is high (MR > 70%): if the Third criterion with TF = 0.001 is satisfied, the algorithm will stop


Simulation Results

• Foreman and Coastguard CIF video sequences with 300 Y frames (352×288 = 101376 samples for each Y frame) and GOP size = 3 (Key, Non-key, Non-key, Key, …)

• The three approaches for comparison (all with default settings)GPSR, TwIST, OMP

• For OMP, block size = 32×32 suggested by V. Stankovic, L. Stankovic, and S. Cheng, “Compressive video sampling,” in Proc.

of European Signal Processing Conf., Lausanne, Switzerland, August 2008 (EUSIPCO2008).


Simulation Results


Simulation Results

The reconstruction complexities for the Foreman sequence


Simulation Results

The PSNR performance at different reconstruction complexities for the Foreman sequence


Simulation Results

(a) Side information (b) Reconstructed frame


Simulation Results

The reconstructed Foreman sequences (352×288 for each frame) at measurement rate (MR) = 0.3 using (a) GPSR (gradient projection for sparse reconstruction) (average PSNR = 27.68 dB) (average reconstruction time = 15.14 seconds per frame); and (b) our DCVS (average PSNR = 29.48 dB) (average reconstruction time = 3.68 seconds per frame) (This example shows the 54-th frame).


Conclusions

• The proposed DCVS approach exploits the two characteristicsdistributed video coding (DVC)compressive sensing (CS)

• The proposed DCVS can outperform or be comparable with the three existing approaches for comparison, especially at lower measurement rates

• The proposed DCVS can significant outperform the three existing approaches at the same reconstruction complexity

distributed compressive video sensing

Documents