International Conference on Image Analysis and Recognition (ICIAR’09). Halifax, Canada, 6-8 July 2009.

Video Compression and Retrieval of Moving

Object Location Applied to Surveillance

William R. Schwartz, Hélio Pedrini, Larry S. Davis

International Conference on Image Analysis and Recognition (ICIAR’09). Halifax, Canada, 6-8 July 2009. 2

Introduction Proposed Method Experimental Results Conclusions and Remarks

Organization

International Conference on Image Analysis and Recognition (ICIAR’09). Halifax, Canada, 6-8 July 2009. 3

Motivation: Surveillance cameras capture hours of data that need

to be store and analyzed.

Introduction

Analysis is based mostly on moving objects (interaction between people in the scene).

Reduce redundancy + store objects.

4Introduction Proposed method Experimental results Conclusions

Goals: Develop a video compression technique that takes

advantage of static cameras (surveillance). Provide information regarding the location of moving

objects (higher-level computer vision tasks).

Key ideas: Estimate eigenspaces for non-overlapping blocks. Use a second encoding scheme to encode regions not

well modeled by the eigenspaces.

Introduction

5Introduction Proposed method Experimental results Conclusions

Use of eigenspaces allow high compression ratio. Store the projection vectors (obtained using PCA). For a new frame, save only projection coefficients (in

case of no changes or linear changes in the block).

Advantages of Eigenspaces

Blocksize

Compression ratio (assuming float coefficients)

1 coefficient 2 coefficients 3 coefficients 4 coefficients

8x8 16 8 5.33 4

16x16 64 32 21.33 16

32x32 256 128 85.33 64

64x64 1024 512 341.33 256

6Introduction Proposed method Experimental results Conclusions

Eigenspaces do not model non-linear changes (i.e. moving person).

Use a second encoding method for these cases.

Two-Stage Scheme

7Introduction Proposed method Experimental results Conclusions

Segment the image area into non-overlapping blocks.

Estimate eigenspaces for each block. For new frames, project blocks onto the

eigenspaces and compute the reprojection error. If the reprojection error is acceptable, save the

scores, otherwise set the image block to be encoded using MPEG-4.

Proposed Method

8Introduction Proposed method Experimental results Conclusions

For each block, sample frames free of non-linear transformations (≈ 200 frames).

Estimate projection vectors P = {p1,…, pk}.

Estimate reconstruction error distribution δp for

each pixel in the block (used to locate moving objects).

Learning the Eigenspaces

9Introduction Proposed method Experimental results Conclusions

Compression Algorithm

10Introduction Proposed method Experimental results Conclusions

The reprojection error is high when there are non-linear transformations within a block (moving objects / non-linear local illumination changes).

Given that a block was compressed using MPEG-4, check the pixels that do not satisfy the error distribution δp to decide if there is a moving

object on that block.

Location of Moving Objects

11Introduction Proposed method Experimental results Conclusions

Our method was tested on four video sequences. Compared to MPEG-4 and H.263 using

MEncoder. Experiments:

Compression with a constant PSNR. Moving object location accuracy.

Experimental Results

12Introduction Proposed method Experimental results Conclusions

Frames were converted to YCbCr. Using blocks of 16 × 16 pixels. Initial 200 frames were used to learn the

eigenspaces. Number of PCA coefficients kept is estimated for

each block (bounded by a maximum).

Video Compression

13Introduction Proposed method Experimental results Conclusions

Video Compression

videosequence

Size &# frames

PSNR (dB)

Compression Ratio

MPEG-4 H.263 proposed

camera 1 768x288x2695 39.00 33.78 34.33 37.27

camera 2 720x528x5333 38.00 40.44 40.27 45.06

station 720x576x2370 39.50 51.27 49.94 105.64

robbery 720x480x3320 38.00 34.70 34.66 61.14

camera 2

camera 1

robbery

station

14Introduction Proposed method Experimental results Conclusions

Easy to retrieve object location due to the encoding scheme used.

Evaluation compares the location obtained by the method to the ground truth location.

At a false positive rate of 0.025 obtained a false negative rate of 0.051.

Moving Object Location

15Introduction Proposed method Experimental results Conclusions

High compression rates: Robust to linear transformations in illumination. No need for saving key-frames.

Fast to encode (≈ 5 fps in MATLAB). Useful for higher level computer vision tasks due

to the storage of moving object locations. The use of multiple size blocks might increase

the compression ratio.

Conclusions and Remarks