Network-Aware H.264/AVC Region-of-Interest Coding for a Multi-Camera

Wireless Surveillance Network

NetworkNetwork--AwareAware H.264/AVC H.264/AVC RegionRegion--ofof--InterestInterest CodingCoding forfor a a MultiMulti--CameraCamera

WirelessWireless SurveillanceSurveillance NetworkNetwork

PierpaoloPierpaolo BaccichetBaccichet, Xiaoqing Zhu and Bernd , Xiaoqing Zhu and Bernd GirodGirod

Information Systems LaboratoryInformation Systems LaboratoryStanford UniversityStanford University

P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 2

OutlineOutline

Overview of network-aware wireless surveillance

H.264/AVC compliant Region-of-Interest (ROI) coding

Cross-Layer design

– Reference Picture Selection

– Dynamic Rate Control

P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 3

Wireless Surveillance Camera NetworkWireless Surveillance Camera Network

Important to capture all (and possibly only) significant changes– ROI coding was proposed for JPEG 2000 [Hata et al., MMSP 2005]

Dynamic, distributed resource allocation– Network driven rate control [Zhu et al., MMSP 2005]

Data Collection

P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 4

Simulation ScenarioSimulation Scenario

1 0

6

7 3

2

4

5

Residential Area

Central Location 1

6

73

2

4

5

0Collection Node

(Stanford)

P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 5

OverviewOverview of the of the ProposedProposed SchemeSchemeInput video

Noise reduction

filter

Pre-processor

-

Framestore

ROI / SKIPdetermination

H.264/AVCencoder

Video Packets

Feedback

ROI signalling Network

A simple pre-processor identifies important regions/pictures to code– The frame is skipped if no changes are detected– Only the region-of-interest is encoded for “active” frames

Network feedback exploited to dynamically adapt encoding parameters– Reference Picture Selection– Dynamic rate control

P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 6

H.264/AVC Compliant ROI CodingH.264/AVC Compliant ROI Coding

-

0 1 0 0 0 0 0 0 0 0 00 0 1 0 1 4 1 0 0 0 00 0 0 0 5 8 3 0 0 0 00 0 0 0 14 13 6 0 0 0 00 0 0 0 4 11 3 0 0 0 00 0 0 0 0 4 3 0 0 0 00 0 0 0 0 0 1 1 0 0 00 0 0 0 0 0 0 1 1 0 00 0 0 0 0 0 1 0 1 1 0

0 1 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 0 00 0 0 0 0 0 1 1 0 0 00 0 0 0 0 0 0 1 1 0 00 0 0 0 0 0 1 0 1 1 0

SliceGroup 0

SliceGroup 1

SliceGroup 2

Slice Group 3

Stored image

Difference signalSignificance Map

(MAE for each Macroblock)Current image

ROI to be encoded

Picture Parameter Setwith the FMO2 mapping

ROI

P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 7

BitBit--Rate SavingsRate Savings

0 50 100 150 200 250 300 35032

34

36

38

40

42

Bit-rate [Kbps]

Lum

a P

SN

R (R

OI)

[dB

]

Camera 5

ConventionalFrame-skippingFrame-skipping + ROI

One intra MB line inserted every 4 frames to recover from errors

P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 8

Coded Frame SizeCoded Frame Size

5 10 15

0

500

1000

1500

2000

2500

Time [s]

Enc

oded

Fra

me

Siz

e [b

ytes

]

Camera 5

ConventionalFrame-SkippingFrame-Skipping + ROI

Original uncompressed H.264/AVC (~ 1.16Mbit/sec)

Frame skipping(~ 630 Kbit/sec)

Frame skipping + ROI(~ 180 Kbit/sec)

P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 10

Impact of Packet LossesImpact of Packet Losses

300 400 500 600 700 80026

27

28

29

30

31

Bit-rate [Kbps]

Lum

a P

SN

R (R

OI)

[dB

]

Camera 5

ConventionalFrame-skippingFrame-skipping + ROI

Simulation performed using ns2, maximum delay 0.5 secAutomatic retransmission every 0.2 sec

P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 11

ReferenceReference Picture Picture SelectionSelection

Error control by means of Reference Picture Selection– ACK based approach

1 2 3 4 5 6 7 8 9

ACK(3) ACK(4) ACK(6)ACK(1)

P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 12

ReferenceReference Picture Picture SelectionSelection

200 400 600 800 200 400 600 80026

27

28

29

30

31

32

33

34

35

36

37

Bit-rate [Kbps]

Lum

a P

SN

R (R

OI)

[dB

]Camera 5

noRPS - Frame-skippingnoRPS - Frame-skipping + ROIRPS - Frame-skippingRPS - Frame-skipping + ROI

P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 13

252729313335373941

Lum

a PS

NR

(RO

I) [d

B]

1 2 3 4 5 6 7 AVGCamera number

Fixed QP Adaptive QP

NetworkNetwork--AwareAware Rate ControlRate Control

QP manipulated according to:– number of unACKed packets– end-to-end delay

Video Video SequenceSequence ComparisonComparison

Conventional H.264/AVC Network Aware ROI coding

P. Baccichet – Network-Aware ROI coding for Wireless Surveillance – PCS 2006 15

ConclusionsConclusions

ROI coding and automatic frame rate adaptation leads to more efficient representation of surveillance video

Reference Picture Selection and dynamic rate control enablebetter usage of available network resources

Highest sustainable quality increased by 4 to 10 dB