speed up in feedback channel for a ldpca base distributed video coding system on mobile device

Speed up in feedback channel for a LDPCA base distributed video coding

system on mobile device在手機裝置上對低密度奇偶校驗碼為基礎之分散式編碼中的回饋通道加速

Chen,chun-yuan 陳群元Advisor: Prof. Wu, Ja-Ling 吳家麟教授

2012/5/18

outline Motivation and introduction Mobile video trans

Traditional video codec Distributed video codec

DVC architecture Channel coding Ldpca

The video communication system on Mobile device Speed up DVC decoding time in feedback channel Experiment Result Conclusion

Digital Video Service

Video compression is an essential component of broadcast and entertainment media

Multimedia video everywhere!

mobile video conference

mobile cameras phone

Video surveillance

Wireless sensor network

Conventional Video Codec

MPEG-2, H.264, HEVC(H.265)ENCODER DECODERLightweig

htHeavyweight

Application of DVC Video conferencing with mobile

devices

DVC to H.264 Transcoder

CloudComputational Resource

DVC encoder(Low Complexity)

H.264 decoder(Low Complexity)

DVC encoded bitstream

H.264 encoded bitstream

Distributed Video Coding

RY =H (Y)

Source encoder

Correlation is exploited by

motion estimation

RY =H (X|Y)Source encoder

H (X|Y)H (Y)-------Distributed-------

Slepian-Wolf Theorem(1973) Wyner-Ziv Theorem(1976)

Channel encoder

Source decoder

Side Information creation

Channel decoder

X

Y

Virtual channelVirtual channel

H (X, Y)


D. Varodayan, A. Aaron, and B. Girod, “Rate-Adaptive Codes for Distributed Source Coding,”EURASIP Signal Processing Journal, Special Issue on Distributed Source Coding,,November 2006.

Channel Encoder

Channel Decoder

LDPCEncoder

LDPCDecoder



Channel Encoder

Channel Decoder

LDPCEncoder

LDPCDecoder

Key frame

Key frame

WZ frame

WZ frame

WZ frame

GOP size 4

Key frame

Key frame

WZ frame

GOP size 2

Quantization

DCT coefficients bands

DCT coefficient band b1 : { S11, S21, S31, …SN1 }DCT coefficient band b2 : { S12, S22, S32, …SN2 }

DCT coefficient band b16 : { S116, S216, S316, …SN16 }

…

DC band

AC bands

Block1

S11 S12 S16 S17

S13 S15 S18 S113

S14 S19 S112 S114

S110 S111 S115 S116

Block2

S21 S22 S26 S27

S23 S25 S28 S213

S24 S29 S212 S214

S210 S211 S215 S216

Block3

S31 S32 S36 S37

S33 S35 S38 S313

S34 S39 S312 S314

S310 S311 S315 S316

…



Channel Encoder

Channel Decoder

LDPCEncoder

LDPCDecoder

Quantization

Q8

2^7

2^6

2^6

2^5

2^5

2^5

2^4

2^4

2^4

2^4

2^3

2^3

2^3

2^2

2^2

63bits for one block

Q8

2^52^4

2^4

2^3

2^2

2^32^2

2^32^22^2

2^22^2

2^2

37bits for one block

Bit plane Extraction

00100 00001

00000 11110

Bit planes of DC band:

Bit plane 1:

Bit plane 2:

Bit plane 3:

Bit plane 4:

Bit plane 5:

Independently Channel Encode (LDPCA)

4 6

7

0 6

3

1 7

7

30 1

5

For each DCT coefficient band…

MSB

LSBZig-zag order

LDPC decodingSum-Product Algorithm

(Message Passing)

Side Information(real number)+ 0 - 1

4 6 7

甲乙丙

3 521

decode outputhard decision

a b c d e f g

a25 b25 c25 d25 e25 f25 g25

Vertical processing

Horizontalprocessing

a1 b1 c1 d1 e1 f1 g1

0 1 1

From DVC encoder(syndrome bits)

Kschischang, F.R., Frey, B.J., and Loeliger, H.-A. 2001. Factor graphs and the sum-product algorithm. IEEE Trans. Inform. Theory

LDPC Accumulate (LDPCA) codes

Rate adaptivity

D. Varodayan et al., "Rate-adaptive codes for distributed source coding," EURASIP Signal Processing Journal, Special Section on Distributed Source Coding, 2006

65

LD

PC

cod

es

3



Channel Encoder

Channel Decoder

LDPCEncoder

LDPCDecoder



Channel Decoder

LDPCDecoder

Key frame

request

WZ Bit stream1’th bitstream&CRC

63’th bitstream&CRC

Amdahl's law

Maximum speedup can be reached by improving the most critical part of the system

Feedback channel in the DVC coding.

time ratio

SI generationLDPCA DecFeedback channeletc.

29%

66.6%

Amdahl's law



time ratio(baseline)

SI generationLDPCA Dec+Feedback channeletc.95.6%

Amdahl's law


Feedback channel in the DVC coding. estimate syndromes size per bitplanetime ratio(estimate syndromes size

per bitplane)

SI generation LDPCA Dec+Feedback channel

etc.92.8%

1.7 X

Amdahl's law


Feedback channel in the DVC coding. estimate syndromes size per WZ frametime ratio(estimate syndromes size

per WZ frame)


etc.82.8%

4.6 X

Methods to speed up at feedback channel

So we propose two methods to decrease time consuming at feedback channel. Estimate the syndromes size per WZ

frame Estimate the syndromes size per bitplane

Number of requests per bitplane per band

1234567123456123456123451234512345123412341234123412312312312120

200

400

600

800

1000 bit rate

bit rate

DC AC1 AC2 AC3 AC4 AC5 AC6 AC7 AC8AC9AC10AC11

AC12AC13

AC14

Bitplane Number

Estimate by previous bands

1.formula 2.time ratio


1234567123456123456123451234512345123412341234123412312312312120

200

400

600

800

1000 bit rate

bit rate


AC12AC13

AC14

Bitplane Number


1234567123456123456123451234512345123412341234123412312312312120

200

400

600

800

1000

bit rateestimated bit rate


AC12AC13

AC14

Bitplane Number

formula

(ESacn bt)WZn= ( Sdc bt-1 )WZn

WZn means the n’th WZ frame. acn means the n’th AC band. bt means the bitplane index of this AC band. Which ESacn

bt mean the estimated syndrome size for the bt’th bitplane in n’th AC band.

dc means the DC band. Sdc bt-1 means the syndromes bits for bt-1’th

bitplane in DC band.

Amdahl's law





406.39 S

Amdahl's law



time ratio(estimate syndromes size per bitplane)


etc.92.8%

1.7 X

237.87 S

Estimate by reference frame

1.formula 2.time ratio

formula

(ESbt)WZn=(Sbt)WZ(n-GOPsize)

WZn means the WZ frame’s index n-GOPsize means the correspond WZ

frame in the previous GOP. bt is the bitplane index. (ESbt)WZn means the estimated

syndromes bit for the bt’th bitplane of WZn frame.

(Sbt)WZ(n-GOPsize) means the syndromes

Amdahl's law





406.39 S

Amdahl's law



time ratio(estimate syndromes size per WZ frame)


etc.82.8%

4.6 X

87.99 S

Test condition

12 CPU, 24 processor Intel(R) Xeon(R) CPU X5650 @ 2.67GHz

GPU: Tesla M2050 Mobile device: HTC sensation

DVC encoder on mobile

14 s for foreman sequence, GOP8 , intra mode on.

4 s for foreman sequence, GOP8 , intra mode off.

Test sequences :

QCIF, 15Hz, all frames GOP Size 2, 4 and 8 Only luminance component is used

Test materials

Soccer Foreman Coastguard Hall Monitor

Motion: High Low

Speed up

Bitrate

RD curve

Foreman with LRSS,GOP8

RD curve

Hall monitor with LRSS,GOP8

conclusion

A practical DVC coding system is built on mobile devices. DVC encoder and H.264 decoder keep

the mobiles light weight. We propose two method to reduce

transmission time spent in feedback channel. Estimate the syndromes size per WZ

frame Estimate the syndromes size per bitplane

Thank you

speed up in feedback channel for a ldpca base distributed video coding system on mobile device

Documents