overview of error resiliency schemes in h.264/avc standard
DESCRIPTION
Overview of Error Resiliency Schemes in H.264/AVC Standard. Sunil Kumar, Liyang Xu, Mrinal K. Mandal, and Sethuraman Panchanathan. Elsevier Journal of Visual Communication and Image Representation, to appear in2005. - PowerPoint PPT PresentationTRANSCRIPT
Overview of Error Overview of Error Resiliency Schemes in Resiliency Schemes in H.264/AVC StandardH.264/AVC Standard
Overview of Error Overview of Error Resiliency Schemes in Resiliency Schemes in H.264/AVC StandardH.264/AVC Standard
Sunil Kumar, Liyang Xu, Mrinal K. Mandal, Sunil Kumar, Liyang Xu, Mrinal K. Mandal, and Sethuraman Panchanathanand Sethuraman Panchanathan
Elsevier Journal of Visual Communication and Image Representation, to appear in2005.
S. Wenger, “H.264/AVC over IP,” IEEE Trans. Cir. Syst. Video Technol., vol. 13, pp. 645–656, July 2003
Outline• Introduction• Error-Resilience Tools in
H.264/AVC• Experimental Results• References
Introduction• H.264/AVC Video Coding Standard
– http://vc.cs.nthu.edu.tw/home/paper/codfiles/cycho/200310132110/Overview_of_the_H.264_AVC_Video_Coding_Standard.ppt
Introduction• Error-resilience tools in older video c
ompression standard– Picture segmentation (slices, GOBs);– Placement of Intra MBs, Intra slices, and i
ntra pictures;– Reference picture selection (with and wit
hout feedback);– Data partitioning
MPEG 1 part 2MPEG 2 part 2H.261H.263
H.263
Error-Resilience Tools in H.264/AVC
• Reference picture selection– Feedback-based
– Without feedback• e.g. Video redundancy coding
• Picture segmentation– Slice
• Flexible macroblock ordering (FMO)– 7 modes– Overhead bits highly depends on the picture forma
t, the content, and the QP. (S. Wenger and M. Horowitz)
• < 5% penalty at QP = 16; on average 20% at QP = 28.
Encoder Decoder
feedback channel
I0p1 p3 p5 p7
p9
p2 p4 p6 p8Channel 1
Channel 2
Checker board modeInterleaving mode
Error-Resilience Tools in H.264/AVC
• SP/SI frame http://vc.cs.nthu.edu.tw/home/paper/codfiles/cycho/200402181718/The_SP-_and_SI-Frames_Design_for_H.264_AVC.ppt
– Instantaneous refreshing– Gradual refreshing
P1, n-2 P1, n-1 S1, n P1, n+1
S 21, n
SI1, n
P1, n-3
Error-Resilience Tools in H.264/AVC
• Intra placement– Used to combat drifting effects.– ConstrainedIntraPrediction flag
• An Intra block can not be predicted from an Inter block.
– IDR slice and IDR picture• An IDR picture contains only IDR slices.• An IDR slice can only be part of an IDR picture.• An IDR picture invalidates all short-term reference me
mory buffers.
Intra
Intra
inter
Intra
I P IDR P
Error-Resilience Tools in H.264/AVC
• Data partitioning– For unequal error protection (UEP).– Three different partition types in H.264.– Type A
• Header information (MB types, QP, MVs)– Type B
• Intra coded block patterns (CBPs) and Intra coefficients– Type C
• Inter CBPs and Inter coefficients– Importance
• A > B > C
Error-Resilience Tools in H.264/AVC
• Redundant slices (RSs)– For a MB, an encoder can place redundant repr
esentation of the same MBs into the same it stream.
– e.g.• One slice is coded using different QP.• If the slice of low QP is available, the decoder discard
s the RS; otherwise, the RS is reconstructed by the decoder
Slice AQP1
Slice AQP2
Decoder
Error-Resilience Tools in H.264/AVC
• Parameter sets– Including picture size, entropy coding method,
MV resolution, and so on.– Sequence parameter set
• Containing all information related to the picture sequence between two IDR pictures.
– Picture parameter set• Containing all information related to all slices in a pic
ture.– e.g. Sending multiple copies of SPSs so to enha
nce the arrival rate.– e.g. SPSs can be sent out-of-band.
Error-Resilience Tools in H.264/AVC
• Intra block refreshing by RD control– – Better than periodically inserting intra M
Bs.
))()((minarg* oRoDoXo
Experimental Results
Stephan Wenger
Experimental Results• Error concealment scheme
– Intra
– Inter• }|)mv(|{
1},,,{
argmin NYYdN
j
OUTj
INj
dirsm
rightleftbottopdir
foremanparis
Stephan Wenger
Experimental Results• Experiment 1: One picture, one
packet, without any error-resilience– Packet loss rates > 3% unprotected
H.264 video becomes unusable.Foreman
Paris
Stephan Wenger
Experimental Results• Experiment 2: One picture, one pa
cket, with intra macroblock refresh– Very good PSNR results– Very high amount of Intra MBs of signi
ficant size• A coarser quantizer is needed for CBR – l
ess spatial detail.
Stephan WengerForeman
PSNR = 35
PLR = 0%
Experimental Results• Experiment 3: 2-4 slices in one
frame– Unfavorable PSNR results for
Foreman.– 2nd best PSNR results for Paris.
ForemanParisPacket header overhead
?
Stephan Wenger
Experimental Results• Experiment 4: Slices interleaving
– Better than Exp. 3 for Foreman.– Worse than Exp. 3 for Paris.
ForemanParis
Stephan Wenger
Experimental Results• Experiment 5: Data partitioning
– Partition A is sent twice (3% PLR) or 3 times (>3% PLR).• Degraded PSNR due to CBR.
ForemanParis
Stephan Wenger
Experimental Results• Experiment 6: FMO
– Randomly inserting Intra MB. (3% of PLR)
– Best PSNR results
ForemanParis
Stephan Wenger
Experimental Results
QCIF, 10fps,100Kbps
M. M. Ghandi and M. Ghanbari
Experimental Results
QCIF, 10fps,100Kbps
M. M. Ghandi and M. Ghanbari
References• S. Kumar, L. Xu, M. K. Mandal, and S. Panchanathan, “Overview of
Error Resiliency Schemes in H.264/AVC Standard,” Elsevier Journal of Visual Communication and Image Representation (Special issue on H.264/AVC Video Coding Standard), to appear in 2005.
• S. Wenger, “H.264/AVC over IP,” IEEE Trans. Cir. Syst. Video Technol., vol. 13, pp. 645–656, July 2003.
• S. Wenger and M. Horowitz, “Scattered slices: A new error resilience tool for H.26L,” in JVT-B027, Feb 2002.
• M. M. Ghandi and M. Ghanbari, “Layered H.264 video transmission with hierarchical QAM,” Elsevier J. of Visual Communication and Image Representation (Special issue of H.264/AVC), to appear in 2005.