digital video compression fundamentals and standards

Digital Video Compression Fundamentals and StandardsWeb Technology

Digital Video Compression Fundamentals and Standards

OutlineIntroductionVideo Compression StandardsSimulation Reference SoftwareFuture Work and Conclusions


Introduction (1/2)Why video compression technique is important ?One movie video without compression720 x 480 pixels per frame30 frames per secondTotal 90 minutesFull colorThe total quantity of data = 167.96 G Bytes !!


Introduction (2/2)What is the difference between video compression and image compression?Temporal RedundancyCoding method to remove redundancyIntraframe CodingRemove spatial redundancyInterframe CodingRemove temporal redundancy


The most intuitive method to remove Spatiotemporal redundancy

3-Dimensional DCTRemove spatiotemporal correlationGood for low motion videoBad for high motion video


The most popular method to remove temporal redundancyThe Block-Matching Algorithm


Matching FunctionThe dissimilarity between two blocks and

The matching criteriaMean square error (MSE) High precision is neededMean absolute difference (MAD) Low precision is enough


The Exhaustive Block-Matching AlgorithmReference FrameCurrent FrameSearch RangeMotionVector11 22 33 44 55 66 77 8811 22 33 44 55 66 77 8811 22 33 44 55 66 77 8811 22 33 44 55 66 77 8811 22 33 44 55 66 77 8811 22 33 44 55 66 77 8811 22 33 44 55 66 77 8811 22 33 44 55 66 77 8812 22 33 44 55 66 77 8812 22 33 44 55 66 77 8812 22 33 44 55 66 77 8812 22 33 44 55 66 77 8811 23 34 44 55 66 77 8811 23 34 44 55 66 77 8811 23 34 44 55 66 77 8811 23 34 44 55 66 77 881 0 0 0 0 0 0 01 0 0 0 0 0 0 01 0 0 0 0 0 0 01 0 0 0 0 0 0 00 1 1 0 0 0 0 00 1 1 0 0 0 0 00 1 1 0 0 0 0 00 1 1 0 0 0 0 0|A|=12


Fast Block-Matching AlgorithmsEBMA needs Intensive computation Fast Algorithm is neededFind the possible local optimalReference FrameCurrent Frame


Fast Block-Matching Algorithms The characteristics of fast algorithmNot accurate as EBMASave large computationTwo famous fast algorithm2-D logarithm Search MethodThree Steps Search Method


2-D logarithm Search Method1111122233444441234


Three Step Search Method1111122221111222233333333123


Multiresolution Motion Estimation (1/3)The number of levels is Ll-th level images of the target frames

where is set of pixels at level L At the l-th level, the MV is At the l-th level, the estimated MV isDetermine update such that error is minimized

The new motion vector is


Multiresolution Motion Estimation (2/3) Variable block size method


Multiresolution Motion Estimation (3/3)


The Development of Video Compression Standards


5-point 1D DCT

6-point 1D DCT

5-point 1D DCT

6-point 1D DCT

The MPEG-1 StandardGroup of Pictures Motion EstimationMotion CompensationDifferential CodingDCTQuantizationEntropy Coding


Group of Pictures (1/2)I-frame (Intracoded Frame)Coded in one frame such as DCT. This type of frame do not need previous frame

P-frame (Predictive Frame)One directional motion prediction from a previous frameThe reference can be either I-frame or P-frameGenerally referred to as inter-frame

B-frame (Bi-directional predictive frame)Bi-directional motion prediction from a previous or future frameThe reference can be either I-frame or P-frameGenerally referred to as inter-frame


Group of Pictures (2/2)The distance between two nearest P-frame or P-frame and I-framedenoted by MThe distance between two nearest I-framesdenoted by N


The MPEG-1 Encoder (1/4)DCTQEntropyCodingQ-1IDCTMotionCompensationFrameMemoryMotionEstimationDCTQEntropyCodingDCTQQ-1IDCTMotionCompensationFrameMemoryMotionEstimationIntra-frameEntropyCodingResidueMotion VectorInter-frame


The MPEG-1 Encoder (2/4)Differential Coding is the input image is the predictive imageDCT


The MPEG-1 Encoder (3/4)QuantizationIntra quantization matrixInter quantization matrix


The MPEG-1 Encoder (4/4)Motion CompensationExploit motion vector and the previous reconstructed frame to generate the predictive frame is the compensated image is the previous image is the motion vectorReference frameTarget frame


The MPEG-2 StandardField/Frame DCT CodingField/Frame Prediction Mode SelectionAlternative Scan Order Various Picture Sampling FormatsUser Defined Quantization Matrix


Progressive Scan and Interlaced ScanProgressive ScanInterlaced Scan


Field/Frame DCT CodingThe field type DCT Fast motion videoThe frame type DCT Slow motion video


Alternative Scan OrderZigzag scan orderFrame DCTAlternative scan orderField DCT


The MPEG-2 Encoder (2/2)QuantizationUser can change the quantization if necessaryIntra quantization matrixInter quantization matrix

Various picture sampling formats4:4:44:2:2 4:2:0


The MPEG-2 Encoder (1/2)Base LayerBasic quality requirementFor SDTVEnhanced LayerHigh quality serviceFor HDTVDCTQEntropyCodingQ-1IDCTMotionCompensationFrameMemoryMotionEstimationQEntropyCodingQ-1+++++Q-1Bits EnhanceSNR Enhanced Layer+Base LayerBits Base


H.264/AVCVariable Block SizeMultiple Reference FramesInteger TransformIntra PredictionIn-loop Deblocking Filtering1/4-pel Resolution Motion Estimation CAVLC


Variable Block SizeThe fixed block size may not be suitable for all motion objectsImprove the flexibility of comparisonReduce the error of comparison7 types of blocks for selection


Multiple Reference FramesThe neighboring frames are not the most similar in some casesThe B-frame can be reference frameB-frame is close to the target frame in many situations


Integer Transform for Reducing The Spatial Redundancy (1/2)The transform matrix C

44 Block SizeSeparable Integer TransformThe transform coefficients are CXCT


Integer Transform for Reducing The Spatial Redundancy (2/2)CXCTE


Intra PredictionPredict the similarity between the neighboring pixels in one frame in advance, and exploit differential coding transform coding to remove the redundancy. Vertical right


Remove Perceptual RedundancyIn-loop deblocking filteringRemove blocking artifactResult from block based motion compensationResult from block based transform codingQP


1/4-pel Resolution Motion Estimation


The H.264/AVC EncoderTransform/QuantizationEntropyCodingInverseTransform/De-QuantizationMotionCompensationDe-blockingFilterMotionEstimationResidueMotion VectorIntra-framePredictionCoderControllerControl Data


H.264 Reference SoftwareJMCurrent software version: JM 15Benchmarkhttp://iphome.hhi.de/suehring/tml/


Reference Software Demo


Future WorkFast Mode Decision AlgorithmInterpolation Filter DesignDeblocking Filter DesignDCT-Based Motion EstimationImplementation Based on TI DSP


Fast Mode Decision AlgorithmThe computational cost of H.264 is largeVariable block-size ME Multiple reference framesFast mode decision is needed for reducing the computation time


Interpolation Filter DesignIn order to estimate and compensate the fractional-pel displacementsAdaptive Interpolation filter for replacing the fixed coefficient filter


Deblocking Filter DesignBlock based ME and Transform result in the annoying blocking artifactReduce the blocking artifact can increase the quality of compressed video


DCT Based Motion EstimationRobust even in noisy environmentComplexity comparisonDCT Based MEO(M2)Block Based MEO(N2M2)ConceptPseudo PhaseSimilar to


Implementation Based on TI DSPTMS320C6416

TI DM642


Conclusions

Thank YouQ & A


References (1/2)[1] Yun Q.Shi and Huifang Sun, Image and Video Compression for Multimedia Engineering: Fundamentals, Algorithms, and Standards, CRC press, 2000.[2] Yao Wand, Jorn Ostermann and Ya-Qin Zhang, Video Processing and Communications, Prentice Hall, 2007.[3] Richardson, Lain E. G., Video Codec Design: Developing Image and Video Compression Systems, John Wiley & Sons Inc, 2002.[4] Barry G, Haskell, Atul Puri and Arun N. Netravali, Digital Video : An Introduction to MPEG-2, Boston : Kluwer Academic, 1999.[5] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, Overview of the H.264/AVC video coding standard, IEEE Trans. on Circuits and systems for video Technology, vol. 13, no. 7, pp. 560-576, July 2003.[6] G. Sullivan and T. Wiegand, Video Compression - From Concepts to the H.264/AVC Standard, Proceedings of the IEEE, Special Issue on Advances in Video Coding and Delivery, December 2004.[7] 2004.


References (2/2)[8] Thomas Wedi, Adaptive Interpolation Filters and High-Resolution Displacements For Video Coding, IEEE Trans. on Circuits and Systems For Video Technology, vol.[9] Dong-Hwan Kim, Hwa-Yong Oh, Oguzhan Urhan, Sarp Ertrk and Tae-Gyu Chang, Optimal Post-Process/In-Loop Filtering for Improved Video Compression Performance, IEEE Trans. on Consumer Electronics, vol. 53, no. 4, Nov. 2007.[10] Shu-Fa Lin, Meng-Ting Lu, and Homer Chen, Fast Multi-Frame Motion Estimation for H.264 and Its Applications to Complexity-Aware Streaming, IEEE International Symposium on Circuits and Systems, 2005.[11] Kai-Ting Cheng and Hsueh-Ming Hang, Acceleration and Implementation of H.264 Encoder and Scalable Extension of H.264 Decoder on TI DSP Platform, master thesis, June 2007


digital video compression fundamentals and standards

Documents