the vp8 video codec
DESCRIPTION
A presentation covering the VP8 video codecTRANSCRIPT
![Page 1: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/1.jpg)
The VP8 Video Codec
Multimedia CodecsSS 2011
Thomas Maier <[email protected]>Dominik Hübner <[email protected]>Sven Pfleiderer <[email protected]>
![Page 2: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/2.jpg)
Problem Definition● No standardized codec for web video● Currently used:
● H264: patent licensing royalties needed ● Theora: royalty free, outdated technology
● Heterogenous client hardware● Bandwidth constraints
![Page 3: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/3.jpg)
30.06.2011 The VP8 Video Codec 3
History● On2 Technologies developed VP8● Announced September 2008 to replace VP7● Acquisition of On2 by Google early 2010● Open letter from the Free Software Foundation
to Google demanding open sourcing of VP8
![Page 4: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/4.jpg)
30.06.2011 The VP8 Video Codec 4
History● Release of VP8 under a BSD-like license● Launch of the WebM and WebP projects● Faster VP8 decoder written by x264 developers
in July 2010● RFC draft of bitstream guide submitted to IETF
(not as a standard) in January 2011
![Page 5: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/5.jpg)
30.06.2011 The VP8 Video Codec 5
Patent Situation● Patent situation unclear● VP8 affects patents of h264
● Possible prior art by Nokia in ~2000● MPEG LA announced a call for patents against
VP8
![Page 6: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/6.jpg)
30.06.2011 The VP8 Video Codec 6
The WebM-Project● Founded by Google in May 2010● Royalty free media file format● Open-sourced under a BSD-style license● Optimized for the web
● Low computational complexity● Simple container format● Click and encode
![Page 7: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/7.jpg)
30.06.2011 The VP8 Video Codec 7
The WebM-Project● Container is a subset of Matroska
● VP8 for video● Vorbis for audio
● *.webm extension● Internet media types
● video/webm● audio/webm
![Page 8: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/8.jpg)
30.06.2011 The VP8 Video Codec 8
Web Video● HTML5 video tag < video >
● Replacement for Flash and Silverlight● Customizable video controls with CSS● Scriptable with standardized JavaScript APIs
● No standardized video format● h264● VP8● Theora
![Page 9: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/9.jpg)
30.06.2011 The VP8 Video Codec 9
Application
http://en.wikipedia.org/wiki/HTML5_video#Table
Browser Theora H.264 VP8 WebMInternet Explorer
Manual Install
9.0 Manual Install
Mozilla Firefox
3.5 No 4.0
Google Chrome
3.0 Yes (removed in future)
6.0
Safari Manual Install
3.1 Manual Install
Opera 10.50 No 10.60Konquerer 4.4 Depends on
QTYes
Epiphany 2.28 Depends on GStreamer
Depends on GStreamer
![Page 10: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/10.jpg)
30.06.2011 The VP8 Video Codec 10
Application● Youtube successively converts to VP8● Flash support announced
● Important for DRM● Skype 5.0● Nvidia announced 3D support
![Page 11: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/11.jpg)
30.06.2011 The VP8 Video Codec 11
ApplicationTools and libraries● GStreamer● FFmpeg● libvpx● ffvp8
![Page 12: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/12.jpg)
30.06.2011 The VP8 Video Codec 12
ApplicationHardware support● AMD● ARM● Broadcom● MIPS● Nvidia● Texas Instruments ● Open IP for hardware decoders
![Page 13: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/13.jpg)
VP8 in-depth
![Page 14: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/14.jpg)
30.06.2011 The VP8 Video Codec 14
Color Space● YUV 4:2:0 sub-sampling
R
G
B
Y
U
V
![Page 15: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/15.jpg)
30.06.2011 The VP8 Video Codec 15
VP8 Encoding Overview
![Page 16: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/16.jpg)
30.06.2011 The VP8 Video Codec 16
Block Generation
![Page 17: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/17.jpg)
30.06.2011 The VP8 Video Codec 17
Block GenerationY U V
16x16 8x8 8x8
Macroblock
![Page 18: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/18.jpg)
Prediction
![Page 19: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/19.jpg)
30.06.2011 The VP8 Video Codec 19
Intra Frame Prediction
![Page 20: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/20.jpg)
30.06.2011 The VP8 Video Codec 20
Intra Frame Prediction● Exploits spacial coherence of frames● Uses already coded blocks within current frame● Applies to macroblocks in an interframe as well
as to macroblocks in a key frame● 16x16 luma and 8x8 chroma components are
predicted independently
![Page 21: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/21.jpg)
30.06.2011 The VP8 Video Codec 21
Chroma Prediction Modes● H_PRED● V_PRED● DC_PRED● TM_PRED
![Page 22: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/22.jpg)
30.06.2011 The VP8 Video Codec 22
H_PRED● Horizontal Prediction● Fills each pixel column with a copy of left
neighboring column (L)● If current macroblock is on the left column, a
default value of 129 is assigned
![Page 23: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/23.jpg)
30.06.2011 The VP8 Video Codec 23
H_PRED
L0
L1
L2
L3
L
![Page 24: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/24.jpg)
30.06.2011 The VP8 Video Codec 24
V_PRED● Vertical Prediction● Fills each pixel row with a copy of the row
above (A)● If current macroblock is on the top column, a
default value of 127 is assigned
![Page 25: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/25.jpg)
30.06.2011 The VP8 Video Codec 25
V_PRED
A0 A1 A2 A3
A
![Page 26: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/26.jpg)
30.06.2011 The VP8 Video Codec 26
DC_PRED● Fills each block with a single value● This value is the average of the pixels left and
above of the block● If block is on the top: The average of the left
pixels is used● If block is on the left: The average of the above
pixels is used● If block is on the left top corner: A constant
value of 128 is used
![Page 27: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/27.jpg)
30.06.2011 The VP8 Video Codec 27
DC_PRED
A0 A1 A2 A3
L0
L1
L2
L3
AVG
L
A
![Page 28: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/28.jpg)
30.06.2011 The VP8 Video Codec 28
TM_PRED● TrueMotion Prediction● Uses above row A, left column L and a pixel P
which is above and left of the block● Most used intra prediction mode● X ij=LiA j−P
![Page 29: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/29.jpg)
30.06.2011 The VP8 Video Codec 29
TM_PRED
X21
A0 A1 A2 A3P
L0
L1
L2
L3
L
A
X21 = L2 + A1 - P
![Page 30: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/30.jpg)
30.06.2011 The VP8 Video Codec 30
TM_PRED Code
![Page 31: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/31.jpg)
30.06.2011 The VP8 Video Codec 31
Luma Prediction Modes● Basically all chroma prediction modes● With 16x16 macroblocks● Additional B_PRED mode
![Page 32: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/32.jpg)
30.06.2011 The VP8 Video Codec 32
B_PRED● Splits 16x16 macroblock into 16 4x4 sub-blocks● Each sub-block is independently predicted● Ten available prediction modes for sub-blocks
![Page 33: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/33.jpg)
30.06.2011 The VP8 Video Codec 33
B_PRED Modes● B_DC_PRED: predict DC using row above and
column● B_TM_PRED: propagate second differences a
la TM● B_VE_PRED: predict rows using row above● B_HE_PRED: predict columns using column to
the left● B_LD_PRED: southwest (left and down) 45
degree diagonal prediction
![Page 34: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/34.jpg)
30.06.2011 The VP8 Video Codec 34
B_PRED Modes● B_RD_PRED: southeast (right and down)● B_VR_PRED: SSE (vertical right) diagonal ● B_VL_PRED: SSW (vertical left)● B_HD_PRED: ESE (horizontal down)● B_HU_PRED: ENE (horizontal up)
![Page 35: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/35.jpg)
30.06.2011 The VP8 Video Codec 35
Motion Estimation
![Page 36: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/36.jpg)
30.06.2011 The VP8 Video Codec 36
Motion Estimation● Determine motion vectors which transform one
frame to another● Uses motion vectors for 16x16, 16x8, 8x16, 8x8
and 4x4 blocks● Motion vectors from neighboring blocks can be
referenced
![Page 37: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/37.jpg)
30.06.2011 The VP8 Video Codec 37
Motion Estimation● Motion vector: Horizontal and vertical
displacement● Only luma blocks are predicted, chroma blocks
are calculated from luma● Resolution: 1/4 pixel for luma, 1/8 pixel for
chroma● Chroma vectors are calculated by averaging
vectors from luma blocks
![Page 38: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/38.jpg)
30.06.2011 The VP8 Video Codec 38
Motion Vector Types● MV_NEAREST● MV_NEAR● MV_ZERO● MV_NEW● MV_SPLIT
![Page 39: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/39.jpg)
30.06.2011 The VP8 Video Codec 39
MV_NEAREST● Re-use non-zero motion vector of last decoded
block
![Page 40: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/40.jpg)
30.06.2011 The VP8 Video Codec 40
MV_NEAR● Re-use non-zero motion vector of second-to-
last decoded block
![Page 41: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/41.jpg)
30.06.2011 The VP8 Video Codec 41
MV_ZERO● Block has not moved● Block is at the same position as in preceding
frame
![Page 42: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/42.jpg)
30.06.2011 The VP8 Video Codec 42
MV_NEW● New motion vector● Mode followed by motion vector data● Data is added to buffer of last encoded blocks
![Page 43: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/43.jpg)
30.06.2011 The VP8 Video Codec 43
MV_SPLIT● Use multiple motion vectors for a macroblock● Macroblock can be split up into sub-blocks● Each sub-block can have its own motion vector● Useful when objects within a macroblock have
different motion characteristics
![Page 44: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/44.jpg)
30.06.2011 The VP8 Video Codec 44
Motion Compensation
![Page 45: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/45.jpg)
30.06.2011 The VP8 Video Codec 45
Motion Compensation● Apply motion vectors to previous frame● Generate a predicted frame● Only difference between predicted and actual
frame needs to be transmitted
![Page 46: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/46.jpg)
30.06.2011 The VP8 Video Codec 46
Sub-pixel Interpolation● If “full pixel” motion vector, block is copied to
corresponding piece of the prediction buffer● If at least one of the displacements affects sub-
pixels, missing pixels are synthesized by horizontal and vertical interpolation
![Page 47: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/47.jpg)
30.06.2011 The VP8 Video Codec 47
Inter Frame Prediction
![Page 48: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/48.jpg)
30.06.2011 The VP8 Video Codec 48
Inter Frame Prediction
Exploits the temporal coherence between nearby frames
Components:● Reference Frames● Motion Vectors
![Page 49: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/49.jpg)
30.06.2011 The VP8 Video Codec 49
Inter-Frame Types● Key Frames
● Decoded without reference to other frames● Provide seeking points
● Predicted Frames● Decoding depends on all prior frames up to last
Key-Frame● No usage of B-Frames
![Page 50: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/50.jpg)
30.06.2011 The VP8 Video Codec 50
Prediction Frame Types● Previous Frame● Alternate Reference Frame● Golden Reference Frame● Each of these three types can be used for
prediction
![Page 51: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/51.jpg)
30.06.2011 The VP8 Video Codec 51
Previous Frame● Last fully decoded frame● Updated with every shown frame
![Page 52: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/52.jpg)
30.06.2011 The VP8 Video Codec 52
Alternate Reference Frame● Fully decoded frame buffer● Can be used for noise reduced prediction● In combination with golden frames:
Compensate lack of B-frames
![Page 53: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/53.jpg)
30.06.2011 The VP8 Video Codec 53
Golden Reference Frame● Fully decoded image buffer● Can be partially updated● Can be used for error recovery● Can be used to encode a cut between scenes
![Page 54: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/54.jpg)
30.06.2011 The VP8 Video Codec 54
Updating Frame Buffers● Key frame: Updates all three buffers● Predicted frame: Flag for updating alternate or
golden frame buffer
![Page 55: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/55.jpg)
30.06.2011 The VP8 Video Codec 55
Error Recovery
Source: http://webm.googlecode.com/files/Realtime_VP8_2-9-2011.pdf
![Page 56: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/56.jpg)
Transformation
![Page 57: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/57.jpg)
30.06.2011 The VP8 Video Codec 57
Transformation
![Page 58: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/58.jpg)
30.06.2011 The VP8 Video Codec 58
Decorrelation● Necessary for efficient entropy encoding● Achieved with hybrid transformation
● Discrete Cosine Transformation● Walsh-Hadamard Transformation
![Page 59: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/59.jpg)
30.06.2011 The VP8 Video Codec 59
88
16
16Y
U/V
4
4Y
4
4U4 *
16 *
4
4V4 *
TransformationPreparation for transfomation process: Divide Macroblocks into Subblocks
Frame SubblockMacroblock
![Page 60: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/60.jpg)
30.06.2011 The VP8 Video Codec 60
Discrete Cosine Transformation● 16 luma blocks / 4 + 4 chroma blocks● Transform each block into spectral components
using the 2D - DCT
∣255 0 255 0255 0 255 0255 0 255 0255 0 255 0
∣ ∣510 195.1686 0 471.17860 0 0 00 0 0 00 0 0 0
∣DCT
Values based on dct2() function of Matlab
![Page 61: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/61.jpg)
30.06.2011 The VP8 Video Codec 61
TransformationThe DC components of all subblocks are often correlated among each other
∣85 85 85 8585 85 85 8585 85 85 8585 85 85 85
∣
∣145 145 145 145145 145 145 145145 145 145 145145 145 145 145
∣
DCT ∣340 0 0 00 0 0 00 0 0 00 0 0 0
∣
∣580 0 0 00 0 0 00 0 0 00 0 0 0
∣DCT
Values based on dct2() function of Matlab
Macroblocks
![Page 62: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/62.jpg)
30.06.2011 The VP8 Video Codec 62
Walsh-Hadamard Transformation● Use the correlation of the DC components with a 2nd
order transformation● The WHT works with a simple transformation matrix
→ Transformation is a matrix multiplication
H=∣1 1 1 11 1 −1 −11 −1 1 −11 −1 −1 1
∣ H= 14∣1 1 1 11 1 −1 −11 −1 1 −11 −1 −1 1
∣Normalized Walsh-Hadamard matrix
![Page 63: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/63.jpg)
30.06.2011 The VP8 Video Codec 63
Walsh-Hadamard TransformationExample
A=∣340 340 340 340340 340 340 340580 580 580 580580 580 580 580
∣1st order transformation DC components
H∗A∗H
(Re) Transformation
B=H∗A=∣1840 1840 1840 1840−480 −480 −480 −4800 0 0 00 0 0 0
∣ C=B∗H=∣1840 0 0 0−480 0 0 00 0 0 00 0 0 0
∣
H=∣1 /2 1 /2 1 /2 1 /21 /2 1 /2 −1 /2 −1 /21 /2 −1/2 1 /2 −1 /21 /2 −1/2 −1 /2 1 /2
∣Normalized transformation matrix
![Page 64: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/64.jpg)
Quantization
![Page 65: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/65.jpg)
30.06.2011 The VP8 Video Codec 65
Quantization
![Page 66: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/66.jpg)
30.06.2011 The VP8 Video Codec 66
Quantization● Quantization of the transformation coefficients:
● Less data per coefficient● More zeros!
● Scalar quantization● Designed for quality range of
~30dB to ~45dB SNR
![Page 67: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/67.jpg)
30.06.2011 The VP8 Video Codec 67
QuantizationFor each frame different factors for:● 1st order luma DC● 1st order luma AC● 2nd order luma DC● 2nd order luma AC● Chroma DC● Chroma AC
AC
AC
ACDC
DC
DC
1st order luma(DCT)
2nd order luma(WHT)
Chroma(DCT)
![Page 68: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/68.jpg)
30.06.2011 The VP8 Video Codec 68
Quantization● 128 quantization levels with given factors● Quantization table for DC coefficients in Y1 planes
![Page 69: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/69.jpg)
30.06.2011 The VP8 Video Codec 69
Quantization
Example: 1st order luma AC coefficients ● Quantization level: 3
→ Quantization factor from table: 6● DC coefficient is ignored here
A=∣−312 7 1 01 12 −5 22 −3 3 −11 0 −2 1
∣ Q=round 16∗A =∣0 1 0 0
0 2 −1 00 −1 1 00 0 0 0
∣
![Page 70: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/70.jpg)
30.06.2011 The VP8 Video Codec 70
QuantizationAdaptive Quantization● Up to 4 different segments (q0-q3)● Each segment with n macroblocks and its own
quantization parameter set
![Page 71: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/71.jpg)
30.06.2011 The VP8 Video Codec 71
QuantizationThe quantized coefficients are read in zig-zag order
0-19 1 0
0 2 -1 0
-1 0 0 0
0 0 0 0
vals=[−19,1,0,−1, 2,0 ,0 ,−1,0 ,0,0,0,0,0, 0,0]
![Page 72: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/72.jpg)
Adaptive Loop Filtering
![Page 73: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/73.jpg)
30.06.2011 The VP8 Video Codec 73
Adaptive Loop Filtering
![Page 74: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/74.jpg)
30.06.2011 The VP8 Video Codec 74
Adaptive Loop FilteringProblem● Strong quantization (“worst” case: only DC)● Many pixels with same values● Blocking artifacts
A=∣128 128 128 128128 128 128 128128 128 128 128128 128 128 128
∣ 64128
4 4
4B=∣64 64 64 6464 64 64 6464 64 64 6464 64 64 64
∣Subblocks
![Page 75: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/75.jpg)
30.06.2011 The VP8 Video Codec 75
Adaptive Loop Filtering● VP8 has two filter modes
– Simple– Normal
● Configuration in frame-header● Two parameters
– loop_filter_level– sharpness_level
![Page 76: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/76.jpg)
30.06.2011 The VP8 Video Codec 76
Adaptive Loop Filtering
● Filter order per macroblock1. Left macroblock edge2. Vertical subblock edges3. Macroblock edge at the top4. Horizontal subblock edges
3 2
1
4
● Macroblock processing in scan line order
![Page 77: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/77.jpg)
30.06.2011 The VP8 Video Codec 77
2
4
6
8
Adaptive Loop FilteringFilter segments● n segments per edge
n = blocklength● 2,4,6 or 8 taps wide● Pixels before edge: px● Pixels after edge: qx
p2 p1 p0 q1 q2q0
![Page 78: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/78.jpg)
30.06.2011 The VP8 Video Codec 78
Adaptive Loop FilteringSimple Mode● Segments 4 or 6 taps wide● sharpness_level ignored● Filter edge if total difference > threshold● Threshold derived from loop_filter_level,
quantization level and other factors
![Page 79: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/79.jpg)
30.06.2011 The VP8 Video Codec 79
4 4
4
Adaptive Loop FilteringSimple Mode – Example
p1 p0 q0 q1
p1: 128p0: 128q0: 64q1: 64
q=3∗64644
=64
p=3∗1281284
=128
a=q−p4
=64−1284
=−16
p0=p0a=128−16=122
q0=q0−a=128−−16=80
![Page 80: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/80.jpg)
30.06.2011 The VP8 Video Codec 80
Adaptive Loop FilteringNormal Mode● Segments 2,4,6 or 8 taps wide● Different adjustments for different positions● Different weightings for inner positions
![Page 81: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/81.jpg)
30.06.2011 The VP8 Video Codec 81
Adaptive Loop FilteringAdaptive?
Heavy Motion → Strong Filtering
Low Motion → Slight Filtering
No Motion → No Filtering
![Page 82: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/82.jpg)
30.06.2011 The VP8 Video Codec 82
Adaptive Loop FilteringSIMD processors aka Vector CPUs● Loop filter optimized for SIMD operations● Sources already implemented
![Page 83: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/83.jpg)
30.06.2011 The VP8 Video Codec 83
Adaptive Loop FilteringProblem: Dependencies between macroblocks
0 1
m
m Macroblocks
![Page 84: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/84.jpg)
Entropy Coding
![Page 85: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/85.jpg)
30.06.2011 The VP8 Video Codec 85
Entropy Coding
![Page 86: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/86.jpg)
30.06.2011 The VP8 Video Codec 86
Frame Format● Frames are divided in 3 partitions
● Uncompressed header chunk● Macroblock coding modes and motion vectors● Quantized transform coefficients
FrameHeader Partition 1 Partition 2
![Page 87: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/87.jpg)
30.06.2011 The VP8 Video Codec 87
Entropy Encoding● Entropy coding minifies redundancy
● 2 steps ● Huffman Tree with a small alphabet● Binary arithmetic coding
![Page 88: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/88.jpg)
30.06.2011 The VP8 Video Codec 88
Entropy Encoding● DCT and WHT coefficients are precoded to
tokens using a predefined tree structure
● Goal● Reduce number of reads from raw binary stream
● Solution ● Create tokens for symbol values● Minimize necessary reads for most frequent
symbols
![Page 89: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/89.jpg)
30.06.2011 The VP8 Video Codec 89
Entropy Encoding● Token types
● Single numbers– Coefficient value
● 0, 1, 2, 3, 4 ● Number ranges
– 6 ranges of coefficient values ● 5-6, 7-10, 11-18, 19-34, 35-66, 67-2048
● EOB (End Of Block)– No more non-zeros values remaining in macroblock
![Page 90: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/90.jpg)
30.06.2011 The VP8 Video Codec 90
Entropy Encoding● How are these tokens created?
● Step 1: Read quantized DCT/WHT coefficientsfrom 4x4 sub-blocks
∣187 0 0 02 0 0 01 0 0 00 0 0 0
∣ 187, 0, 2, 1, 0, 0, 0, 0, 0, ...
![Page 91: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/91.jpg)
30.06.2011 The VP8 Video Codec 91
Entropy Encoding● Step 2: Lookup regarding tokens for each value
Remaining values: 187, 0, 2, 1, 0, 0, 0, 0, 0, ...Output:
![Page 92: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/92.jpg)
30.06.2011 The VP8 Video Codec 92
Entropy Encoding● Step 2: Lookup regarding tokens for each value
Remaining values: 187, 0, 2, 1, 0, 0, 0, 0, 0, ...Output: 11111111
![Page 93: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/93.jpg)
30.06.2011 The VP8 Video Codec 93
Entropy Encoding● Step 2: Lookup regarding tokens for each value
Remaining values: 187, 0, 2, 1, 0, 0, 0, 0, 0, ...Output: 11111111 10
![Page 94: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/94.jpg)
30.06.2011 The VP8 Video Codec 94
Entropy Encoding● Step 2: Lookup regarding tokens for each value
Remaining values: 187, 0, 2, 1, 0, 0, 0, 0, 0, ...Output: 11111111 10 1100
Why not 11100? We can save 1 bit!
![Page 95: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/95.jpg)
30.06.2011 The VP8 Video Codec 95
Entropy Encoding● Step 2: Lookup regarding tokens for each value
Remaining values: 187, 0, 2, 1, 0, 0, 0, 0, 0, ...Output: 11111111 10 1100 110
![Page 96: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/96.jpg)
30.06.2011 The VP8 Video Codec 96
Entropy Encoding● Step 2: Lookup regarding tokens for each value
Remaining values: 187, 0, 2, 1, 0, 0, 0, 0, 0, ...Output: 11111111 10 1100 110 0
![Page 97: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/97.jpg)
30.06.2011 The VP8 Video Codec 97
Entropy Encoding● Restoring coefficients from value ranges
● Add some extra bits as offset from base of the current range
Output: 11111111 10 1100 110 0
Range: 67 – 2048Number: 187Offset: 187 – 67 = 120
Extra Bits: 11Binary Offset: 0000 0111 1000
New Output: 11111111 0000 0111 1000 10 1100 110 0
![Page 98: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/98.jpg)
30.06.2011 The VP8 Video Codec 98
Entropy Encoding● Binary arithmetic encoding
● Extra bits are encoded with pre-set, constant probabilities
● Token probabilities reside in 96 probability tables● Token bits are encoded with
– Default probabilities whenever keyframes are updated– Regarding probability tables can be updated with each
new frame
![Page 99: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/99.jpg)
30.06.2011 The VP8 Video Codec 99
Entropy Encoding● Binary arithmetic encoding
● Token probability tables are chosen according to 3 contexts– Plane (Y, U, V)– Band (position of the coefficient)– Local complexity (value of the preceding coefficient)
![Page 100: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/100.jpg)
30.06.2011 The VP8 Video Codec 100
Entropy Encoding● Binary arithmetic encoding
![Page 101: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/101.jpg)
Parallel Processing
![Page 102: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/102.jpg)
30.06.2011 The VP8 Video Codec 102
Parallel Processing● Partition 2 (DCT/WHT coefficients) can be
divided in 8 sub-partitions
FrameHeader Partition 1 Partition 2
Sub-Partition ...
Sub-Partition 3
Sub-Partition 2
Sub-Partition 1
Sub-Partition 8
![Page 103: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/103.jpg)
30.06.2011 The VP8 Video Codec 103
Parallel Processing● Partition 2 (DCT/WHT coefficients) can be
divided sub-partitions● Support for up to 8 cores
Core 1
Core 2
Core 3
Core 4
![Page 104: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/104.jpg)
30.06.2011 The VP8 Video Codec 104
BenchmarksTools● ffmpeg● libvpx● libx264● custom scripts● qpsnr (qpsnr.youlink.org)
![Page 105: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/105.jpg)
30.06.2011 The VP8 Video Codec 105
Benchmarks
![Page 106: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/106.jpg)
30.06.2011 The VP8 Video Codec 106
Benchmarks
![Page 107: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/107.jpg)
Demos
![Page 108: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/108.jpg)
30.06.2011 The VP8 Video Codec 108
Conclusions● “Good enough” for web video● Maybe new default choice for web video● “Thereʼs no way in hell anyone could write a
decoder solely with this spec alone.” - x264 developer
● Patent situation still unclear
![Page 109: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/109.jpg)
30.06.2011 The VP8 Video Codec 109
Conclusions
![Page 110: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/110.jpg)
30.06.2011 The VP8 Video Codec 110
Resources● http://x264dev.multimedia.cx● http://multimedia.cx/eggs● http://www.slideshare.net/DSPIP/google-vp8● http://qpsnr.youlink.org/vp8_x264/VP8_vs_x264.html● http://tools.ietf.org/html/draft-bankoski-vp8-bitstream-01● Google VP8 Paper
![Page 111: The VP8 Video Codec](https://reader034.vdocuments.site/reader034/viewer/2022052310/5404a5688d7f72a6768b47fb/html5/thumbnails/111.jpg)
Questions?