shengkai lin@cmlab, ntucsie 2004 1 ch. 6- h.264/avc part i (pp.160~199) sheng-kai lin

Shengkai Lin@CMLAB, NTUCSIE 2004

1

Ch. 6- H.264/AVCCh. 6- H.264/AVCPart I (pp.160~199)Part I (pp.160~199)Ch. 6- H.264/AVCCh. 6- H.264/AVC

Part I (pp.160~199)Part I (pp.160~199)

Sheng-kai LinSheng-kai Lin


2

Outline• Introduction• Baseline profile

– Reference Picture Management– Slice– Inter-prediction– Intra-prediction– Deblocking filter– Transform and Quantisation


3

H.264/AVC Introduction (I)

• 4:2:0• Define VCL and NAL

– VCL: Video Coding Layer– NAL: Network abstraction

• Variable block size• Multiple reference frame• ¼ sample motion compensation


4

H.264/AVC Introduction (II)

• CAVLC• CABAC


5

Terms (I)• Picture (frame)

– one or more slices

• Slice– group of macroblock (1~ )– minimal inter-dependency between

coded slices can limit the error propagation

• Macroblock


6

Terms (II)• I slice

– I macroblock only

• P slice– P, I macroblock

• B slice– B,I macroblock

• SI slice• SP slice


7

Terms (III)• I macroblock

– intra prediction from decoded samples in the current slice• for entire macroblock• for each 4*4 block

• P macroblock– inter prediction using past references

• 16*16, 16*8, 8*16, 8*8, 4*8, 8*4, 4*4


8

Terms (IV)• B macroblock

– inter prediction using past/ future references

• list 0– past reference frame buffer

• list 1– future reference frame buffer

• RBSP (Raw Byte Sequence Payload)– NAL payload


9

H.264/AVC profiles


10

H.264/AVC Encoder


11

H.264/AVC Decoder


12




13

Reference Picture Management (I)

• Reference pictures are stored in decoded picture buffer (DPB)

• short/long term reference picture• A decoded frame may be marked as

– unused for reference– short term picture– long term picture


14

Reference Picture Management (II)

• “Sliding Window” memory management– Keep #(long_term_pic+ short_term_pic)– Remove short term picture if lack of space

• Adaptive memory control– issued by encoder– change the type of the ref frame

• IDR (Instantaneous Decoder Refresh)– clear ref buffer– I frame


15




16

Redundant coded picture

• Send the duplicated part or all of a coded picture

• Error resilience


17

Arbitrary Slice Order (ASO)

• The decoding order of the slices could be arbitrary

• Application example– reduce end-end transmission delay in

RT app


18

Slice Group (I)• former called “FMO”

– Flexible Macroblock Ordering• a subset of the macroblocks and may

contain one or more slices• Application example

– Error resilience


19

Slice Group (II)


20




21

Inter Prediction• Variable block size• ¼ pixel motion compensation


22

Interpolation Samples (I)

1. Generate ½ samples by six tap FIR• b, h, m, s • FIR para :(1/32, -5/32, 5/8, 5/8, -5/32, 1/32)

2. Generate center ½ sample by FIR• j

3. Generate ¼ sample by linear interpolation


23

Interpolation Samples (II)


24

Motion vector prediction (I)

• Encoding MV can cost a lot of bits– especially small partition sizes

• Neighboring MV is highly correlated– predictable– encode the diff MVD and transmit it


25

Motion vector prediction (II)

• Partitions excluding 16 × 8 and 8 × 16, MVp = (MVA+ MVB + MVC) /3

• For 16 × 8 partitions, MVp of the upper 16 × 8 = MVB ;MVp for the lower 16 × 8 = MVA


26

Motion vector prediction (III)

• For 8 × 16 partitions, MVp for the left 8 × 16 = MVA ;MVp for the right 8 × 16 = MVC

• For skipped macroblocks, do as 16 × 16 Inter mode


27

Motion vector prediction (IV)


28




29

Intra Prediction• For Luma samples

– 4*4 block• 9 prediction modes

– 16*16 block• 4 modes

– I_PCM• transmit the encoded samples w/o pred. & tra

ns


30

4*4 luma prediction modes


31

16*16 prediction modes


32

8*8 Chroma Prediction Modes

• Similar to 16*16 luma intra-prediction except the different numbering.– DC(mode 0) – horizontal (mode 1)– vertical(mode 2) – plane (mode 3).


33

Signaling Intra Prediction Modes (I)

• The choices of the mode should be signaled to the decoder.– might cost lots of bits.

• Intra modes for neighboring 4 × 4 blocks are often correlated– predict the parameters !

• The prediction mode for luma coded in Intra-16×16 mode or chroma coded in Intra mode is signalled in the macroblock header and the following method is not used.


34

Signaling Intra Prediction Modes (II)

• If A and B are available, C = min (A,B)• else if (neither A nor B are available) C = 2

(DC)• else C = available (A,B)

B

CA


35

Signaling Intra Prediction Modes (III)

• Flag prev_intra4×4_pred_mode– 1: most probable prediction mode is

used.– 0: Flag rem_intra4×4_pred_mode

indicates the change of the mode


36

Signaling Intra Prediction Modes (IV)

• If rem_intra4×4_pred_mode < most_probable_mode, the prediction mode = rem intra4×4 pred mode

• else the prediction mode = (rem_intra4×4_pred_mode+1)


37




38

Deblocking filter• Filter 4 vertical/horizontal boundaries of luma• Filter 2 vertical/horizontal boundaries of chroma• Affect up to 3 samples on the either side.• The filter is stronger at places where there is likely

to be significant blocking distortion– e.g.: such as the boundary of an intra coded macroblock

or a boundary between blocks that contain coded coefficients.


39




40

Transform and Quantisation

• 3 transforms– DCT-base transform for all 4*4 residual b

lock– Hadamard transform for 4*4 luma DC co

efficient (in 16*16 intra)– Hadamard transform for 2*2 chroma DC

coefficient


41

DCT-base transform (I)• Integer transform• Might be zero mismatch• The code part of the algo need add and shift

only• Scale multiplication could be integrated into

quantiser.• Inverse operation could be down by 16-bit ar

ithmetic.


42

DCT-base transform (II)• a=1/2, b = (2/5)1/2, d = 1/2


43

Quantisation• Combine with the scale of the DCT• Skip the rest details


44

4*4 DC luma Transform and Quantisatoion

• |ZD(i, j)| = (|YD(i, j)| MF(0,0) + 2f ) >> (qbits +1)• sign (ZD(i, j)) = sign (YD(i, j))


45

2*2 DC chroma Transform and Quantisatoion

• |ZD(i, j)| = (|YD(i, j)| MF(0,0) + 2f ) >> (qbits +1)• sign (ZD(i, j)) = sign (YD(i, j))


46

• Thank you• Next time, we’ll start at CAVLC• Bye…

shengkai lin@cmlab, ntucsie 2004 1 ch. 6- h.264/avc part i (pp.160~199) sheng-kai lin

Documents

shengkai lin

quantisation slide

shengkai lin slide

picture frame

avc decoder slide

avc encoder slide

rt app slide

cavlc cabac slide