Basic concept of Pixel and MPEG data structure

• Basic concept– Pixel format– MPEG data structure– En/decode basic concept

Pixel format• Usually framebuffer use RGB format

– For example, rgb565 is a 16bits pixel that [4:0] bits represent blue, [10:5] bits represent green, [15:11] bits represent red.

– Display controller grab pixel data from framebuffer, transfer it to LCD panel.• If data is 0xffff, Panel will trun on maximum voltage on RGB component. • Then this light look just like white color.

2

Pixel format• Every pixel component in panel contains three component.

3

Pixel format• YUV is usually the format of en/decode process. – Y represent the luminance.– Human is sensitive to ‘Y’, not ‘U’&’V’, even ‘U’&’V’ has

lower resolution than ‘Y’.– It can be converted to RGB format.

4

Pixel format• Interlaced and progressive

format

5

Pixel format• Planar and packed format.

6

MPEG data structure• The MPEG 2 video data stream is constructed in layers from lowest to highest as

follows:– - PIXEL is the fundamental unit– - BLOCK is an 8 x 8 array of pixels– - MACROBLOCK consists of 4 luma blocks and 2 chroma blocks– - Field DCT Coding and Frame DCT Coding– - SLICE consists of a variable number of macroblocks– - PICTURE consists of a frame (or field) of slices– - GROUP of PICTURES (GOP) consists of a variable number of pictures– - SEQUENCE consists of a variable number of GOP"s

• The resulting Elementary Stream may be packetized to form a Packetized Elementary Stream.

7

MPEG data structure

• Usually a frame is a VOP.• VOP types:– I-VOP: a frame encoded like a sill image.– P-VOP: a frame which encode only those

difference from previous frame.– B-VOP: a frame which encode only those

difference from previous frame , and also next frame.

8

MPEG data structure• Example of unpacked stream: GOP structure, M=3, N=9

– M means distance between I-VOP and P-VOP.– N means distance between I-VOPs.

9

MPEG data structure• Unpacked stream’s playback issue : Rearrange first

then delay display

– Reads and decode frame 0 (I), display nothing– Reads and decode frame 1 (P), display frame 0– Reads and decode frame 2 (B), display frame 2– Reads and decode frame 3 (B), display frame 3– Reads and decode frame 4 (P), display frame 1

10

MPEG data structure• Packed stream format : no need of delay display.

11

MPEG data structure

• Packetized elementary stream– A packetized elementary stream, or PES, consists of a single ES which

has been made into packets, each starting with an added packet header. A PES stream contains only one type of data from one source, e.g. from one video or audio encoder.

– PES packets have variable length, not corresponding to the fixed packet length of transport packets, and may be much longer than a transport packet.

– Each PES packet header includes an 8-bit stream ID identifying the source of the payload. Among other things, the PES packet header may also contain timing references.

12

En/decode basic concept

• Demux: – Separate video and audio stream from container

(PS/TS/avi/mov/mpg/rmvb)– Send each stream(packetized elementary stream)

to corresponding decoder.

13

En/decode basic concept• Decode– Decode read encoded frame from stream, one by one.– If consuming time of frame decoding is smaller than interval (e.g. 1/30s), there will be smooth play of multimedia.  

14

En/decode basic concept• Flow chart :

15