mpeg video compression technology and testing · 2012-08-25 · p x mpeg video compression...

p x MPEG Video Compression

Technology and Testing

MPEG Video Compression Technology and Testing

CONVERGENCE SYMPOSIUM

TEKTRONIX

Video Services Telecommunications Mobile Communications



Seminar Topics

•  Modern Television System –  Video and Compression Standards

•  MPEG-2 Compression •  MPEG-2 System •  Testing in Compressed Systems

–  Traditional test methods –  Picture quality assessment – MPEG-2 protocol analysis



Other Programs

Program Demultiplexing Transmission

Program Decompression

Program Display or

Reuse

Program Production

Program Compression

Multi-Program Multiplexing

Modern Television System



Video Production Formats

R G B

Component Analog Video (RGB)

4xFsc Sample (PAL=17.7 MHz)

(NTSC=14.4 MHz)

270 MHz clock

Component Analog Video (R-Y, B-Y, Y)

Y B-Y R-Y

10 bits

6.75 MHz Sample

13.5 MHz Sample

Multiplexed 27 Mwords/Sec

...Y / R-Y / Y / B-Y / Y...

Clock X10

Parallel Digital Component

Serial Digital Component 270 Mb/Sec “Rec 601”

Serial Digital Composite (144 Mb/Sec) (177 Mb/Sec)

Parallel Digital Composite

Y

B-Y

R-Y

C a m e r a

M a t r i x

Composite Encoder

A to D Conv Serializer

Analog Composite

Video

10 bits Parallel

A to D Conv

A to D Conv

A to D Conv

Serializer

AES/EBU Digital Audio



Video Compression Standards •  JPEG, still images (Joint Photographics Experts Group)

–  M-JPEG; motion JPEG, not a standard, generally proprietary

•  H.261 (px64), video conferencing –  px64 kb/s (p=1, 2,…. 32)

•  H.263, video conferencing, emphasis on low bitrates •  MPEG-1, CD-ROM and multimedia (Motion Picture Experts Group) •  ETSI 300 174, Broadcast distribution and contribution •  MPEG-2, Broadcast entertainment/contribution and DVD •  Non-DCT methods

–  Wavelets, Fractal, DPCM –  Lossless (e.g., special JPEG mode)

•  MPEG-4, very low bitrate coding (possibly wavelets)



Typical Video Data Rates

•  10-bit Rec 601 270 Mbps •  8-bit Rec 601 216 Mbps •  8-bit Rec 601 (active only) 167 Mbps •  Digital Betacam ~90 Mbps •  MPEG-2 4:2:2P@ML 15-50 Mbps •  MPEG-2 MP@ML 1.5-15 Mbps •  MPEG-1 constrain. param. 0.5-1.8 Mbps •  H.261 videoconferencing 64 kbps - 1.5 Mbps •  H.263 videoconferencing 4 kbps - 0.5 Mbps



MPEG-2 Applications •  RF Transmission

–  DVB-S Digital Video Broadcasting - Satellite –  DVB-C Cable, DVB-T Terrestrial broadcast –  ENG Electronic (satellite) news gathering

•  Broadband Network –  Contribution quality programs –  Video on demand

•  Storage Media –  DVD Digital Versatile Disk –  Video servers

•  Intra-studio –  Point-to-point (being developed by SMPTE) –  Networking (being specified by EBU/SMPTE)



MPEG-2 Standards Documents

MPEG-2 ISO/IEC 13818 •  Part 1 Systems •  Part 2 Video •  Part 3 Audio •  Part 4 Conformance testing (for 1, 2 and 3) •  Part 5 Software simulation •  Part 6 System extensions - DSM-CC

(Digital Storage Media - Command & Control) •  Part 7 Audio extension - NBC (non backward compatible) •  Part 9 System extension RTI (real time interface) •  Part 10 Conformance extension - DSM-CC



Standards Organizations •  ITU

Place des Nations 1211 Geneve 20, Switzerland Ph: 41 22 730 6003 http://www.itu.ch

•  SMPTE 595 West Hartsdale Ave White Plains, NY 10607 Ph: 914-761-1100 http://www.smpte.org

•  ISO Case Postale 56 1 rue de Varembe 1211 Geneve 20, Switzerland Phone: +41 22 749 01 11 http://www.iso.ch

•  Documents for sale Global Engineering 15 Inverness Way East Englewood, CO 80112 Ph: 800-854-7179



Seminar Topics






MPEG-2 Video Compression

•  Pre-processing – Clean-up pictures and prepare video samples

•  Temporal Compression (IntER-frame) – Compresses the data from multiple frames

•  Spatial Compression (IntRA-frame) – Compresses the data within one frame –  (Similar to JPEG)

•  Rate Control – Constant bitrate – Constant (or nearly so) quality



Important Points about MPEG

•  Only specifics bitstream syntax and decoding •  Encoding algorithms are not defined

– Open to invention and generally proprietary –  Future improvements are compatible with all decoders

•  Asymmetric Compression –  Encoder is very complex –  Encoder contains a decoder model – Decoder definition emphasizes low complexity (cost)



Pre-Processing

•  Required – Decode from composite to component –  Produce correct picture size – Reduce 10-bit samples to 8-bit samples – Convert to 4:2:0 sampling (entertainment quality)

•  Optional – Noise reduction – Other picture clean up



Picture Sizes

•  Rec 601/656 525/30/2:1 720 x 486 •  Rec 601/656 625/25/2:1 720 x 576 •  MPEG-2 30 fps (quasi-std) 704 x 480 •  MPEG-2 422P@ML 30 fps 720 x 512 •  MPEG-2 422P@ML 25 fps 720 x 608 •  SIF (30fps, 25 fps) 352 x 240,288 •  CIF (always 30 fps) 352 x 240 •  QSIF (30fps, 25 fps) 176 x 128,144 •  QCIF (always 30 fps) 176 x 144



Rec 601 Component Video Sampling

(313)

(314)



4:2:2 Rec 601

1 Luminance sample Y

2 Chrominance samples Cb, Cr

4:2:0

4:1:1

4:2:0 Chroma Sub-Sampling



Redundancies

•  Spatial Redundancies – Redundant information in the horizontal and vertical picture

dimensions. Data that is similar or repeats itself in picture areas which are close to one another.

•  Temporal Redundancies – Redundant data over a given time. Data that is similar or

repeats itself from moment to moment, even if its location in the picture area changes.



Discrete Cosine Transform (Definition)

The NxN two dimensional DCT is defined as:

Encoder:

Decoder:



DCT Function

•  8 X 8 pixel Blocks are converted from the spatial domain to the spatial frequency domain.

•  Transformed blocks are numerically represented as 8 X 8 DCT coefficients.

•  DCT coefficients are more suitable for bit rate reduction techniques.

The transform process does not result in bit rate reduction.



DCT Example

Picture Sample Values DCT Coefficients

223 191 159 128 98 72 39 16

223 191 159 128 98 72 39 16

223 191 159 128 98 72 39 16

223 191 159 128 98 72 39 16

223 191 159 128 98 72 39 16

223 191 159 128 98 72 39 16

223 191 159 128 98 72 39 16

223 191 159 128 98 72 39 16

43.8 -40 0 -4.1 0 -1.1 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

720 Pixels

8x8 Pixels 480 Lines

(Pixels)



“Picture” of the DCT Coefficients H

V " " " " " "" ""

" """

"

"" "

" "

""

"

"

"

"

"

"

"

" "

" "

""

"

"

""

"

"

""

"

"

"

"

"

"

""

" "

"" "

"

"

"

"

"

"

"

"

"

"" "

" "

"

""

"""

"

"" "

" "

" ""

"

"

" ""

"

"

"

"

" ""

"

"

""

""

""

""

""

""

""

""

"

"

Horizontal spatial frequency waveforms



Quantization

•  Divides each DCT coefficient by a frequency-dependent value and truncates the results to an integer.

•  Many of the resulting integers are zero or small values (e.g., 1, 2, 3, … 12, 13, .)

•  Quantization coefficients can be tailored to complement limitations of the human visual system

•  Quantization causes information to be irretrievably lost. Reconstructed pixels usually differ in value from the original

7842 199 448 362 342 112 31 22

198 151 181 264 59 37 14 3

142 291 218 87 27 88 27 12

111 133 159 119 58 65 36 2

49 85 217 50 8 3 14 12

58 120 60 40 41 11 2 1

30 121 61 22 30 1 0 1

22 28 2 33 24 51 44 81

8 16 19 22 26 27 29 34

16 16 22 24 27 29 34 37

19 22 26 27 29 34 34 38

22 22 26 27 29 34 37 40

22 26 27 29 32 35 40 48

26 27 29 32 35 40 48 58

26 27 29 34 38 48 56 69

27 29 35 38 46 56 69 83

980 12 23 16 13 4 1 0

12 9 8 11 2 1 0 0

7 13 8 3 0 2 0 1

5 6 6 4 2 1 0 0

2 3 8 1 0 0 0 0

2 4 2 1 1 0 0 0

1 4 2 1 0 0 0 0

0 0 1 0 0 0 0 0

Code Linear Non-Linear Quant Scale Quant Scale

1 2 1

8 16 8

16 32 24

20 40 40

24 48 56

28 56 88

31 62 112

Input DCT Coefficients (a more complex block)

Quant Scale Values Not all code values are shown

One value used for complete 8x8 block

Output DCT Coefficients Value for display only

not actual results

Quant Matrix Values Value used corresponds

to the coefficient location

Divide by Quant Matrix

Divide by Quant Scale



Processing Sequences

Zigzag or Classic (nominally for frames)

Alternate (nominally for fields)



Entropy Coding

•  Run length coding uses a special code for repeating values (e.g., 13 “0s”, 5 “1s”, 4 “2s”)

•  Variable length coding uses shorter code words for more probable symbols (like Morse code)

Symbol A B C D E F

Probability 0.5 0.25 0.125 0.0625 0.03125 0.03125

Code Word 0 10 110 1110 11110 11111



Convert 4:2:2 to

8-bit 4:2:0 DCT Quantize Entropy

Coding Buffer Full

Bitrate 10-bit Data

Compressed Data

Rate Control

Information lost Data reduced

No Loss No Data reduced

Data reduced (information lost)

Data reduced (no loss)

Quantizing Reduce the number of bits for each coefficient.

Give preference to certain coefficients. Reduction can differ for each coefficient.

Variable Length Coding Use short words for

most frequent values (like Morse Code)

Run Length Coding Send a unique code

word instead of strings of zeros

Entropy Coding

Quantizing Data

INTRA-Frame Coding



Redundancies

•  Spatial Redundancies – Redundant information in the horizontal and vertical picture

dimensions. Data that is similar or repeats itself in picture areas which are close to one another.

•  Temporal Redundancies – Redundant data over a given time. Data that is similar or

repeats itself from moment to moment, even if its location in the picture area changes.



Temporal Redundancies •  Frame to Frame redundancies •  New location same data

•  New data uncovered



Motion Estimation

Frame N Frame N + 1

Macro Block 16x16 Pixels

Search Range

Motion Vector



INTER-Frame Coding (Forward Prediction)

Motion Vectors

Predicted Present Frame

Video in

Present Frame

Motion Vectors

Difference Frame Much less information if

the prediction is good

Previous Frame or Reconstructed Present Frame to use in the next Prediction

Motion Estimation

Motion Compensation

Fixed Store (previous frame)

Subtract/Pass

SUM



MPEG Video Compression P-pictures only

Motion Vectors

Motion Vectors

Predicted Present Frame

Video in

Rate Control

Quantizing Data

Group of Pictures Control

Motion Estimation

Motion Compensation

Fixed Store

Subtract DCT Q RLC VLC MUX

Buffer

Q-1

DCT-1

SUM



N = Distance between anchors

Group of Pictures

I pictures: Inter-coding only P pictures: Contain forward motion compensation B pictures: Contain forward, backward & bi-directional motion compensation

I M = Distance between I pictures

I P B P B B B B

Bi-directional Prediction

Forward Prediction

0 1 2 3 4 5 6 7



15/3 Frame Sequence I

B B

P B

B P

B B

P B

B P

B B

I



Time Sequence of Pictures



MPEG-1

•  Design focused on non-interlaced SIF (352x240) •  Application was media storage e.g., CD-ROM •  Uses most of the H.261 techniques •  Introduced the concept of B-frames •  Trick modes are supported

–  Fast search – Reverse, etc

•  Used in early DTV testing



MPEG-2

•  MPEG-2 = MPEG-1 + interlace tools + Profiles & Levels •  New field/frame prediction and DCT modes for interlace •  Quantization with greater range and adaptivity •  New intra-frame VLCs (variable length codes) •  New adaptive coefficient VLCs •  Scalability extensions; Spatial, SNR, Temporal •  System layer for multiple program transport streams •  Audio extended to 5-channel sound



HIGH 4:2:0 4:2:0. 4:2:2 1920 x 1152 1920 x 1152 80 Mb/s 100 Mb/s I, P, B I, P, B

HIGH-1440 4:2:0 4:2:0 4:2:0, 4:2:2 1440 x 1152 1440 x 1152 1440 x 1152 60 Mb/s 60 Mb/s 80 Mb/s I, P, B I, P, B I, P, B

MAIN 4:2:0 4:2:0 4:2:0 4:2:0, 4:2:2 760 x 576 720 x 576 720 x 576 720 x 576 15 Mb/s 15 Mb/s 15 Mb/s 20 Mb/s I, P I, P, B I, P, B I, P, B

LOW 4:2:0 4:2:0 352 x 288 352 x 288 4 Mb/s 4 Mb/s I, P, B I, P, B

LEVEL

PROFILE SIMPLE MAIN SNR SPATIAL HIGH

MPEG-2 Profiles and Levels



4:2:0 352 x 288

4 Mb/s I, P, B

HIGH

HIGH-1440

MAIN

LOW

LEVEL PROFILE

4:2:0 720 x 576 15 Mb/s

I, P

SIMPLE

4:2:0 1920 x 1152

80 Mb/s I, P, B

4:2:0 1440 x 1152

60 Mb/s I, P, B

4:2:0 720 x 576 15 Mb/s I, P, B

MAIN

4:2:0 720 x 576 15 Mb/s I, P, B

4:2:0 352 x 288

4 Mb/s I, P, B

SNR

4:2:0 1440 x 1152

60 Mb/s I, P, B

SPATIAL

4:2:0, 4:2:2 1920 x 1152

100 Mb/s I, P, B

4:2:0, 4:2:2 1440 x 1152

80 Mb/s I, P, B

4:2:0, 4:2:2 720 x 576 20 Mb/s I, P, B

HIGH

MPEG-2 4:2:2 Profile

4:2:2 720 x 608 50 Mb/s I, P, B

4:2:2 PROFILE

4:2:2 1920 x 1088 300 Mb/s I, P, B



•  Quality –  Better Chroma Resolution than MP@ML –  Higher quality (bit rate) than MP@ML –  Good multi-generation performance

•  Flexibility –  Short GOPs for editability –  Capability to pass all active video, some vertical info

•  Economy –  Storage costs –  Transmission costs –  Compatibility

MPEG-2 4:2:2 Profile for Production



How Good is MPEG-2 4:2:2?

•  The MPEG committee has conducted subjective assessment tests to verify the performance of the MPEG-2 4:2:2 profile.

•  Tests demonstrated that, with proper choices of data rate and GOP structure, MPEG-2 4:2:2 can meet professional requirements.



Achieving Quality

I IB IBBP 20 Mb/s

30 Mb/s

50 Mb/s

Bit

Rat

e

GOP Structure

Higher Quality

Lower Quality



Application Examples News and Acquisition 18 Mb/s IB GOP Structure

Archive 30 Mb/s IB GOP Structure

Post-Production 50 Mb/s

I only GOP Structure

Distribution 20 Mb/s

IBBP GOP Structure

Fully Compliant MPEG-2

4:2:2 Decoders



•  If you can’t hear it, don’t send it •  Psychoacoustic Models

–  Pre-Masking –  Post-Masking –  Simultaneous Masking

•  Data Structures

MPEG Audio



Temporal Masking

Post- masking

Pre- masking

time

sound pressure



Simultaneous Masking

20 kHz

Masking threshold

1 kHz sinewave

Threshold in quiet

20 Hz 1 kHz

sound pressure



Audio Frame input PCM samples 384 for Layer 1

3 * 384 = 1152 for Layer 2

MPEG Audio Encoder

Filterbank 32

Subbands

Scaler and

Quantizer

512 Point FFT

MUX

Masking Thresholds

Dynamic Bit and

Scale Factor Allocator

and Coder



Filterbank 32 Subbands

Audio Time Frame

32 Samples

32 Samples

32 Samples

32 Samples

12 Sections of 32 Samples

32 Samples = 0.66 msec (@ 48 kHz)

1 MPEG audio layer 1 frame = 8 msec of audio

1 MPEG audio layer 2 frame = 24 msec

12 x 32 Samples



Header

CR

C

Bit A

llocation

Scalefactors

Anc D

ata

20 Bit System

12 B

it Sync

Optional

4 bit linear

6 bit linear

Unspecified

Length

Subband Samples

GR0

0 1 2 31

GR1 GR2 GR11

Layer I Frame Structure

384 PCM Audio Input Samples Duration 8 msec @ 48 kHz



Seminar Topics






System encoder

PES Syntax

System decoder

PES Syntax

Elementary Streams

Elementary Streams

MPEG-2 Standards Not Standardized

Transport Stream

MPEG System

Audio encoder

Video encoder

Audio decoder

Video decoder



MPEG-2 System Mux

Audio Encoder

Video Encoder

Video Data

Audio Data

Packetizer

Packetizer Video PES

Audio PES

Program Stream (DVD)

Program Stream

MUX

Single Program Transport Stream

Transport Stream

MUX

Elementary Stream

Data



Transport Stream Formation



Multi-program Transport Stream



Transport Packets

•  The Transport Stream (TS) is a continuous data stream in 188 byte packets containing format (syntax) information and payload data

Header Payload

Packet Packet Packet Packet Packet Packet Packet Packet

188 Bytes



Transport Packet Header

Sync Byte

8

Transport Error

Indicator 1

Start Indicator

1

Transport Priority

1

PID

13

Scrambling Control

2

Adaptation Field

Control 2

Continuity Counter

4

Adaption Field Payload

Header Payload

188 Bytes

Adaptation Field

Length 8

Discontinuity Indicator

1

Random Access

Indicator 1

Elem Stream Priority

Indicator 1

5 Flags

5

Optional Fields

Stuffing Bytes

OPCR

48

Splice Countdown

8

Transport Private

Data

Adaption Field

Extension PCR

48

Minimum 4-byte header



Program Clock Model

variable delay = e(n) variable delay = d(n) constant trans delay = Ctrans

constant total delay = Ctotal

PCR clock phase

generator

PCR: encoder stamps

departure time of packet

PCR clock frequency generator

PCR: arrival time of packet

PES Syntax

System MUX

PES Syntax

Display

System DEMUX

PCR clock recovered

Audio Decoder

Video Decoder

Audio Encoder

Video Encoder



Reference Clock Synchronization

Elementary Stream

_ _ _ _

PCR = X PCR = X plus the time of exactly n bits n bits 188 byte packets

PCR

Load

Local PCR

Receiver 27 MHz clock

Video In

Transport Stream

Decoder Compare

Low Pass Filter

27 MHz Xtal VCO

27 MHz Clock

27 MHz Clock

Transport Stream

Formation Video

Encoder



Decoding the Transport Stream Program Specific Information (PSI)

•  Program Association Table (PAT) –  PID = 0, must be present in every transport stream

•  Program Map Table (PMT) –  PID values assigned by transmission system (DVB, ATSC, etc.)

•  Conditional Access Table (CAT) –  PID = 1

•  Network Information Table (NIT) –  PID values assigned by transmission system –  DVB considers this part of System Information (SI)

•  Null Packets –  PID = 8191 (1FFFhex = 13 “1”s binary)



PSI Example Program Association Table (PID 0)

Program Map Tables

Private Network

Data

Network Information Table

Transport Stream

Conditional Access Table (PID 1)

Prog 1 MAP

33 22 48 82 19 19 54 19 0

Program 0 16 Program 1 22 Program 3 33

... ...

Program k 55 ... ...

Stream 1 Video 54 Stream 2 Audio 48 Stream 3 Audio 49

... ... ...

Stream k Data 66 ... ... ...

Stream 1 Video 19 Stream 2 Audio 81 Stream 3 Audio 82

... ... ...

Stream k Data 88 ... ... ...

PAT Prog 3 MAP EMM Prog 1

Audio 1 Prog 3

Audio 2 Prog 3 Video 1

Prog 3 Video 1

Prog 1 Video 1

Prog 3 Audio 1

Prog 3 Video 1

1 81

Conditional Access Data



DVB System •  MPEG-2 Transport Streams

–  Service Information (SI) in addition to MPEG-2 (PSI)

•  SI includes –  NIT - Network Information –  SDT - Service Description –  EIT - Event Information –  TDT - Time and Date –  BAT - Bouquet Association –  RST - Running Status –  ST - Stuffing Tables

•  DVB Systems provide: –  Common Scrambling systems –  A common Conditional Access Interface –  Facilities for reverse channel operation



DVB Channel Coding

•  Provides error correction over the channel –  ( 1 in 10-4 on channel to 1 in 10-11 on Transport Stream )

•  Outer Coding - energy dispersal and RS •  Inner Coding - interleaving and viterbi

– Not used for cable transmission

Reed Solomon Coding 16 Bytes

204 Bytes per packet

Transport Stream Packet 188 Bytes









Seminar Topics






Test Function

Video Testing Layers Signal

Standardized Video Signals Video Quality

MPEG-2 Transport

Stream Protocol Analysis

INTRA-Facility Connections

INTER-Facility Connections SDH/ATM or

Modulated RF Transmission

Channel Analysis

Studio Connections

Program Compression

Transmission Channel

Formatting



Signal vs Picture Quality

•  Analog and digital video systems are linear –  Superposition applies – Results are time invariant and signal independent –  Test signals can be substituted for program material –  Testing in the vertical interval is equivalent to full-field tests –  Static test signals are sufficient (Indirect measurement)

•  Compression video systems are non-linear –  Test signals are easily/accurately compressed –  Picture quality is a function of; data rate, picture complexity and

encoding algorithm capabilities –  Test with complex motion sequences (Direct measurement)



Seminar Topics






Traditional Test Methods

A significant portion of the modern television system is analog or full bandwidth digital video

These signals will continue to be used in the foreseeable future

Input picture quality to the compression system must be maintained



Analog Video Testing

VIDEO Operational Monitoring

Technical Measurements

SYNCHRONIZING WAVEFORMS Technical Measurements

PHYSICAL LAYER (COAX) Technical Measurements

Functional Layers

Waveform Monitors

and Measurement

Sets

TDRs

RGB PAL NTSC

PAL NTSC

Composite Encoder

Transmission or

Operation

Record or

Display Camera



Analog Tests

Amplitude Rise/fall times Bandwidth Group delay Sig/Noise ratio Non-linearities Color gamut Diff Phase/Gain

No one test will do it all



Digital Video Testing

VIDEO SIGNAL CODING (Rec 601)

DIGITAL FORMATTING (Rec 656) DIGITAL WAVEFORM (Rec 656)

PHYSICAL LAYER (COAX/FIBER)

Functional Layers

Waveform Monitors, Measurement Sets

with Analog or Digital

Capabilities

TDRs, OTDRs

PAL NTSC

PAL NTSC

Rec 601/656 Rec 601/656

Composite Encoder

Transmission or

Operation

Record or

Display Camera

Studio Interconnect,

Operation

Decode A/D

Encode D/A

RGB



More Analog Tests

•  A/D and D/A Converters – Converters add distortions to the signal – Measure SNR on a shallow ramp – Differential gain/phase

–  Full ramp for “analog” measurements –  Shallow ramp for “digital” measurements

•  Ringing on digitally generated signals



Rec 601 Digital Measurements

•  Digital coding –  Levels, excluded values

•  Formatting –  Synchronization data, Embedded audio

•  Error Detection •  Waveform (eye pattern)

– Amplitude, risetime, overshoot –  Jitter by eye pattern or demodulation – Headroom (cable length)



Seminar Topics






Factors Affecting Video Quality in a Compression System

•  Quality of the input video – Amplitude, dc level, bandwidth, ringing, jitter – Noise, composite/component decoding artifacts –  Prefiltering to eliminate the above problems

•  Nature of the input video –  Picture spatial and temporal complexity



Factors Affecting Video Quality

•  Encoding parameters used –  Profile/level, field/frame, output data rate, GOP

•  Encoding algorithm –  Speed required, hardware vs software – Multiple pass (iteration of parameters) – Algorithm design

– Quantizing table selection – Use of motion vectors, search range



Compression Impairments Blocking: appearance of underlying block structure Error Blocks: a form of block distortion

–  One or more blocks bear no resemblance to the current or previous scene and often contrast greatly with the adjacent blocks

Edge busyness: distortion concentrated at edges of objects –  Characterized by temporal and spatial features

Mosquito noise: edge busyness associated with movement –  Characterized by moving artifacts or blotchy noise patterns superimposed

over the objects Quantization noise: snow or salt & pepper

–  Similar to random noise but not uniform over the image Blurring: distortion of the entire image,

–  Characterized by reduced sharpness of edges and spatial details Jerkiness: smooth, continuous motion now perceived as a series of distinct images



Blocking Blurring



Picture Quality Measurements

•  We can not actually measure “picture quality” – We can measure picture degradation comparison to reference

•  Subjective measurements –  ITU-R BT.500 has been updated –  Further work to extend subjective methods

•  Objective measurements are most useful if they have good correlation with subjective results



Subjective Tests (Human Viewer Trials)

Strengths •  Produces valid results in

conventional and digital television system applications

•  Provides a scalar result –  Mean Objective Score

•  Works well over a wide range of video (and still image) applications

Weaknesses •  Requires meticulous setup and

control •  Needs lots of participants •  Is time-consuming

Subjective tests are only applicable for development purposes. They do not lend themselves to operational monitoring,

production line testing or troubleshooting.



Quest for Objective Measurements

•  Intuition has led many developers to the same starting point:

–  Picture quality is related to the differences between the original and impaired scenes.

– A measurement of the magnitude of these differences is somehow related to “picture quality”

–  Therefore, construct a device which indicates the magnitude of these differences.

•  There are two approaches to objective measurements –  Feature Extraction –  Picture Differencing



Feature Extraction

Low Bandwidth Data

Results

Reference Picture

Degraded Picture Processing System

Feature Extraction

Feature Extraction

Feature Difference



Picture Differencing

Picture Data

Results

Picture Data

Reference Picture

Degraded Picture Processing System

Image Processing

Image Processing

Picture Data

Difference



Mean Squared Error

•  Prediction of quality derived from the result of computing the mean of the squares of the differences

– As the result approaches zero, the more identical are the original and copy.

– Conversely, as the result grows, the more different is the copy from the original.

•  Peak Signal to Noise Ratio is a variation of MSE:

PSNR=10 log10 MSE2 2552



But MSE (and PSNR) is Easily Fooled!

MSE = 27.10 MSE = 21.26



Objective Picture Measurements

•  ANSI T1.801.03 –  Peak Sig/noise Average gain Offset level –  Spatial shift Spatial info Temporal info – Added/lost; spatial frequencies, motion/edge energy – Radial average of spatial frequencies

•  Insufficient for comparison of systems •  Useful for yesterday/today comparisons

– Unfortunately most systems are not constant – Bit rate changes – Concatenation of different coding systems



Human Vision System

•  A number of picture assessment methods have been proposed based on the human vision model

•  A robust metric of image quality –  Independent of nature of the video material –  Independent of the type of impairments –  Independent of the compression system

•  Principles of HVS models – Contrast sensitivity –  Spatio-temporal response – Color perception



The Tektronix/Sarnoff Method

•  In a review of 32 human visual models –  “All but one of the models can be regarded as simplified

versions of the Lubin model.”

•  Dr. Lubin at Sarnoff Labs –  Extended the work to cover picture quality –  Temporal chroma models have been added

•  “JND Image Quality Metric” –  Tektronix/Sarnoff cooperative product development



Just Noticeable Differences

•  Central features – Approximates the optics of the eye and retinal structure –  Filters that decompose image into subbands – Directional filters –  Psycho-visually valid calibration curves –  Pooling process to combine local results

•  JND scale – At a value of 1 JND, 3 out of 4 can detect a difference –  Values above 1 have more noticeable differences –  Values below 1 have less noticeable differences



JND Algorithm

...

contrast pyramid

impaired image

transducer

JND map

oriented responses

gain control

JND value

sampling

optics

Identical Process

...

reference image

distance



Reference Image JND Map

Model Output: JND Maps

•  Local magnitude of JND Map indicates probability of seeing a difference between two images at that point

•  Average of JND Map indicates overall magnitude of visible differences

Degraded Image



JND vs. MSE

MSE = 27.10 Average JNDs = 0.75

MSE = 21.26 Average JNDs = 2.52



Objective Measurement Operation

•  Close matching of “presentation” – Chroma/luma gain and dc level –  Spatial and temporal alignment

•  Test motion sequences – Difficult but not killer –  Several different program types

•  Program material not okay (for now) – Compute power/time for matching – Material often not difficult



Basic Concept

Transport Encoder Decoder

Test Sequence Source

Picture Quality Meter Reference

Sequence Source

Play test sequences 5+ seconds

Perform automatic measurements

2 seconds



Use of Calibration Stripes

Special markings used to derive gain/level/shift.

Original

Transported

Compare original and transported



Seminar Topics






MPEG Protocol Analysis

MPEG Transport Stream

Hardware I/O Interface

Real-time Firmware

Display key errors

Hard-disk Storage

Generation and Analysis Software

Display header and Timing Information

•  Tektronix MTS 100 provides creation, generation and in-depth analysis of MPEG-2 transport streams

•  Real-time analysis and display of key errors is planned



MTS Analysis •  Analyzes transport and/or PES

packets –  Syntax (structural)

•  Extracts –  PES packets –  Elementary streams

•  Hierarchical view –  PSI (program specific

information) –  Programs –  Channels

MPEG TS

MPEG TS Data Files on the NT

Disk

Multiple Hardware Output Types

Windows NT Application

Control


MPEG Transport Stream

Analysis Program

MTS Data Files in the Data

Store System



MTS Generator •  ES Data Files are supplied by

Tektronix or the user •  The ES Data Files can

be on disk or CD ROM •  MTS Data Files are built using

multiplexing software –  Formatted ES –  User defined configuration –  Application of MPEG rules

•  MTS outputs the transport stream to the device under test

• 

MPEG TS

ES Data Files on the NT

Disk

Multiple Hardware Output Types


Control


Control MTS Formatter

and Multiplexer

MTS Data Files in the Data

Store System

CD ROM

User Defined

Configure File



Transport Stream Hierarchical Display



Transport Packet Field Description



Transport Packet Data Analysis



Transport Packet Hex Display



Program Association Table Analysis



0.804640 0.985496 1.166352

PCR number 21 - 30

PCR Base : 79659 units PCR Extension : 93 units PCR Value (PCRV) : 0.885103444 sec Interpolated PCR (PCRI) : 0.885104096 sec PCRI - PCRV : 0.000000652 sec Previous PCR Arrived Since : 0.040156800 sec Packet Number : 5886 Packet Number of PID 50 : 2279

PCR number 23

PCR Data Analysis



Time Stamp Analysis



System Target Decoder

•  An abstract model for deciding legality of streams and decoders (This is really control on encoder operation.)

•  Based on passing of streams through cascading buffers •  Overflow of any buffer implies illegality of stream •  Sometimes underflow is allowed; sometimes it isn’t.



System Target Decoder Simulation



Automatic Analysis

•  Selectable functions •  Error tables available for

quick access to data



Service Information

•  Selection of SI data



•  Decoded Service Description Table

Service Information Interpreted



Sarnoff Compliance Bitstreams STRM100

•  A visual inspection of Video Decoder Operation – No digital interface or capture equipment required –  Ease of use: pass-fail testing, rapid fault isolation –  Look for gray embossed “VERIFY” without artifacts

•  One parameter or technique at a time •  Loopable



Tests without B-pictures

Tests with B-pictures

Sarnoff Compliance Bitstreams: How They Work

IPPP .... PP PP .... P PP .... PP “Verify” (60 to 90 frames) -- does not change image

Test section (1 or more frame)

Start Title (operator convenience only)

IPPP .... PP PBBBB .... BBBBP Test and Verify (60 to 90 frames) -- the test runs continuously, each picture should be “verify”

Start Title (operator convenience only)



Sarnoff Compliance Bitstream Tests •  Different picture types •  Motion vector range •  DC differential range •  Macroblock pattern range •  Macroblock type range •  AC run/level test •  Macroblock address range •  Dynamic GOP structure •  Dynamic slice size •  Macroblock & zero stuffing •  Postprocessing

–  Zone plate –  Color bars –  Ramp

•  Downloadable quant matrices •  Different bitrates •  Dynamic picture size •  2:3 pulldown insertion •  VBV buffer sizes •  Mixed MPEG-1 & MPEG-2 •  Alternate scan pattern •  Alternate AC run/level VLC •  Frame/field coding modes •  Frame/field prediction modes •  Non-linear quantization scale •  Pan and Scan



Summary

•  Modern television systems are much more than “DTV” – Cameras and displays will continue to be analog –  Program production uses analog and full bandwidth digital

•  MPEG-2 will be the dominant compression method –  Entertainment quality video to the home

–  Satellite, Terrestrial, Cable – Digital Versatile Disk (DVD) –  Video servers – Contribution quality, reusable video for studios



Summary

•  MPEG only specifies bitstream syntax and decoding – A non-symmetrical system

– Complex encoders, Simple (inexpensive) decoders –  Encoding can improve and be compatible with today’s decoders

–  Increasing compute power for complex algorithms – Manufacturers can compete with proprietary algorithms

•  4:2:2 profile provides contribution quality – Decoders must be compliant for flexibility of application

•  MPEG-2 System for multi-program transmission



Summary

•  System testing requires a layered approach –  Signal/Picture quality – MPEG-2 system protocol –  Transmission channel

•  Signal quality testing uses traditional methods –  Indirect measurement, Static test signals, vertical interval

•  Picture quality testing is a complete new paradigm – Dynamic, complex test scenes utilize the complete channel – Reliable objective measurements use human visual model –  Picture differencing methods provide best results



Summary

•  MPEG-2 system protocol testing –  Generation of valid (and invalid) test signals is important –  In-depth testing provides complete data analysis off-line –  Real-time testing provides limited analysis of all packets

•  Tektronix equipment for video and protocol testing –  TSG422 Digital Video Test Signal Generator –  SPG422 Digital Video Sync Pulse Generator –  TG2000 Multi-format Test Signal Generator –  1700 series Composite/Component Waveform Monitors –  WFM600 Series Digital Video Monitors –  VM700T Video Signal Measurement Set –  MTS100 MPEG Transport Stream Analyzer/Generator



Thank You for Your Attendance

. . . providing 50 years of technical innovation and leadership.



References