an algorithm for constant- quality compressed video michael f. ringenburg richard e. ladner eve a....

An Algorithm For Constant-An Algorithm For Constant-Quality Compressed VideoQuality Compressed Video

Michael F. Ringenburg Michael F. Ringenburg

Richard E. LadnerRichard E. Ladner

Eve A. RiskinEve A. Riskin

UW CSE Industrial Affiliates MeetingUW CSE Industrial Affiliates MeetingFebruary 26, 2004February 26, 2004

The Quick Introduction to The Quick Introduction to Video CompressionVideo Compression

4 Steps:4 Steps: Motion compensation.Motion compensation.

Outputs motion vectors, which predict the frame Outputs motion vectors, which predict the frame from the previous frame.from the previous frame.

Actual - Predicted = ResidualActual - Predicted = Residual Transform the residual.Transform the residual.

Concentrates the energy (information).Concentrates the energy (information). Lossy encoding of the transformed residual.Lossy encoding of the transformed residual. Decoding.Decoding.

Generates an approximation of the original frame.Generates an approximation of the original frame.

Motion CompensationMotion Compensation

T=1 (reference) T=2 (current)

Thanks to Gidon Shavit

Residual Examples (Boosted)Residual Examples (Boosted)

Original Residual


Transform Example (DCT)Transform Example (DCT)




Outputs motion vectors, which predict the frame from the Outputs motion vectors, which predict the frame from the previous frame.previous frame.

Actual - Predicted = ResidualActual - Predicted = Residual

Transform the residual.Transform the residual. Concentrates the energy (information).Concentrates the energy (information).

Lossy encoding of the transformed residual.Lossy encoding of the transformed residual. Quantization vs. Embedded CodingQuantization vs. Embedded Coding

Decoding.Decoding. Generates an approximation of the original frame.Generates an approximation of the original frame.

Embedded Bit Plane CodingEmbedded Bit Plane Coding

The n-th bit plane consists of the n-th bit of The n-th bit plane consists of the n-th bit of each coefficient. each coefficient.

Each bit plane is compressed with an Each bit plane is compressed with an embedded, lossless code. embedded, lossless code.

Send the bit planes in order of Send the bit planes in order of significance.significance. The most important information is sent first.The most important information is sent first.

Can terminate the stream between or Can terminate the stream between or within bit planes.within bit planes.

The GTV CoderThe GTV Coder

An embedded bit plane video coder developed An embedded bit plane video coder developed at the University of Washington.at the University of Washington. Written by Jeff West, Gidon Shavit and Michael Written by Jeff West, Gidon Shavit and Michael

Ringenburg, with Richard Ladner and Eve Riskin.Ringenburg, with Richard Ladner and Eve Riskin. Motion Compensation on 16 x 16 blocksMotion Compensation on 16 x 16 blocks 8 x 8 Discrete Cosine Transform8 x 8 Discrete Cosine Transform Lossy coding based on the GT-DCT image Lossy coding based on the GT-DCT image

coder developed by Ed Hong with Richard coder developed by Ed Hong with Richard Ladner and Eve RiskinLadner and Eve Riskin

All results in this talk use the GTV coderAll results in this talk use the GTV coder

Interframe Bit AllocationInterframe Bit Allocation

Given : An embedded coder, a maximum Given : An embedded coder, a maximum bit budget bit budget BB and a video with and a video with FF frames. frames.

How many bits How many bits bbii should we allocate to should we allocate to

each frame each frame ffii in order to achieve constant in order to achieve constant

quality video? quality video? Formally, our constraint is:Formally, our constraint is:

Why Is This Important?Why Is This Important?

Some frames require more bits (are Some frames require more bits (are harder) to encode, because …harder) to encode, because … Scene changes prevent good motion Scene changes prevent good motion

compensation.compensation. As do lighting changes.As do lighting changes. And, to a lesser extent, dramatic actions.And, to a lesser extent, dramatic actions.

For example, set your TiVo to it’s lowest For example, set your TiVo to it’s lowest quality/highest compression setting. Watch quality/highest compression setting. Watch the scene change blockiness at playbackthe scene change blockiness at playback

Why Is This Difficult?Why Is This Difficult?

Motion Compensation!!!Motion Compensation!!! The quality of frame The quality of frame nn is a function of the allocation to is a function of the allocation to

frame frame n,n, and the quality of frame and the quality of frame (n-1(n-1).). Quality of frame Quality of frame (n-1)(n-1) is a function of the allocation to is a function of the allocation to

frame frame (n-1(n-1) and the quality of frame ) and the quality of frame (n-2(n-2). Et cetera.). Et cetera. Thus, the quality of frame Thus, the quality of frame nn is a function of is a function of nn

variables.variables. The quality function (a.k.a. rate-distortion curve) The quality function (a.k.a. rate-distortion curve)

is different for every frame.is different for every frame. The rate-distortion curves are The rate-distortion curves are not not monotonic.monotonic.

Approximately monotonic, though.Approximately monotonic, though.

MultiStage AlgorithmMultiStage Algorithm

A global allocation algorithm (allocates A global allocation algorithm (allocates across the entire video).across the entire video).

Alternates between two stages which Alternates between two stages which provide feedback to each other.provide feedback to each other. The Constant Quality StageThe Constant Quality Stage

The Target Allocation StageThe Target Allocation Stage After every stage, check if we have After every stage, check if we have

constant quality and the target allocation.constant quality and the target allocation.

Example, Step 1Example, Step 1Constant QualityConstant Quality

0

100

200

300

400

500

600

700

800

900

1000

1 2 3 4

Frame

Bytes

20

25

30

35

40

PSNR

Bytes

PSNR

T=2000

Example, Step 2Example, Step 2Target AllocationTarget Allocation

0

100

200

300

400

500

600

700

800

900

1000

1 2 3 4

Frame

Bytes

20

25

30

35

40

PSNR

Bytes

PSNR

T=1000

Example, Step 3Example, Step 3Constant QualityConstant Quality

0

100

200

300

400

500

600

700

800

900

1000

1 2 3 4

Frame

Bytes

20

25

30

35

40

PSNR

Bytes

PSNR

T=800

Example, Step 4Example, Step 4Target AllocationTarget Allocation

0

100

200

300

400

500

600

700

800

900

1000

1 2 3 4

Frame

Bytes

20

25

30

35

40

PSNR

Bytes

PSNR

T=1000

Method ComparisonMethod Comparison

ConclusionsConclusions

Developed two new global interframe bit Developed two new global interframe bit allocation algorithms for embedded video codersallocation algorithms for embedded video coders MultiStage algorithmMultiStage algorithm Ratio algorithm (not presented here)Ratio algorithm (not presented here)

MultiStage has the lowest variance, and the MultiStage has the lowest variance, and the highest minimum PSNR. Nearly constant quality.highest minimum PSNR. Nearly constant quality.

Full paper at:Full paper at: http://www.cs.washington.edu/homes/miker/http://www.cs.washington.edu/homes/miker/

RingenburgAllocation.pdfRingenburgAllocation.pdf

an algorithm for constant- quality compressed video michael f. ringenburg richard e. ladner eve a....

Documents

original frame

quality of frame n

previous frame

frame fi

good motion compensation

outputs motion vectors

motion compensationt

transformed residual