technion - iit dept. of electrical engineering signal and image processing lab transrating and...
TRANSCRIPT
Technion - IITDept. of Electrical Engineering
Signal and Image Processing lab
Transrating and Transcoding of Coded
Video Signals
David Malah
Ran Bar-Sella Michael Lavrentiev
Research Proposal to
STRIMM Meeting June 25, 2003
TranscoderCoded
Bit StreamTranscodedBit Stream
Converting a coded bit stream to another bitstream to match a destination profile in terms of:
– Media Formats {e.g., MPEG-2 H.264}
– Resolutions (Spatial, Temporal)
– Bit Rates (Transrating)
– Color depths / formats {e.g., 4:4:4 4:2:0}
• • •
Transcoding
Cell-phone
Generic Hybrid Video Encoder
..Image sequence
0101...
Bit stream+
DCT Q VLC
Q-1
DCT -1
+
MEMMC
-
+
Intra / Interswitch
++
ME
MC -Motion Compensation
ME -Motion Estimation
MEM -Frame store
DCT -Discrete Cosine Transform
Q -Quantization
VLC -Variable Length Code
MPEG Frame Structure
H.264
Intra Spatial Prediction
• Frame dropping (B, P)
• Color suppression
• Discarding high-frequency DCT coefficients
• Reducing spatial resolution (size reduction)
• DCT coefficients Re-quantization
Transrating Methods (Bit-Rate Reduction - BRR)
Cascading Decoder and Encoder
FullDecoder
FullEncoder
CodedBit Stream
TranscodedBit Stream
Transcoder Architectures
Related Issues• Open Loop • Closed Loop - Pixel-Domain - DCT-Domain
PartialEncoder
Complexity-Reduced Transcoding(Compressed Domain)
PartialDecoder
Processing
Motion vectors / coding modes
CodedBit Stream
TranscodedBit Stream
Re-quantization Error
Q1
Q2 Original pixel value
Quantized values with Q1Quantized values with Q2
q2/q1
q2/q1
q2/q1
q2/q1
Re-quantization Step-size Selection(Wang and Woods, 1998)
1q
2q
Preferred Step-size Ratios(Wang and Woods, 1998)
MSE Re-quantization
Finer quantizer
p(x)
x
d1 d2
Coarser quantizerR1 R2
D
c
c
Centroid
Q1
Q2
MAP Re-quantization
p(x)
xp1 p2
d1 d2
p2 < p1 p2 > p1
Finer quantizer
R1 R2
DCoarser quantizer
Q1
Q2
Constrained Minimization problem:
Lagrangian cost function becomes sum of independent MB level calculated parts:
( ) min{ ( ) ( )}k
k k k k kq
J d q r q
Lagrangian parameter, λ≥0, is iteratively updated to achieve desired bit-rate RT
Re-quantization by Lagrangian Optimization
(Assuncao and Ghanbari, 1997)
• Recent result (Laventer and Porat – ICIP-03): Optimal quantizer steps are obtained at values close to specific multiples of the input quant. step.
Variable Length Coding
Macro-Block
Trellis Re-quantization
q*kmax
q*k2
block_1
q*k1
block_2 block_N
HVS-based Bit Allocation
• Different regions have different perceptual importance and may be encoded at different rates for similar subjective quality.
• Video frames can be partitioned (segmented) into regions having different characteristics:
- Textured regions- Smooth regions- Edges- Moving objects
• For efficiency, segmentation should be based on compressed- domain data: Block DCT Coeff., MB type, MB q-step, MB rate, and Motion Vectors (MV).
• Possible approach: Block classification, Region-growing, and Segment tracking.
• Bit allocation to each region should reflect perceptual importance.
MPEG-2 H.264 Transcoding
• MEC
- 7 modes (block sizes 16x16 – 4x4) - ¼ Pel resolution- Improved MV coding
• Transform- Integer transform (4x4)
• Intra Prediction
- spatial prediction (9 modes)- 16x16 or 4x4 blocks
• Inter Prediction
- Multiple reference frames (1 – 5)- SP frame for stream transition
H.264 – Main Differences vis-à-vis MPEG-2
• Quantization and Coeff. Scanning- 32 logarithmic steps- no ‘dead-zone’- 2 scan modes
• De-blocking Filter (4x4)
• Entropy Coding - Universal VLC (UVLC) - Arithmetic Coding (CABAC)
• Network Friendly Representation
- Video Coding Layer - Network Adaptation Layer
(packetization, priority control)
Proposed Activity
Re-quantization• Restricted Lagrangian optimization with MSE / MAP• Trellis re-quantization (MSE optional)
• Bit-rate Control
• Integrating other rate-reduction techniques (temporal and spatial resolution reduction – BGU)
I. Transrating (MPEG2 MPEG2)
HVS-based bit-rate reduction• Frame segmentation and tracking
• HVS-based bit allocation to segments
• Restricted Lagrangian optimization in each segment• Trellis re-quantization in each segment
Proposed Activity (Cont’d)
II. Transcoding (MPEG-2 H.264)
Base-line Transcoder
• Maximum matching to MPEG-2
- MEC – MB based; ½ pixel res.- B&P frames, Single ref. frame- No Intra spatial prediction- Same MB types decisions
• Adapted H.264 features:
- Syntax
- UVLC
- Transform
- Quantization
• Reduced rate- Adapting developed re-quantization techniques from MPEG-2- Integrating other rate- reduction techniques (temporal and spatial resolution reduction – BGU)
Full Transcoder
• Similar rate- Intra prediction- All motion modes- Multiple ref. Frames- Coding control decisions
END