scalable video conferencing using subband transform coding and layered multicast transmission
DESCRIPTION
Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission. Mathias Johanson Swedish Research Institute for Information Technology [email protected]. Scalability in Videoconferencing. Large number of video receivers (and senders) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/1.jpg)
Scalable Video ConferencingUsing Subband Transform Coding
and Layered Multicast Transmission
Mathias JohansonSwedish Research Institute for Information Technology
![Page 2: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/2.jpg)
Scalability in Videoconferencing
• Large number of video receivers (and senders)
• Multiple quality levels in a single multipoint conference session
• Differentiated host and network requirements
• Realizable over public internetworks
![Page 3: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/3.jpg)
• CODEC operates at fixed bandwidth
• Multipoint operation involves gateways
• Differentiated quality levels in a multipoint session require transcoders that are expensive and introduce latency
• Often dependent on level 2 network protocols (e.g. ISDN systems)
Limitations of Traditional Videoconferencing Systems
![Page 4: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/4.jpg)
Approach...
• Scalable codec based on subband transform coding
• Receiver-driven layered IP-multicast transmission
• Software implementation + DSP-based implementation
![Page 5: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/5.jpg)
Layered Video Coding
• Temporal layering– Increased number of refinement layers correspond to
higher framerate
• Spatial layering– Increased number of refinement layers correspond to
higher image resolution
• Layered quantization– Increased number of refinement layers correspond to
finer quantization
![Page 6: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/6.jpg)
Temporal Layering
Channel 1
Channel 2
Channel 3
Channel 4
Transmission channels that can be received independently
Images of a video sequence
![Page 7: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/7.jpg)
Spatial Layering
Channel 1
Transform
Channel 3
Channel 2
Original imageBase signal + refinement signals
![Page 8: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/8.jpg)
Layered image and video encoding/compression formats
• Hierarchical JPEG
• MPEG-2 scalable mode– temporal, spatial, SNR scalability
• H.263 scalable mode
• Wavelets
Block-based DCT
Subband transform
![Page 9: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/9.jpg)
Base layer
Refinementlayer
Down-sample
x(t)Encode
Encode
Decode
Upsample
Spatial scalability in block based image and video encodings
![Page 10: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/10.jpg)
Wavelet-based approach to spatial scalability
Glow
x(t)
(t)y0
(t)y1
2
2Ghigh
base layer
refinement layer
Quadrature mirror filters implementing the wavelet transform
Encode
Encode
![Page 11: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/11.jpg)
Wavelet transform
Iterate….
horizontal transform vertical transform
Original image
Transformed image
![Page 12: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/12.jpg)
Wavelet compression
• Colorspace conversion and subsampling– RGB -> YCrCb 4:2:2
• Wavelet transform (separately on Y, Cr, Cb)– Subband decomposition
• Quantization of each subband/component– Lossy compression step
• Huffman encoding– entropy coding
![Page 13: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/13.jpg)
Communication Architecture
• Transmit the subbands of the transformed images on separate channels that can be received independently
• Multicasting
• Leaf-initiated JOIN-mechanism
RLMReceiver-driven Layered (IP) Multicast
![Page 14: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/14.jpg)
224.3.4.5
224.3.4.6
224.3.4.7
224.3.4.8
Refinement layers
Base layer
RInternet
Sender Receiver (4 layers)
Receiver (1 layer)
High bandwidth
Low bandwidthMulticast router
![Page 15: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/15.jpg)
Wavelet RTP header
FOLW
HQ1
Q2Q3
Fragmentation OffsetLayer NumberWidth
HeightY Quantization Factor
Cr Quantization FactorCb Quantization Factor
H1H2H3
L1L2
L3
Y Huffman Table SizeCr Huffman Table SizeCb Huffman Table Size
Y Data LengthCr Data Length
Cb Data Length
FO L W H Q1 Q2 Q3 H1 H2 H3 L1 L2 L3
0 32 64 96 128 160 192
![Page 16: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/16.jpg)
Prototype implementation
• Based on Smile!
• Software wavelet codec
• Receiver-driven layered IP multicast network module
• RTP/RTCP
• Spatial and temporal scalability
• SGI O2, MIPS R5000 processor
![Page 17: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/17.jpg)
Usage Scenario highly heterogeneous environment
RHigh-speed LAN
Internet
Dial-up access
Medium qualityLow quality
High quality
Leased Line
Leased Line
Transmitter
![Page 18: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/18.jpg)
![Page 19: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/19.jpg)
![Page 20: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/20.jpg)
![Page 21: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/21.jpg)
![Page 22: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/22.jpg)
![Page 23: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/23.jpg)
![Page 24: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/24.jpg)
Performance Tests
Image quality scalability Bandwidth scalability
![Page 25: Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813bec550346895da519e7/html5/thumbnails/25.jpg)
Future work...
• Temporal compression
• DSP implementation (TMS320C80 or similar)
• Automatic selective refinement based on ”bandwidth discovery”
• Subband audio coding