cs 414 - spring 2012 cs 414 – multimedia systems design lecture 12 – mpeg-2/ mpeg-4 (part 6)...
Post on 27-Dec-2015
216 Views
Preview:
TRANSCRIPT
CS 414 - Spring 2012
CS 414 – Multimedia Systems Design Lecture 12 – MPEG-2/MPEG-4 (Part 6)
Klara Nahrstedt
Spring 2012
CS 414 - Spring 2012
Administrative MP1
Outline MPEG-2/MPEG-4 Reading:
Media Coding book, Section 7.7.2 – 7.7.5 http://www.itu.int/ITU-D/tech/digital-broadcasting/kiev/References/mpe
g-4.html http://en.wikipedia.org/wiki/H.264
Available software Xvid – free software http://www.bing.com/videos/watch/video/xvid-free-download-mpeg-4-vi
deo-codec-for-pc
CS 414 - Spring 2012
MPEG-2 Standard Extension MPEG-1 was optimized for CD-ROM and
apps for 1.5 Mbps (video strictly non-interleaved)
MPEG-2 adds to MPEG-1: More aspect ratios: 4:2:2, 4:4:4Progressive and interlaced frame codingFour scalable modes
spatial scalability, data partitioning, SNR scalability, temporal scalability
CS 414 - Spring 2012
MPEG-1/MPEG-2
Pixel-based representations of content takes place at the encoder
Lack support for content manipulatione.g., remove a date stamp from a video turn off “current score” visual in a live game
Need support manipulation and interaction if the video is aware of its own content
CS 414 - Spring 2012
MPEG-4 Compression
CS 414 - Spring 2012
Interact With Visual Content
CS 414 - Spring 2012
Original MPEG-4
Conceived in 1992 to address very low bit rate audio and video (64 Kbps)
Required quantum leaps in compressionbeyond statistical- and DCT-based techniquescommittee felt it was possible within 5 years
Quantum leap did not happen
CS 414 - Spring 2012
The “New” MPEG-4 Support object-based features for content
Enable dynamic rendering of contentdefer composition until decoding
Support convergence among digital video, synthetic environments, and the Internet
CS 414 - Spring 2012
MPEG-4 Components Systems – defines architecture, multiplexing structure, syntax Video – defines video coding algorithms for animation of synthetic and
natural hybrid video (Synthetic/Natural Hybrid Coding) Audio – defines audio/speech coding, Synthetic/Natural Hybrid Coding
such as MIDI and text-to-speech synthesis integration Conformance Testing – defines compliance requirements for MPEG-4
bitstream and decoders Technical report DSM-CC Multimedia Integration Framework – defines Digital Storage
Media – Command&Control Multimedia Integration Format; specifies merging of broadcast, interactive and conversational multimedia for set-top-boxes and mobile stations
CS 414 - Spring 2012
MPEG-4 Example
ISO N3536 MPEG4 CS 414 - Spring 2012
MPEG-4 Example
ISO N3536 MPEG4 CS 414 - Spring 2012
MPEG-4 Example
Daras, P. MPEG-4 Authoring Tool, J. Applied Signal Processing, 9, 1-18, 2003
CS 414 - Spring 2012
Interactive Drama
http://www.interactivestory.netCS 414 - Spring 2012
MPEG-4 Characteristics and Applications
CS 414 - Spring 2012
Example of MPEG-4 Scene
Source: http://mpeg.chiariglione.org/standards/mpeg-4/mpeg-4.htm
Media Objects
An object is called a media objectreal and synthetic images; analog and
synthetic audio; animated faces; interaction
Compose media objects into a hierarchical representation form compound, dynamic scenes
CS 414 - Spring 2012
Logical Structure of Scene
CS 414 - Spring 2012
Example of Composed Scene
CS 414 - Spring 2012
Sprite ObjectPlayer Object
Composed Scene
Video Syntax Structure
CS 414 - Spring 2012
New MPEG-4 Aspect: Object-based layered syntactic structure
Content-based Interactivity Achieves different qualities for different objects
with a fine granularity in spatial resolution, temporal resolution and decoding complexity
Needs coding of video objects with arbitrary shapes
Scalability Spatial and temporal scalability Need more than one layer of information (base and
enhancement layers)
CS 414 - Spring 2012
Examples of Base and Enhancement Layers
CS 414 - Spring 2012
Coding of Objects Each VOP corresponds to an entity that after
being coded is added to the bit stream Encoder sends together with VOP
Composition information where and when each VOP is to be displayed
Users are allowed to change the composition of the entire scene displayed by interacting with the composition information
CS 414 - Spring 2012
Spatial Scalability
CS 414 - Spring 2012
VOP which is temporally coincident with I-VOP in the base layer, is encoded as P-VOP in the enhancement layer. VOP which is temporally coincident with P-VOP in the base layer is encoded as B-VOP in the enhancement layer.
Examples of Base and Enhancement Layers
CS 414 - Spring 2012
Temporal Scalability
CS 414 - Spring 2012
High Level Codec for Generalized Scalability
CS 414 - Spring 2012
Composition (cont.)
Encode objects in separate channelsencode using most efficient mechanism transmit each object in a separate stream
Composition takes place at the decoder,rather than at the encoder requires a binary scene description (BIFS)
BIFS is low-level language for describing:hierarchical, spatial, and temporal relations
CS 414 - Spring 2012
MPEG-4 Part 11 – Scene Description BIFS – Binary Format for Scenes
Coded representation of the spatio-temporal positioning of audio-visual objects as well as their behavior in response to interactions
Coded representation of synthetic 2D and 3D objects that can be manifested audibly and/or visibly
BIFS – MPEG-4 scene description protocol to Compose MPEG-4 objects Describe interaction about MPEG-4 objects Animate MPEG-4 objects
BIFS Language Framework – XMT (textual representation of multimedia content using XML) Accommodates SMIL, W3C scalable vector graphics and VRML (now
X3D)
CS 414 - Spring 2012
MPEG-4 Rendering
ISO N3536 MPEG4 CS 414 - Spring 2012
Major Components on MPEG-4 Terminal
http://mpeg.chiariglione.org/standards/mpeg-4/mpeg-4.htmCS 414 - Spring 2012
Integration and Synchronization of Multiple Streams
CS 414 - Spring 2012
Source: http://mpeg.chiariglione.org/technologies/mpeg-4/mp04-bifs/index.htm
Synchronization
Global timeline (high-resolution units)e.g., 600 units/sec
Each continuous track specifies relatione.g., if a video is 30 fps, then a frame should
be displayed every 33 ms.
Others specify start/end time
CS 414 - Spring 2012
Data Recovery
CS 414 - Spring 2012
• Reversible Variable Length Codes (RVLC) : variable length code words are designed such that they can be read both in forward and reverse directions
QP – quantization parameterHEC – header extension codeMP – macro-block
MPEG-4 parts MPEG-4 part 2
Includes Advanced Simple Profile, used by codecs such as Quicktime 6
MPEG-4 part 10 MPEG-4 AVC/H.264 also called Advanced
Video Coding Used by coders such as Quicktime 7Used by high-definition video media like Blu-
ray DiscCS 414 - Spring 2012
MPEG-4 System Layer Model
http://mpeg.chiariglione.org/standards/mpeg-4/mpeg-4.htm
SL – sync layerDMIF – Delivery Multimedia Integration Framework
MPEG-4 File with Media Data
http://mpeg.chiariglione.org/standards/mpeg-4/mpeg-4.htm
OD – Object DescriptorBIFS – Binary Format for ScenesMdat – media data atoms
Interaction as Objects
Change colors of objects Toggle visibility of objects Navigate to different content sections Select from multiple camera views
change current camera angle Standardizes content and interaction
e.g., broadcast HDTV and stored DVD
CS 414 - Spring 2012
Hierarchical Model
Each MPEG-4 movie composed of trackseach track composed of media elements (one
reserved for BIFS information)each media element is an objecteach object is a audio, video, sprite, etc.
Each object specifies its:spatial information relative to a parent temporal information relative to global timeline
CS 414 - Spring 2012
Conclusion A lot of MPEG-4 examples with interactive capabilities Content-based Interactivity
Scalability Sprite Coding
Improved Compression Efficiency (Improved Quantization)
Universal Accessibility re-synchronization data recovery error concealment
CS 414 - Spring 2012
top related