mpeg 3d graphics compression offer
DESCRIPTION
I made this presentation in order to convince Web3D and Khronos folks that community and industry need one single standard for 3D graphics compression. It contains a list of MPEG-4 tools for graphics compression that are royalty free and for which open source software implementation exist. A JavaScript implementation of decoder and WebGL complaient MPEG-4 player is also introduced.TRANSCRIPT
MPEG offer for 3D Graphics Compression
Web3D Conference, June 2013@San Sebastien
Marius PREDA
Institut MINES Telecom, France
MPEG 3DG Chairman
Hard problem Data heterogeneity
Scene Geometry Attributes Animation
Different Representation Methods Different Application Requirements
3D Graphics Compression
A good compression method is agnostic to the signal but exploits the signal nature
3DG Comp: N dimensional problemImage and Video is easy, only color
MPEG 3DG Technologies
MPEG-1/2(AV content)
1992/4
VRML
1997
• Part 11 - BIFS: -Binarisation of VRML -Extensions for streaming -Extensions for server command -Extensions for 2D graphics - Real time augmentation with audio & video• Part 2 - Visual: - 3D Mesh compression - Face animation
1998
• Part 2 – Visual - Body animation
1999
MPEG-4 v.1
MPEG-4 v.2
First form of broadcast signal augmentation
MPEG 3DG Technologies
MPEG-4
2003
•AFX 2nd Edition: - Animation by morphing - Multi-texturing
2005
• AFX 3rd Edition - WSS for terrain and cities - Frame based animation
2007
MPEG-4
MPEG-4
• Part 16 - AFX: - A rich set of 3D graphics tools - Compression of geometry, appearance, animation
• AFX 4th Edition - Scalable complexity mesh coding
2011
MPEG-4A rich set of 3D Graphics representation and compression tools
MPEG 3DG Technologies
MPEG-4
2003
•AFX 2nd Edition: - Animation by morphing - Multi-texturing
2005
• AFX 3rd Edition - WSS for terrain and cities - Frame based animation
2007
MPEG-4
MPEG-4
• Part 16 - AFX: - A rich set of 3D graphics tools - Compression of geometry, appearance, animation
• AFX 4th Edition - Scalable complexity mesh coding
2011
MPEG-4
2009
• Part 25 - Compression of third-party XML (X3D, COLLADA)
MPEG-4
2004
• Part 16 - X3D Interactive Profile
MPEG-4
Today’s Topic : Royalty Free
Static Geometry Compression: TFAN
Animation and Animated Geometry Compression: BBA and FAMC
Web 3D Graphics Coding
Compression Benchmark Platform
Focus on 3D Object Compression
Decompose a triangle mesh into a set of triangle fansEncoder each triangle fan with a special code depending on the configuration of its neighborhood Arithmetically encode the TFAN codes
TFans: 1675, 62937, 2984
Low complexity: linear encoding/decoding complexity Handles directly (without stitching) arbitrary connectivity (manifold or not, oriented or not, with or without holes, any genus) Offers the possibility to preserve the Vertices Order (VO) and Triangles Order (TO)
Principle
Advantages
Triangle FAN
Connectivity encoding
Triangle FAN
From K. Mammou, Phd Thesis 2008
Bone-base Animation - BBA
Geometry Seamless mesh: shapes sharing the same vertices list
TextureImage Mapping on vertices sub-set
HierarchySkeleton layerMuscle layer
Animation Parameters1D controllers: bones and muscles
Courtesy of VRLab, EPFL
Courtesy of IMT
Widely supported by ATs
Able to build and animate any kind of skeleton
Courtesy of Televirtual
Realistic and high quality animation
Bone-base Animation - BBA
Frame #nUncompressed BBA
Binary file
Frame P
Prediction
Arithmetic coding
Quantization
Frame I
Arithmetic coding
Quantization
Segment #nUncompressed BBA
DCT
Binary file
Segment P
Prediction
DC Q
Segment I
Huffman coding
DC Q
DC Coeff. AC Coeff.
Huffman coding
AC Q
Bone-base Animation - BBA
Two compression methods: frame-based and DCT-based
5-30 kbps
Frame-based Mesh Compression - FAMC
What if no skeleton is defined, only animated meshes?
FAMC: cluster the vertices and encode cluster animationparameters plus corrections
Frame-based Mesh Compression - FAMC
LPCA : Linear PCAAWC : Anisotropic Wavelets Compression CPCA : Clustered PCATWC : Temporal Wavelet-based Compression
Compression benchmarking
more than 10 000 objects of various kinds distortion vs bitrate encoding and decoding time per object and global statistics
Compression benchmarking
API to integrate third party codecs
3D Studio Max
MPEG-4 File
Exp
ort
MP4toJS Export
MPEG Web3DG Format
JSONDescription
Texture(JPEG)
SC3DMC BBA
Web3DCoding Scripts
GraphicsCodec
WebGLEngine
BBA Decoder
JSON Parser
SC3DMC Decoder
glIndexedFaceSet
MPEG Web3DG
JSONDescription
Texture(JPEG)
SC3DMC
BBA
{"object" : { "name": "rabbit", "shapes":[ { "filename": "rabbit_0.s3d", "transform": [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0], "materials":[ { "texture": "rabbit_0_0.jpg" } ] ,"animation":{ "anm_filename": "rabbit_0.anm", "bba_streams": [ { "name": "rabbit_0_0.bba", "fileName": "rabbit_0_0.bba" }]}}]}}
Web 3D Graphics Coding
Web3DCoding Scripts
GraphicsCodec
WebGLEngine
BBA Decoder
JSON Parser
SC3DMC Decoder
glIndexedFaceSet
MPEG Web3DG
JSONDescription
Texture(JPEG)
SC3DMC
BBA
{"object" : { "name": "rabbit", "shapes":[ { "filename": "rabbit_0.s3d", "transform": [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0], "materials":[ { "texture": "rabbit_0_0.jpg" } ] ,"animation":{ "anm_filename": "rabbit_0.anm", "bba_streams": [ { "name": "rabbit_0_0.bba", "fileName": "rabbit_0_0.bba" }]}}]}}
AMD of MPEG-4 Part 16 JSON support for object graph light (textual) representation of bones hierarchy requirements for supporting GPU texture formats
Web 3D Graphics Coding
TFAN and BBA open source implementation 3DSMax to MPEG-4 (TFAN and BBA) open source exporter JS Implementation of TFAN and BBA decoders WebGL Implementation
Open Resources
www.MyMultimediaWorld.com : Software/OpenSource Tab
The outside world is waiting us with a UNIQUE compression solution for A 3D Object
Time to set the common base and work TOGETHER for a common profile
Many technologies are already available, time to do SELECTIONS
Conclusions