RESTful APIs and Delivery Formats for 3D ResourcesKristian Sons (DFKI) Rémi Arnauld (AMD)
Johannes Behr (Fraunhofer IGD)
The multimedia point of view
A.G. MalamosMultimedia Lab
Dept. of Informatics EngineeringTEI of Crete, Heraklion, Crete, Greece
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA
The multimedia dimension of graphics
Graphics and especially virtual worlds have a clear multimedia dimension in the sense of coexistence of multiple medias presented together in real time to provide an integrated audiovisual result.
Performance in that case is a convolutional result of several factors applied to graphics, video and image textures and sound. Such performance factors might be...LOD (resolution), Delay, Jitter and of course.... user expectations.
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA
Our experiments in real-time 3D delivery
1. HTTP vs Websockets (The two most popular approaches on application networking)
2. The MPEG DASH (Dynamic Adaptive Streaming over HTTP)
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA
HTTP vs Websockets
The 1st experiment
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA
HTTP vs Websockets (1/5)
What is websockets?
WebSocket is a protocol providing full-duplex communications over a single TCP connection. The WebSocket protocol was standardized by the IETF as RFC 6455 in 2011, and the WebSocket API is being standardized by the W3C.
Client Server
Handshake (upgrade HTTP)
Full-duplex Data Channel
Persistent Connection
Time
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA
HTTP vs Websockets (2/5)
TCP/IP
WEBSOCKETSSTOMP
Simple (or Streaming) Text Orientated Messaging Protocol
HTTP
The websockets testbed
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA
HTTP vs Websockets (3/5)The data-set
We have currently uploaded several models from tens of thousands to millions of point coordinates, including colour, faces and normals information
1. The dome of the church of St. Ioannis in the TEI area, in Crete2. Traditional Cretan musical instrument 3. Ancient Theater Demetriados4. An olive tree5. Egyptian Cat
from the The Classic Archaeology Collectionin the Museum of Archeology and History of Art of the University of Athens
6. an amphora7. a wheel-made plate8. Corinthian oinochoe 9. a wheel-made lekythos) 19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA
HTTP vs Websockets (4/5)Models are in X3D-X3DOM format
Three Web-based distribution platforms available to compare:
1. Direct display
2. progressive streaming over HTTP requests
3. Websocket-STOMP progressive transmission
Optional use of LZW compression into streams
Two progression modes:1. Serial (neighborhood) progression2. Area sort (biggest triangles first)
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA
HTTP vs Websockets (5/5)a. Serial progression b. Area sort progression
Webservices have better performance than HTTP requests
Entropy Compression (LZW) increase performance in Webservices and it degrades performance of http requests
Streaming has better performance than model downloading19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGY
AUGUST 8-10 – VANCOUVER, CANADA
MPEG DASH in real-time graphics delivery?
The 2nd experiment
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA
MPEG DASH (1/4)
The MPEG-DASH streaming adaptation approach is conceptually simple.
A client (player) application accesses the Media Presentation structure of a media, normally stored in an XML file with the extension .mpd, and selects the appropriate Adaptation Sets corresponding to the characteristics of the end-device and the end-user (e.g. audio tracks in specific languages, video tracks in specific encodings).
Then, the client begins downloading and playing Representations from these adaptation sets, while at the same time probing for QoE metrics (e.g. achieved FPS rate, remaining power, computational resources available).
Source: MPEG DASH IF
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA
MPEG DASH 2/4
Mesh parameters
Texture Image/Vide
o
3D Model
MPD Structure
…
Representation #1
Representation #2
Representation #3
Multimedia Presentation Description
Stream preparation
X3D VR World
The media (3D) file preparation
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA
MPEG DASH 3/4
Host Server1
Host Server2
Host Server(n)
Presentation Quality (LOD)
Decision
Client
Network
Memory
Battery
FPS
Timer
Metrics Probing
X3DOMDisplay
Web Server
The MPEG DASH player
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA
MPEG DASH 4/4The test-bed and results
http://mclab1.medialab.teicrete.gr:8081/indexdash.html
19TH INTERNATIONAL CONFERENCE ON 3D WEB TECHNOLOGYAUGUST 8-10 – VANCOUVER, CANADA