virtual reality (p-5)

7/28/2019 Virtual Reality (P-5)

1/20

Virtual Reality Movies - Real-Time Streaming of 3D ObjectsStephan Olbrich

1999 RRZN/RVS, University of Hannover

TNNC 9908.06.1999

1

Virtual Reality Movies -

Real-Time Streaming of 3D Objects


2/20



TNNC 9908.06.1999

2

Overview

Introduction Requirements

Tele-immersive, virtual reality environment

Series of complex 3D scenes: result of scientific visualization

High-performance online presentation

Solution Generation of 3D scenes: optimized DVR file format

Streaming server: play-out 3D animations

Efficient DVR viewer: Inline-Plugin for Netscape

Evaluation 3D streaming application: Oceanic Convection Local gigabit ethernet configuration

Future work


3/20



TNNC 9908.06.1999

3

Introduction

Distributed multimedia information systems WWW - Based on Internet Standards (RFCs), e. g.:

Protocols - TCP/IP

Addressing Schemes - URL

Services - HTTP

Hypermedia Content - MIME, HTML, GIF, JPEG, MPEG, VRML, etc.

Client - Generic browserNetscape Communicator, Microsoft Internet Explorer

Extension techniques - Helper applications, Plugins, ActiveX Control, Java

3D/Virtual Reality Technology - application areas Ergonomic user interfaces

Method to navigate in an information space

Online presentation of virtual 3D scenes, for example:Reproduction of real objects

Artificial scenes, such as Scientific Visualization of HPC results

ISO Standard: Virtual Reality Modeling Language (VRML97)Container format for multimedia content in the Web


4/20



TNNC 9908.06.1999

4

Requirements (1)

3D/Virtual Reality Technology - Applications

Information Spaces / 3D User Interfaces Intuitive human-machine-human interface

Ergonomic navigation

Visualization of geometric objects

(modeling / reconstruction) Design: virtual prototyping

Arcitecture, environment planning

Visualization in scientific computing Experiment / Theory / Simulation

Virtual experimentsExploration and demonstration of results

Increase of insights / Teleteaching / Telelearning


5/20



TNNC 9908.06.1999

5

Requirements (2)

Visualization in Scientific Computing

Raw data,

e.g. temperature

SimulationSupercomputer

e.g. Cray T3E

Symbolic

representation

Postprocessing3D / VR

visualization

systems

e.g. AVS on

SGI Onyx2

Acoustic Haptic

Interaction

Visual

VR System

Viewer

2D, 3DDisplay

Multimedia

representation

VE - Vir tual

Environment


6/20



TNNC 9908.06.1999

6

Requirements (3)

Tele-Immersive Distributed Virtual Environment

Visualization ASynchronized

navigation,

annotation,

conferencing, etc.

3D objects,

HPC results,

etc.

data source(s):WWW, Super-

computer

Request

Visualization B

3D objects,

HPC results,

etc.

Request


7/20



TNNC 9908.06.1999

7

Constraints of available Internet-based 3D

systems

Several limits of VRML format and viewers Especially in the context of typical scientific

and industrial research environments with High quality application requirements, e. g.

handling objects with high complexity (> 100,000 polygons)

Network infrastructure offering high bitrates, such as local networks High performance server and client systems

Constraints regarding performance, quality, and

functionality aspects prohibit useful application unacceptable delays

no progressive presentation

low frame rates

low rendering quality

little support of virtual reality presentation and interaction techniques

no 3D stream ing c apabi l i t ies


8/20



TNNC 9908.06.1999

8

Requirements (5)

Analysis and Classification

Pre se ntatio n Co mmunicatio n Re pre se ntatio n

Standards

WWW BrowserPlugin-APIViewer, PluginsGraphics-/GUI-APIs

Network Infra st ru ctureTransport ProtocolsServices, e . g. WWW:

HTTP, URLWWW Serve rCGI-, Proxy-Techn iqu esStorageMM/VR File and Stream

Formats

Performance

Sho rt late ncy

StartupNaviga t ion

High frame rate sRendering rate

High b itrat es

Sho rt late ncyLitt le jitt erOptimized

implementation

Client-side computation

DecodeDecompressParseBuild scen e g rap h

Dat a volume sQuality

ResolutionAntialiasingColor man age men t

Qua lity o f service ResolutionCom pression m et ho dColor man age men t

Functionalit

On-the-flypre sen ta t ion

VR projectionsystems

Naviga t ion comfo rt3D sensors

StreamingHTTP, RTSP, RTP

SynchronizationMedia-specific scaling

Att ribu te s for VRpre senta t ion

Seq ue nces of 3D ob jectsVoo mie Virtual Movie


9/20



TNNC 9908.06.1999

9

Requirements (6)

Complexity of typical rendering primitives

N independent triangles

with normals

Volume: N * 72 byte

N triangles as triangle strips

with normal vectors

Volume: (N+2) * 24 byte

4,3 Mio. triang les/s 800

Mbit/s

(SGI Onyx2 Inf init e Reali ty) Per vertex data:

3D coo rdinates:3 float values per vertex 3 * 4 byte / vertex

Normal vector :3 float values / vertex 3 * 4 byte / vertex


10/20



TNNC 9908.06.1999

10

Solution (1a)

Partitioning/Distributing a Virtual Reality system

Position,

Orientation,

Button states

Navigation

Loop

Parse

Render

LowLevel-

Traverse

Scene repr.

HighLevel

Traverse/

Optimize

Scene graph Display list

3D

2D

File Memory Memory

Local navigation loop

Selectaccessnew

scene

Scene update(a) local file


11/20



TNNC 9908.06.1999

11

Solution (1b)


Scene update(b) WWW integration: VRML viewer

Position,

Orientation,

Button states

Navigation

Loop

Parse

Render

LowLevel-

Traverse

Scene repr.

HighLevel

Traverse/

Optimize


File Memory Memory

Select

WWW

Server

Scene repr.

File

WWW

Browser

HTTP

VRML

Helper application

exec

get URL


12/20



TNNC 9908.06.1999

12

Solution (1c)


Scene update(c) Preprocess VRML to DVR format

Position,

Orientation,

Button states

Navigation

Loop

Parse

Render

LowLevel-

Traverse

Scene repr.

HighLevel

Traverse/

Optimize


File Memory Memory

Select

WWW

Server

Scene repr.

File

WWW

Browser

HTTP

VRML

Helper application

exec

get URL

DVR format

VRML-to-DVR


13/20



TNNC 9908.06.1999

13

Solution (1d)


Scene update(d) Our DocShow-VR Plugin for DVR format (thin client)

Position,

Orientation,

Button states

Navigation

Loop

Render

LowLevel-

Traverse

Select

WWW

Server

Scene repr.

File

WWW

Browser

HTTP

DV

R

Plugin (DVR)

NPP_Write

NPN_GetURL

Display list

Memory buffer


14/20



TNNC 9908.06.1999

14

Solution (2)

Play-out of 3D animations: streaming system

Real-time streaming DVRS Plugin, RTSP/DVRP protocols

Navigation

Loop

Render

LowLevel

TraverseStreaming

Server

3D scene repr.

RTSP

DVR

Files

Plugin (DVRS)

Display list

Double buffer

DVRP

WWW

Server

Meta data

WWW

Browser

HTTP

Streaming

Client

Play, Stop, Slow motion


15/20



TNNC 9908.06.1999

15

Solution (3)

Performance issues

Optimized 3D file format Binary representation - IEEE format / network byte ordering

Preprocessing of VRML data, e. g.:ASCII cleartext to binary format

Linearization of scene graph (objects, attributes, transformations, lights)

Restructuring, precalculation for efficient and progressive rendering support

VRML: Coordinate lists / indexed references

Store and transmit direct values VRML: IndexedFaceSet - Polygons Triangles Triangle strip Calculation and storage of normal vectors

High-performance viewer (Netscape inline plugin) Tightly coupled into data delivery

Netscape Plugin API

Real-time streaming, initiated by special meta data MIME type (DVRS) Efficient 3D rendering implementation, based on OpenGL

Real-time streaming of sequences of 3D scenes: Virtual Reality MoviesControl protocol: RTSP (RFC 2326)

3D data transport: DVRP

Multithreaded: communication, 3D rendering (multi-pipe)


16/20



TNNC 9908.06.1999

16

Solution (4)

Other issues

Quality of Presentation Multisampling antialiasing support (HW-accelerated OpenGL extension)

Colour management support (device-independant presentation)

Functionality Stereoscopic viewing with shutter glasses or stereo projection

Tracking devices Multi-platform support (Win32, UNIX)

Interoperability to ISO StandardVRML97 Preparing files via VRML-to-DVR converter

VRML97 subset supported: static scenes

Command line tool / Web service via HTML form, VRML upload, DVR download

Caching proxy/gateway technique transparent access to any VRML file


17/20



TNNC 9908.06.1999

17

Evaluation (1)

Testbed setup

Client: SGI Onyx2 Infinite Reality (2x 2 RM), 4xR10000, 195 MHz Server: SGI Origin200, R10000, 225 MHz via Gigabit-Ethernet

Network: Gigabit Ethernet (back to back), Alteon Jumbo Frames

Application scenario:

Oceanic convection

Supercomputersimulation results161x161x31 grid

740 time steps

Visualization of

temperature, velocity

100,000 primitives 8 Mbytes / scene

Frames 720-739 used

for measurement


18/20



TNNC 9908.06.1999

18

Evaluation (2)

Measurement

[Mbps] ClientSGI On yx2

Server 1SGI Onyx2

Server 2SGI Origin200

Read DVR files(cached)

250350(9801020)

109110(10901170)

TCP/IP

transmission

790950

(loopback)

560670

i a bit e th ern et

3D ren de ring(OpenGL)

380420

Streaming Max. framerate

280320 324326

Streaming 4 fram es / s

251 251

Streaming 2 fram es / s

126 126


19/20



TNNC 9908.06.1999

19

Evaluation (3)

Illustration

Configuration Client: SGI Onyx2 Infinite Reality

Server: SGI Origin200

Communication: Gigabit Ethernet, Jumbo Frames

Dataset: Oceanic Convection, frames nr. 720-739

1 Graphic Pipe Maximal frame rate 4 frames per second

2 frames per second

2 Graphic Pipes

Maximal frame rate 4 frames per second 2 frames per second
http://localhost/var/www/apps/conversion/tmp/data/dvr4c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr1c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr2c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr3c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr3c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr4c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr4c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr1c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr3c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr1.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr1.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr3c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr1c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr4c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr4c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr3c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr3c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr2c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr1c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr4c.dvrhttp://localhost/var/www/apps/conversion/tmp/data/dvr4c.dvr


20/20


TNNC 9908.06.1999

Future work

Preparation of scenarios in wide-area networks Presentation (Prepared 3D scenes, as described)

Exploration (Steering / High-Performance Computing - HPC)

Discussion (Computer-Supported Cooperative Work - CSCW)

Support of an exploration scenario

Steering of a computation on a supercomputer Dynamic scene generation

Low-latency, on-the-fly transmission and presentation

Support of cooperative working Synchronization of navigation

Telepointer, annotations

Synchronized play-out of 3D animations

Streamed, spatial video and audio integrationVideo texture mapping on a rectangle in 3D space

e. g. SGI Onyx2 IR: DIVO/GVO - ITU-R 601 D1 digital video interfaces

e. g. SGI Onyx2 IR: ADAT - 8-channel digital audio interface

virtual reality (p-5)

Documents