rtc building the metaverse apr21 - khronos group · 2021. 4. 28. · the metaverse? 3d application...

© The Khronos® Group Inc. 2021 - This work is licensed under a Creative Commons Attribution 4.0 International License

Building the Metaverse One Standard at a TimeNeil TrevettPresident, KhronosVice President Developer Ecosystems, NVIDIA


Topics

1. Just what do we mean by ‘the metaverse’?

2. How will it come into existence?

3. What role will standards play – and NOT play!

4. Which Khronos standards can help build the metaverse?


Discussions on connecting ecosystems with interoperable runtimes and economies for portable assets and avatars

Fortnight, PUBG, Valheim

Virtual spaces where gamers create their own content

Roblox, Minecraft, Manticore

Digital twins eventually encompassing the world

GE, Siemens, Bentley, NVIDIA

1992Humans, as avatars, interacting with each other and software agents, in a

three-dimensional space that uses the metaphor of the real world

What is the Metaverse?

3D application interoperability for real-time immersive collaborative simulation and design

NVIDIA Omniverse

Spatially indexing and streaming the digitized world

OGC, Cesium

A spatial evolution of the Web

Augmenting Reality with persistent geo-locking, linking, occlusion and realistic scene illumination

PTC, Open AR Cloud, Niantic

VR environments where users can import their own authored assets

Microsoft Flight Simulator


How Will the Metaverse Evolve?

Evolution will be organicDarwinian mix of ideas, innovations, commercial successes and failuresOverlapping, messy, synergistic,

awesome experimentation

Proven solutions will emerge and be

widely adoptedProviding the foundation for the next round of innovation

What role can Open Standards play in this process?

Not as a single platformNo one design or roadmap can serve all

needs and solve all problems …Synthetic visual reality, practical XR optics,

effective remote social interactions, server scaling to millions of users, online personas and social

connections, realistic avatars, security, privacy, sharable assets, interoperable run-times, online economies and currencies, real-world XR geo-

anchoring with persistence, universal digital twins, IOT sensor networks, pervasive low-latency

wireless connectivity, pervasive 3D commerce, real-time environment scanning and semantic

understanding… and many more …Significant opportunities for all companies

contributing to metaverse solutions!


What is an Open Standard?INTEROPERABILITY standards define precise COMMUNICATION

Through a specification that enables multiple conformant implementations

E.g., Software <-> Hardware Communication through an Application Programming Interface (API)

True OPEN standards are not controlled by any single companyStandards Developing Organizations (SDOs) enable industry cooperation

Inclusive participation, well-defined governance and intellectual property frameworksStronger industry consensus and participation increases potential Adoption

Growing MarketsBy reducing consumer confusion and increasing

capabilities and usability

Reducing CostsBy sharing development between many

companies and driving volume

Speeding Time to MarketWith well-proven functionality, testing

and interoperability

Enabling InnovationCompanies compete on implementationquality, performance, power etc. etc.

Successful Open Standards Share Core Goals


Don’t Need Standards?Making the metaverse

ubiquitous will involve a constellation of standards!

IEC 60038 Standard voltagesIEC 60228 Conductors of insulated cables

IEC 60269 Low-voltage power fusesIEC 60320 C13 Connectors and C14 Inlets

IEC 60884 Household Plugs And Socket-OutletsIEC 61970 APIs for energy management systems

Widely adopted platforms require multiple standards

Standards are the basis for ubiquitous infrastructure


Proprietary Products and Technologies

Rapid InnovationDarwinian testing ground

Smart Innovators can retain long-term advantage

Complex and Interdependent

Relationship

Technology becomes ProvenBusiness interests are better served

by cooperation than competition

Areas of Emerging Consensus Beachhead standardization opportunities

Proprietary Technology and Open Standards

Multi-company GovernanceOpen Standards

Do not R&D!!Satisfy wider need for technologies

Thread of continuity for industry forward progress

Standards Move the Industry Forward Frees competitive energy to find new ways

to innovate value


Open Standards and Open Source

Open Standard = Shared Specification

Implementation

Implementation Implementation

Implementation

Implementation

Best When …Competitive advantage in implementation innovation

Industry needs multiple implementationsNeed a stable design target

But..Can take time to generate consensus on a new version

Conformance testing is vital

Not the same thing! Both are powerful tools in the right situationChoose the right tool to make best forward progress

Often positive synergy between the two e.g.,Open-source sample implementations of open standards

Tools, samples, conformance tests for open standards in open source

Contributor

Contributor

Contributor

Contributor

Contributor

Open Source = Shared Implementation

Best When …No competitive advantage in implementation

Industry consensus to share implementation resourcingNeed rapid updates

But..Can fork and fragment

Need governance model clarity

Many open-source projects have concentrated control – not multi-company governance


Khronos Standards as Metaverse Components

High-performance, low-latency 3D Graphics

Pervasive, universal 3D assetsEnabling reliable deployment of 3D in e-commerce at industrial scale

Vision and sensor processing Accelerated machine

learning and inferencing

Portable interaction with VR/AR sensor, haptic and

display devices


XR Portability

OpenXR provides cross-platform, high-performance access directly into XR device runtimes across multiple platforms

XR App/Experience

GameEngine

Application Interface

WebXR

XR App/Experience

XR App/Experience

AndMore!

Virtual Reality

V

AAugmented Reality


Advanced UI Cross-Vendor OpenXR ExtensionsDevelopers can build cross-platform applications that useadvanced UI solutions from different technology vendors

OpenXR API layers can be used implement extensions

Eye Tracking

Eye gaze interaction for intuitive interfaces 2-Step Interaction

Hand-eye coordinationNatural aiming

Shipping on HoloLens 2

Hand Tracking

26 unique joints per hand for fully articulated hands visible to the user

Shipping on HoloLens 2and Microsoft Hand Mesh Extension

for HoloLens 2 layers over it

Ultraleap developer preview available


OpenXR Widespread Industry AdoptionConformant Implementations

Microsoft HoloLens 2Windows Mixed Reality Headsets

Oculus Rift SQuest and Quest 2 HTC Vive Cosmos

Developer Preview Implementations

Valve SteamVRTransitioning from OpenVR to OpenXR

VarjoPreliminary support shipping now

CollaboraOpen-source Implementation

Engine Support

Unreal EngineSupport in 4.24. Optimizations in 4.25

Unity EngineOpenXR Plugin Preview in 2020.2+

Chromium 81 for Chrome and Edge BrowsersOpenXR default backend for WebXR


The Web Will be Part of the Metaverse

https://xkcd.com/1367/


Bringing XR to the Web

Web XR AppsNative XR Apps

Native 3D Engines

Web 3D Engines

Khronos provides the foundation for native and Web-based 3D/XR

Close cooperation between WebXR

and OpenXR

Lifting OpenXR functionality into

the Web stack


glTF – The JPEG of 3D!

Efficient, reliable transmission

Bring 3D assets into 100s of apps and engines

glTF 2.0 – June 2017Native AND Web APIsPhysically Based Rendering

Metallic-Roughness and Specular-Glossiness

glTF 1.0 – December 2015Primarily for WebGL

Uses GLSL for materials

glTF spec development on open GitHub – get involved!

https://github.com/KhronosGroup/glTF

Compact to TransmitSimple and Fast to LoadDescribes Full Scenes

Runtime NeutralOpen and Extensible

Audio Video Images 3D

MP3 H.264 JPEG

New market opportunities for 3D content creation

and deployment!


https://github.khronos.org/glTF-Project-Explorer/


glTF Universal GPU Textures & PBR Roadmap

KTX 2.0 textures supportBasis Universal supercompression

20th April 2021glTF assets with compact, high-quality textures

that can be efficiently converted to GPU compressed textures on-the-fly on diverse target platforms to reduce both asset file download sizes AND GPU memory usage

https://www.khronos.org/ktx/

Future Waves of glTF PBR Extensions

Subsurface Scattering, Attenuation,

Index of Refraction (IOR), Thickness, Specular Color, Anisotropy, Translucency,

Thin Film (iridescence) and more…

glTF PBR Extensions

December 2020Transmission

ClearcoatSheen


Using Open Standards to Build the Metaverse• Seek friction points in your industry caused by lack of agreed communication protocols- Between people, companies, products, subsystems, components- Is the potential solution to better communication well understood?

• Find or create a safe space for industry cooperation to define interoperability standards- With well defined, consensus-based governance model and effective IP framework- Standardize when the need is wide – not when doing R&D

• Neil Trevett- [email protected] | @neilt3d

Gather industryrequirements for future

open standards

Draft SpecificationsConfidential to Khronos

members

Publicly Release Specifications and Conformance Tests

Gain early insights into industry trends and directions

Influence the design and direction of key open standards that will drive your business

Accelerate your time-to-market with early access to specification drafts

Network with domain experts from diverse companies in your industry

State-of-the-art IP Framework protects your Intellectual Property

Enhance your company reputation as an industry leader through Khronos participation

Benefits of Khronos membership www.khronos.org

rtc building the metaverse apr21 - khronos group · 2021. 4. 28. · the metaverse? 3d application...

Documents