open standards for building virtual and augmented realities · •gltf manages its roadmap very...
TRANSCRIPT
![Page 1: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/1.jpg)
Open Standards for Building Virtual and Augmented Realities
Neil Trevett | Khronos PresidentNVIDIA | VP Developer Ecosystems
![Page 2: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/2.jpg)
2
Khronos Mission
Software
Silicon
Khronos is an International Industry Consortium creating royalty-free, open standards to enable software to access hardware acceleration for
3D Graphics, Virtual and Augmented Reality, Parallel Computing, Neural Networks and Vision Processing
Asian Members
![Page 3: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/3.jpg)
3
Why Open Standards?
Bad Standards - Overprescribes implementation details
- Forces everyone to implement a lowest common denominator
- Stifles innovation-> Commoditization
Good Standards- Prescribes only interoperability
- Enables implementationdiversity
– Encourages innovation-> Differentiation
An INTEROPERABILITY STANDARD enables two interoperating entities to COMMUNICATE
A Truly OPEN standardIs not controlled by a single company – but by the whole industry
Is freely available to use by any company without royalty paymentsHas well defined IP Framework to protect standard AND member’s IP
E.g. Software <-> Hardware
Standards Grow Markets…By reducing consumer confusion and increasing capabilities and usability
… Reduce Costs …By sharing development between many
companies and driving volume
… Accelerate Time to MarketWith well-proven testing and interoperability
![Page 4: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/4.jpg)
4
Khronos Standards for AR and VR
Vision and sensor processing -including neural network
inferencing for machine learning
High-performance, low-latency 3D Graphics
Portable interaction with VR/AR sensor, haptic and
display devices
Download 3D object scene data
![Page 5: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/5.jpg)
5
Vulkan – New Generation 3D Graphics
Modern architecture | Low overhead| Multi-thread friendlyEXPLICIT GPU access for EFFICIENT, LOW-LATENCY, PREDICTABLE performance
Non-proprietary, royalty-free open standard ‘By the industry for the industry’Portable across multiple platforms – both desktop and mobile
VR-Specific FeaturesNOW
Multi-GPU supportMultiview Rendering
Context priorityFront buffer rendering
Evolving 3D API features forVR displays and compositors
IN DISCUSSIONVariable Rate Rendering
Tiled rendering (beam racing)
![Page 6: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/6.jpg)
6
XR = AR + VR
V1.0 - focused on VR
V
AAfter 1.0 – equal focus on AR
![Page 7: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/7.jpg)
7
OpenXR – Solving VR/AR Fragmentation
Proprietary Engine
VR App
1
AR App
2
ARApp
4
VRApp
3
VR Device
1
VR Device
2
VR Device
3
VR Device
5
VR Device
4
After OpenXR Wide interoperability of XR apps and devices
VRApp
1
ARApp
2
ARApp
4
VRApp
3
Proprietary Engine
VR Device
1
VR Device
3
VR Device
5
VR Device
2
VR Device
4
Before OpenXRXR Market Fragmentation
Device Integration Layer
Application Interface
WebXR WebXR
![Page 8: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/8.jpg)
8
OpenXR Architecture
Portable AR/VRInput Device Discovery
Multiple Sensor TrackingDevice Events
Pose NormalizationHaptics Control
Optical Corrections
![Page 9: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/9.jpg)
9
Companies Publicly Supporting OpenXR
OpenXR is a collaborative designIntegrating many lessons from proprietary ‘first-generation’ API designs
![Page 10: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/10.jpg)
10
Input and Haptics Abstraction• Apps use abstracted Input Actions
- E.g. “Move,” “Jump,” “Teleport”• Device Events mapped to Input Actions
- For the specific system in use• Many advantages
- Content can use new devices with no code changes
- Can mix-and-match multiple input sources to create a unified UI
- Easy optional feature support (e.g. eye and body tracking)
- Future-proofing for innovation in input devices and form factors
![Page 11: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/11.jpg)
11
OpenXR Viewport Configuration Flexibility• Applications can:
- Query for runtime supported Viewport Configurations- Applications can then set the Viewport Configurations that they plan to use- Select and change their active configuration over the lifetime of the session
Photo Credit: Dave Pape
Camera Passthrough AR Stereoscopic VR / AR Projection CAVE-like
One Viewport Two Viewports (one per eye) Twelve Viewports (six per eye)
/viewport_configuration/ar_mono/magic_window /viewport_configuration/vr/hmd /viewport_configuration/vr_cube/cave_vr
![Page 12: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/12.jpg)
12
Feedback
OpenXR Development ProcessCall for Participation / Exploratory Group Formation
Fall F2F, October 2016: Korea
Statement of Work / Working Group FormationWinter F2F, January 2017: Vancouver
Specification WorkSpring F2F, April 2017: AmsterdamInterim F2F, July 2017: Washington
Defining the MVPFall F2F, September 2017: Chicago
Resolving Implementation IssuesWinterim F2F, November 2017: Washington
Winter F2F, January 2018: Taipei
First Public InformationGDC, March 2018
Release Provisional Specification
Ratify and release Final Specification and Enable Conformant Implementations to Ship
Present DayComing Soon
Multiple implementations Underway!Final specification will incorporate
implementation feedback
Much more detailed specification overview and SIGGRAPH session videos:
https://www.khronos.org/developers/library/2018-siggraph
First Public Demonstrations!SIGGRAPH, August 2018
Conformance Tests and Adopters Program Finalize Implementations
![Page 13: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/13.jpg)
13
Epic ‘Showdown’ VR Demo at SIGGRAPH
Demo runs portably across StarVR and Microsoft Windows Mixed Reality headsets through the OpenXR APIs via an
Unreal Engine 4 plugin
![Page 14: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/14.jpg)
14
Bringing VR and AR to the WebWeb XR Apps
WebXR
Native XR Apps
System-exposed AR Capabilities
3D Engines 3D Engines
Close ongoing collaboration between
WebXR and OpenXR
Khronos providing the foundation for 3D and XR
in the Web and native applications
Future versions of OpenXR will include
cross-platform extended AR functionality
![Page 15: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/15.jpg)
15
glTF – The JPEG of 3D!
Efficient, reliable transmissionBring 3D assets into 1000s of
apps and engines
glTF 2.0 – June 2017Native AND Web APIsPhysically Based Rendering
Metallic-Roughness and Specular-Glossiness
glTF 1.0 – December 2015Primarily for WebGL
Uses GLSL for materials
glTF spec development on open GitHub – get involved!
https://github.com/KhronosGroup/glTF
Compact to TransmitSimple and Fast to Load
Describes Full ScenesRuntime Neutral
Open and Extensible
![Page 16: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/16.jpg)
16
glTF Ecosystem
Apps and Engines
Creation Tools
Repositories
Sony 3D Creator
Oculus
Paint 3D
Collada2gltf FBX2glTFgltf-vscode
glTF-validator glTF-asset-generator
Experience
Discover
DriveDemand
Create Windows Mixed Reality Home
Users
Modo
Mixed Reality Viewer3D Builder
Prep for 3D printing
![Page 17: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/17.jpg)
17
glTF Roadmap• glTF manages its roadmap very carefully – complexity is the enemy
- Mission #1: ensure widespread, consistent, reliable usage• Rollout new functionality first as extensions: e.g. Draco Mesh Compression
- Open source encoders and decoders are available- Extension is now shipping in more and more tools and engines
• New extensions are being developed…- E.g. Universal Compressed Texture Transmission Format
Mesh Compression Ratios
Stable CoreSpec
Domain-specificextensions stay as
extensions
New widely needed functionality ships first
as extensions
Integrate extensions into new core spec only when:1) Widespread need is confirmed by the industry
2) Widespread reliable implementation is enabled (e.g. open source)
![Page 18: Open Standards for Building Virtual and Augmented Realities · •glTF manages its roadmap very carefully – complexity is the enemy - Mission #1: ensure widespread, consistent,](https://reader035.vdocuments.site/reader035/viewer/2022063003/5f52522172624e7ad7235c7f/html5/thumbnails/18.jpg)
18
Summary• Khronos is creating cutting-edge royalty-free open standards for AR/VR
- For both native and Web applications• Please join Khronos!
- We welcome members from China and Asia- Influence the direction of industry standards- Get early access to draft specifications
• More Information- www.khronos.org- [email protected] @neilt3d
WeChat: neiltrevett