hacking reality: browser-based vr with html5

Download Hacking Reality: Browser-Based VR with HTML5

Post on 02-Dec-2014

860 views

Category:

Technology

5 download

Embed Size (px)

DESCRIPTION

Slides from fall 2014 HTML5 Dev Conf talk on Browser-Based VR

TRANSCRIPT

  • 1. HACKING REALITY: HTML5 AND JAVASCRIPT FOR CROSS-PLATFORM VR TONY PARISI OCTOBER, 2014
  • 2. ABOUT ME CREDS Co-creator, VRML and X3D http://www.tonyparisi.com 10/21/2014 CONTACT tony@vizi.gl skype: auradeluxe http://twitter.com/auradeluxe http://www.tonyparisi.com/ http://www.learningwebgl.com/ GET VIZI https://github.com/tparisi/Vizi GET THE BOOKS! WebGL: Up and Running http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL http://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/ dp/1449362966 MEETUPS http://www.meetup.com/WebGL-Developers-Meetup/ http://www.meetup.com/Web-VR/ BOOK CODE https://github.com/tparisi/WebGLBook https://github.com/tparisi/Programming3DApplications GET GLAM http://www.glamjs.org/ https://github.com/tparisi/glam/ WORK http://www.vizi.gl/
  • 3. THE PROMISED LAND! CONSUMER VR http://www.tonyparisi.com 10/21/2014
  • 4. VR TODAY http://www.tonyparisi.com 10/21/2014 GIANT DOWNLOADS SILO EXPERIENCES CUMBERSOME NATIVE APPS AND INSTALLS PROPRIETARY PLATFORMS AND DEVELOPMENT STACKS VR DOTH BE HARD!
  • 5. WHY I LOVE THE WEB INSTANT PUBLISHING INSTANT ACCESS TO INFORMATION NO TOLLS NOBODY CONTROLS IT CULTURE OF COLLABORATION VIEW SOURCE THE WEB WILL NEVER CLOSE UP SHOP. image: Mark Surman http://commonspace.wordpress.com/2014/03/12/happybirthday/ http://www.tonyparisi.com 10/21/2014
  • 6. THE THREE DS OF THE WEB http://www.tonyparisi.com 10/21/2014 DEVELOPMENT CROSS-PLATFORM VENDOR-NEUTRAL OPEN SOURCE DEPLOYMENT CLOUD PUSH-BUTTON UPDATE AND PUBLISH DISTRIBUTION AND DISCOVERY EMBED IN OTHER PAGES SHARE WITH A HYPERLINK NO APP TO DOWNLOAD
  • 7. VR AND THE WEB: TWO GREAT TASTES ? http://www.tonyparisi.com 10/21/2014
  • 8. WEB VR FAST, CHEAP, AND TOTALLY DEMOCRATIZED. BROWSER-BASED VIRTUAL REALITY WEBGL CSS3 VR SUPPORT NOW IN BROWSER DEV BUILDS!!! NO BIG APP DOWNLOADS AND INSTALLS!!! http://www.tonyparisi.com 10/21/2014 JUST ADD SMART PHONE SMARTVR USING GOOGLE CARDBOARD $25 CHEAP!
  • 9. AN EXAMPLE http://www.tonyparisi.com 10/21/2014 INFORMATION DIVING SHOWCASE http://mozvr.github.io/infodive/ POWERED BY FIREFOX BUILT WITH VIZI https://github.com/tparisi/Vizi
  • 10. THE WEBVR API 1. QUERY FOR VR DEVICE(S) TO RENDER // polyfill var getVRDevices = navigator.mozGetVRDevices /* FF */ || navigator.getVRDevices; /* webkit */ http://www.tonyparisi.com 10/21/2014 var self = this; getVRDevices().then( gotVRDevices ); function gotVRDevices( devices ) { var vrHMD; var error; for ( var i = 0; i < devices.length; ++i ) { if ( devices[i] instanceof HMDVRDevice ) { vrHMD = devices[i]; self._vrHMD = vrHMD; self.leftEyeTranslation = vrHMD.getEyeTranslation( "left" ); self.rightEyeTranslation = vrHMD.getEyeTranslation( "right" ); self.leftEyeFOV = vrHMD.getRecommendedEyeFieldOfView( "left" ); self.rightEyeFOV = vrHMD.getRecommendedEyeFieldOfView( "right" ); break; // We keep the first we encounter } } } get left/right eye (camera) positions get per-eye camera field of view; use WebGL to render scene from two cameras
  • 11. THE WEBVR API 2. GO FULLSCREEN (VR MODE) fullscreen mode requires a DOM element http://www.tonyparisi.com 10/21/2014 var self = this; var renderer = this._renderer; var vrHMD = this._vrHMD; var canvas = renderer.domElement; var fullScreenChange = canvas.mozRequestFullScreen? 'mozfullscreenchange' : 'webkitfullscreenchange'; document.addEventListener( fullScreenChange, onFullScreenChanged, false ); function onFullScreenChanged() { if ( !document.mozFullScreenElement && !document.webkitFullScreenElement ) { self.setFullScreen( false ); } } if ( canvas.mozRequestFullScreen ) { canvas.mozRequestFullScreen( { vrDisplay: vrHMD } ); } else { canvas.webkitRequestFullscreen( { vrDisplay: vrHMD } ); } handle exiting fullscreen mode request fullscreen mode for this VR device
  • 12. THE WEBVR API 3. HEAD TRACKING query HMD device state http://www.tonyparisi.com 10/21/2014 // polyfill var getVRDevices = navigator.mozGetVRDevices /* FF */ || navigator.getVRDevices; /* webkit */ var self = this; getVRDevices().then( gotVRDevices ); function gotVRDevices( devices ) { var vrInput; var error; for ( var i = 0; i < devices.length; ++i ) { if ( devices[i] instanceof PositionSensorVRDevice ) { vrInput = devices[i] self._vrInput = vrInput; break; // We keep the first we encounter } } } // called once per tick from requestAnimationFrame() var update = function() { var vrState = this.getVRState(); if ( !vrState ) { return; } // vrState.hmd.rotation contains four floats, quaternion x,y,z,w setCameraRotation(vrState.hmd.rotation); }; get head-tracking VR device update camera to match HMD device orientation
  • 13. WEBVR AND CARDBOARD GOOGLE CARDBOARD SHOWCASE Mobile Chrome http://g.co/chromevr Desktop Chrome http://vr.chromeexperiments.com/ EVEN EASIER RENDER WEBGL SIDE-BY-SIDE STEREO (NO NEED TO QUERY DEVICES) USE BROWSER DEVICE ORIENTATION API FOR HEAD TRACKING http://www.tonyparisi.com 10/21/2014
  • 14. WEBVR AND THREE.JS THE MOST POPULAR WEBGL LIBRARY http://threejs.org/ LATEST STABLE VERSION (r68) INCLUDES STEREO RENDERING AND HEAD TRACKING RENDERING examples/js/effects/StereoEffect.js (Cardboard) examples/js/effects/VREffect.js (Rift) HEAD TRACKING examples/js/controls/DeviceOrientationControls.js (Cardboard) examples/js/controls/VRControls.js (Rift) http://www.tonyparisi.com 10/21/2014
  • 15. VIZI: A FRAMEWORK FOR WEBVR APPLICATIONS GOAL: MAKE IT EASY TO QUICKLY BUILD INTERESTING 3D APPLICATIONS ARCHITECTURE INSPIRED INSPIRED BY MODERN GAME ENGINE DESIGN COMPONENT-BASED DESIGN EASILY PLUG NEW FEATURES INTO OBJECTS APPLICATION OBJECT HANDLES EVENT LOOP, THREE.JS CREATION, PAGE RESIZE, ROUTING EVENTS TO OBJECTS INTERACTIONS MAKE IT EASY TO PUT MOUSE AND TOUCH INTERACTION ON OBJECTS BEHAVIORS PREBUILT BEHAVIORS AUTOMATICALLY ROTATE, MOVE ETC. PRE-BUILT OBJECTS FOR COMMONLY USED DESIGN PATTERNS SIMILAR IN DESIGN TO UNITY3D USES THREE.JS FOR ALL GRAPHICS ENHANCES THREE.JS VR RENDERING AND CONTROLLERS http://www.tonyparisi.com 10/21/2014
  • 16. OPEN TOOLS FOR CROSS-PLATFORM VR http://www.tonyparisi.com 10/21/2014 game engines/IDEs Goo Enginehttp://www.gootechnologies.com/ Verold http://verold.com/ Turbulenz https://turbulenz.com/ PlayCanvas http://www.playcanvas.com/ Artillery Engine https://artillery.com/ Sketchfab https://sketchfab.com/ Unreal https://www.unrealengine.com/ Unity http://unity3d.com/#unity-5 scene graph libraries/page frameworks Three.js http://threejs.org/ SceneJS http://scenejs.org/ BabylonJS http://www.babylonjs.com/ Vizi https://github.com/tparisi/Vizi Voodoo.js http://www.voodoojs.com/ PhiloGL http://www.senchalabs.org/philogl/ tQuery http://jeromeetienne.github.io/tquery/
  • 17. PRO TOOLS FOR WEB VR Unreal 4 in WebGL https://www.youtube.com/watch?v=c2uNDlP4RiE#t=42 60FPS ported in 5 days Unreal native C++ engine -> JavaScript Emscripten + asm.js http://www.tonyparisi.com 10/21/2014 EMSCRIPTEN - THE COOLEST HACK EVER! https://github.com/kripken/ems cripten CROSS-COMPILE C++ NATIVE CODE TO JAVASCRIPT asm.js- LOW-LEVEL OPTIMIZED JAVASCRIPT UNITY, UNREAL ENGINES USE THIS TO DEPLOY ON THE WEB WATCH OUT: HUGE DOWNLOADS AND HARD TO DEBUG. !
  • 18.