web3d - semantic standards, webgl, hci
Post on 18-Oct-2014
328.000 views
DESCRIPTION
TRANSCRIPT
Web 3D [ semantic standards, webgl, hci ]
passionate about low-level 3D coding
embedded devices
enthusiast
graphics programmer
@victorporof GSoC student at Mozilla h8p://github.com/victorporof
Web 3D
Web 3.0D
Web 3.0?
“computer is generating new information, rather than humans”
Conrad Wolfram
Web 3.0?
Definitions vary greatly
¤ Sematic web ¤ “web of data” that enables machines to understand the
semantics, or meaning of information
¤ machine-readable metadata about pages and how they are related to each other
¤ Personalization
Web 3.0?
“first generation Metaverse”
John Smart
Metaverse
¤ web development layer
¤ extension of Web 2.0's technologies (and social networks)
¤ open video
¤ 3D simulations
¤ augmented reality
¤ human-constructed semantic standards
3.0D?
© pl4n3
3.0D?
The wrong way
¤ X3D/ VRML (slow, XML)
¤ Java 3D (slower)
¤ QuickTime VR (fake panoramas)
¤ 3DMLW (similar to X3D, scripting in Lua)
¤ Flash (proprietary, limited, and of course - slow)
3.0D?
¤ software library
¤ extends the capability of JavaScript
¤ generate interactive 3D graphics
¤ any compatible web browser
¤ context of the canvas HTML element
3.0D?
¤ software library
¤ extends the capability of JavaScript
¤ generate interactive 3D graphics
¤ any compatible web browser
¤ context of the canvas HTML element
3.0D?
¤ software library
¤ extends the capability of JavaScript
¤ generate interactive 3D graphics
¤ any compatible web browser
¤ context of the canvas HTML element
WebGL
3D computer graphics API
without the use of plug-ins
WebGL
3D computer graphics API
without the use of plug-ins
WebGL
3D computer graphics API
without the use of plug-ins
WebGL
3D computer graphics API
without the use of plug-ins
Big players
The WebGL working group includes
Apple, Google, Mozilla, and Opera
Big players
The WebGL working group includes
Apple, Google, Mozilla, and Opera J
their browser doesn’t support WebGL
When?
¤ Canvas 3D experiments started by Vladimir Vukićević ¤ at Mozilla
¤ prototype in 2006
¤ end of 2007: both Mozilla and Opera had made their own separate implementations
¤ early 2009: Mozilla and Khronos started the WebGL Working Group
Libraries
¤ GLGE
¤ C3DL
¤ Copperlicht
¤ SpiderGL
¤ SceneJS
¤ Processing.js
¤ Three.js
Libraries
¤ GLGE
¤ C3DL
¤ Copperlicht
¤ SpiderGL
¤ SceneJS
¤ Processing.js
¤ Three.js
Libraries
¤ GLGE
¤ C3DL
¤ Copperlicht
¤ SpiderGL
¤ SceneJS
¤ Processing.js
¤ Three.js
We’ll see some three.js demos
Hot on the internet a while ago… http://www.chromeexperiments.com/webgl
Yup, it’s made using WebGL http://chrome.angrybirds.com/
Hot on the internet right now… http://www.pixelnerve.com/webgl/sss/sss_singlelight.html
How does it work?
How does it work?
The three laws of a GPU:
1. It’s not a CPU
2. loves geometry & pixel algebra
3. thinks like a SIMD*
The three laws of a GPU:
1. It’s not a CPU
2. loves geometry & pixel algebra
3. thinks like a SIMD*
*SIMD = is a class of parallel computers: single instruction, multiple data
Programs
¤ Vertex shader
¤ Fragment shader
¤ Geometry shader ¤ Hot out of the oven
¤ Not very used
¤ Not quite supported
¤ Not really useful (arguable)
GLSL
¤ GLslang J
¤ high level shading language
¤ based on the C programming language ¤ (yes, it’s high level!)
¤ looks a lot better than first shader languages
Horror
DEFINE LUMINANCE = {0.299, 0.587, 0.114, 0.0}; TEX H0, f[TEX0], TEX4, 2D; TEX H1, f[TEX2], TEX5, CUBE; DP3X H1.xyz, H1, LUMINANCE; MULX H0.w, H0.w, LUMINANCE.w; MULX H1.w, H1.x, H1.x; MOVH H2, f[TEX3].wxyz; MULX H1.w, H1.x, H1.w; DP3X H0.xyz, H2.xzyw, H0; MULX H0.xyz, H0, H1.w; TEX H1, f[TEX0], TEX1, 2D; TEX H3, f[TEX0], TEX3, 2D; MULX H0.xyz, H0, H3; MADX H1.w, H1.w, 0.5, 0.5;
Better
#version 140 uniform Transformation { mat4 projection; mat4 modelview; }; in vec3 vertex; void main() { gl_Position = projection * modelview * vec4(vertex, 1.0); }
Best
Make the triangle pretty Color it red Kthxbye
Best
Make the triangle pretty Color it red Kthxbye
Shader toy
Interaction
¤ Display/ computer screen is 2D
¤ Most input devices are 2D ¤ Mouse, trackpad etc.
¤ 3D environments are 3D
Interaction
¤ Display/ computer screen is 2D
¤ Most input devices are 2D ¤ Mouse, trackpad etc.
¤ 3D environments are 3D
Interaction
¤ Display/ computer screen is 2D
¤ Most input devices are 2D ¤ Mouse, trackpad etc.
¤ 3D environments are 3D
3D interaction
¤ a form of human-computer interaction
¤ users are should be able to move and perform interaction in 3D space
¤ the 3D space used for interaction can be ¤ the real physical space
¤ virtual space representation simulated in the computer
¤ a combination of both
3D interaction
¤ 1962 – Sensorama simulator (Morton Heilig) ¤ 3D video feedback
¤ motion
¤ audio
¤ haptic (tactile)
3D interaction
¤ 1962 – Sensorama simulator (Morton Heilig) ¤ 3D video feedback
¤ motion
¤ audio
¤ haptic (tactile)
3D interaction
¤ should be intuitive ¤ after all, humans interact in three dimensions in the real world
¤ interaction devices – 3D interfaces ¤ used in applications that feature virtual environments, and
augmented and mixed realities
3D interaction
¤ should be intuitive ¤ after all, humans interact in three dimensions in the real world
¤ interaction devices – 3D interfaces ¤ used in applications that feature virtual environments, and
augmented and mixed realities
3D interaction - Output
“Currently, users still have difficulty in interpreting 3D space visuals and
understanding how interaction occurs”
3D interaction - Output
¤ visual displays ¤ Head-mounted displays and CAVEs
Cave Automatic Virtual Environment
¤ semi-immersive displays allow users to see both
¤ auditory displays ¤ specially useful when supplying location and spatial
information to the users
¤ haptic displays ¤ tactile feedback or feeling
3D interaction - Output
¤ natural way for humans to move around in a three-dimensional world
¤ many sensory cues present in real environments are missing from virtual environments
¤ sensory cue = “a statistic or signal that can be extracted from the sensory input by a perceiver, that indicates the state of some property of the world that the perceiver is interested in perceiving”
¤ 2D surface = inconsistencies in depth perception
3D interaction - Input
“Using 3D representations is not enough to create 3D interaction.
The users must have a way of performing actions in 3D as well”
3D interaction - Input
¤ special/ modified input devices ¤ ex: 3D mouse
¤ trackers ¤ detect or monitor head, hand or body movements
¤ important for presenting the correct viewpoint
¤ coordinate the spatial and sound information
¤ ex: motion trackers, eye trackers, data gloves, bodysuits
3D interaction input techniques
¤ selection and manipulation ¤ selecting, rotating and moving an object
¤ direct-hand manipulation is the most natural technique
¤ navigation ¤ wayfinding
¤ travel
¤ system control (menus, gestures, voice commands)
¤ symbolic input (add or edit text, ex: annotation)
Edusim
¤ free and open source
¤ 3D virtual worlds on an interactive whiteboard
¤ direct manipulation ¤ 3D virtual learning models
¤ Constructionist Learning
Open Cobalt Project
¤ free and open source software platform (Smalltalk)
¤ constructing, accessing, and sharing virtual world ¤ on local area networks
¤ across the Internet
¤ without any requirement for centralized servers
¤ create deeply collaborative and hyperlinked multi-user virtual workspaces
Mozilla - Tilt Project
¤ 3D visualization of a webpage ¤ WebGL
¤ visualization tool
¤ a developer-friendly environment for debugging
¤ information displayed on request: ¤ document’s structure
¤ nesting of the DOM tree
¤ each node’s type, class, id, and other attributes if available
Mozilla – Tilt Project
¤ https://github.com/victorporof/Tilt
¤ http://blog.mozilla.com/Tilt
¤ Shedule: April 25th – August 22nd
¤ Deliverables: ¤ Firefox extension + export to compatible browsers
¤ WebGL javascript library designed to facilitate creating web page DOM visualizations
Questions? O_o
© pl4n3