interfaces of the future now available augmented reality - google glass - 3 d sensors
DESCRIPTION
A tidal wave of new wearable tech, 3D sensors and displays is coming that will bring computers into our 3D world, and it's coming this year or the next. This presentation goes over the enabling technology (depth sensor, object tracking & recognition algorithms, better cpus and batteries), surveys several new devices coming out, and finally talks about the societal implications of having computers so much more tightly integrated into our world.TRANSCRIPT
![Page 1: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/1.jpg)
Interfaces of the Future, Available NowAugmented Reality Displays, Google Glass, 3D Sensors bringing computers into the 3D World.
March 10th, 2014
![Page 2: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/2.jpg)
Where’s My Flying Car?
![Page 3: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/3.jpg)
So many future promises from movies, so little delivery!
![Page 4: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/4.jpg)
Minority Report Hand Gesture Interface, Now Available
![Page 5: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/5.jpg)
Leap Sensor ($70) Demonstration
Finger and palm tracking, object translation, scaling and rotation.
![Page 6: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/6.jpg)
Talk Outline● Consumer Technology Enabled Experiences
○ Augmented Reality○ Wearable & Pervasive Computing○ Gestural Control
● Technology○ 3D Sensing○ Displays○ Wearable
● New device survey● Societal Implications
Terminator vision
![Page 7: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/7.jpg)
Sentrinsic Background● Formed in 2006 on patented linear sensing tech from
GaTech lab● Developed embedded control
system to improve efficiency of Air-Operated Diaphragm Pumps
![Page 8: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/8.jpg)
Sentrinsic Background● Mobile Integration○ Bluetooth Pump Monitoring○ Customer iPhone app
■ Scan pumps for documentation, order parts, track maintenance
■ Product: EquipCodes.com
● Augmented Reality○ Maintenance○ Operation Theory○ Training
● Google Glass Manufacturing Assembly Training app
EquipCodes app
![Page 9: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/9.jpg)
Augmented Reality● What is it?● Examples
○ 1st & 10 yellow line○ Simple text
■ translations■ nearby attractions■ instructions
![Page 10: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/10.jpg)
Augmented Reality Demos● Pump Maintenance
http://www.youtube.com/watch?v=NYpl0PD5Le4http://www.youtube.com/watch?v=nHfY56lHZjU
● Circuit Building Guidehttp://www.youtube.com/watch?v=AfVQ4N-u0sk
![Page 11: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/11.jpg)
Vein VisualizationEvana Eyes-On™ Glasses multi-spectral imaging
![Page 12: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/12.jpg)
Enabling Technology● Improved cameras, processors, memory● Pose estimation
○ Fiducial tracking○ Texture feature-based tracking○ SLAM - Simultaneous Location and Mapping○ 3D Sensors
![Page 13: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/13.jpg)
Fiducial Tracking
http://www.hitl.washington.edu/artoolkit/Papers/ART02-Tutorial.pdf ARToolKit Algorithm
Goal: find pose of camera (Rotation and Translation matrix)
![Page 14: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/14.jpg)
Texture Feature Matching● Find interesting points (corners)
○ independent of orientation, scale, lighting● Use surrounding area to develop fingerprint of feature
(128 pt vector, robust to skew).● Based on neuron
receptors in visualcortex, sensitiveto spatial gradients
Distinctive Image Features from Scale-Invariant Keypoints, Lowe 2004
![Page 15: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/15.jpg)
Also used for Image Stitching - Using Feature Recognition
![Page 16: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/16.jpg)
Texture Tracking
Tracking Texture Features tracked using Vuforia SDK
![Page 17: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/17.jpg)
Texture Feature Matching● Compare feature vector against database (Euclidian
distance).● Can’t be regular pattern (checker board)● High contrast required● Glare problematic● Tracker must be large
relative to graphics toprevent jitter, misalignment.
![Page 18: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/18.jpg)
Extensible Tracking● Environment unknown beforehand● SLAM (Simultaneous Localization and Mapping) or PTAM (Parallel
Tracking and Mapping)○ Structure from Motion - Autodesk’s 123D○ Scene should be static○ For overlays, requires initial registration, not for games○ Videos: http://www.youtube.com/watch?v=i7ierVkXYa8 (3D structure from 2D pictures)○ http://www.youtube.com/watch?v=Y9HMn6bd-v8#t=50 (real-time map generation on iPhone)
● For overlays that must match exactly, still need some initial alignment.○ Edge Based tracking using 3D Models
■ Uses extensible tracking, initialized with best fit edge of model. (Metaio SDK, Sept, 2013)
○ Small tracker can still be used
![Page 19: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/19.jpg)
Cameraless TrackingGPS, compass, gryo,
accelerometerIlluminated Tracking
CastAR’s head tracking uses IR LEDsNokia City Lens
![Page 20: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/20.jpg)
Depth Sensors● Skips over many challenges with regular
camera tracking○ Directly measures object depth○ Can handle “featureless” textures (everything same color)○ Still requires object recognition○ Primarily used for body / hand tracking
● Examples○ Kinect, Leap Motion, Laser range finder
![Page 21: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/21.jpg)
How Kinect (ver 1) works● Structured Lighting (Infrared)
video: http://www.youtube.com/watch?v=uq9SEJxZiUg
![Page 22: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/22.jpg)
Kinect (ver 1) Drawbacks● Affected by Sunlight● Multiple kinects would conflict● Minimum distance several feet - bad for
robotics and hand tracking (low resolution)● Very cheap, though!
![Page 23: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/23.jpg)
Time of Flight Sensor● Uses speed of light and arrival time of reflected pulses
to calculate distance● Extremely small times:
○ D = t * c / 2, 1cm → 66ps○ 3 GHz cpu → 333ps
● Advantages: more compact (no separation between sensor and emitter), simplier calculation, higher resolution
● Kinect v2, Soft Kinetic sensor.
![Page 24: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/24.jpg)
3D Sensing Applications● Augmented Reality● Body tracking
○ general computer gestural control, more natural○ gaming - Dance games○ head tracking - CastAR, Virtual Reality○ 3D design / printing
![Page 25: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/25.jpg)
Google Project Tango● Adding depth sensors to phones● Incorporating mapping and tracking
technology● Applications
○ visually impaired navigation○ instant measurement of
home dimensions○ store & shelf navigation
video: http://www.youtube.com/watch?v=Qe10ExwzCqk
![Page 26: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/26.jpg)
Wearable Phones?
Many of the applications make more sense if you don’t have to hold a phone!
● Maintenance - hard to hold iPad with wrench!
● Mapping
![Page 27: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/27.jpg)
Wearable / Pervasive Computing
● Google Glass○ Bad for AR?
■ Battery life■ Camera can’t be on constantly■ Small field of View, not in direct
line of sight○ Can’t focus on glass and outside
world - looking on wiki mid conversationjust as disruptive on Glass as on phone. (driving?)
![Page 28: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/28.jpg)
![Page 29: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/29.jpg)
Google Glass Field of View - Vignette Greg Roberts GlassFAQ
![Page 30: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/30.jpg)
Google Glass Pros● We check our phones 110 times / day● Better for contextual information
○ Incoming emails and texts○ nearby places of interest○ Instructions - following and creating (mfr DEMO)
● Voice recognition is great● Capturing spontaneous or 1st person videos● Looks cool, lightweight, viable wearable● No tether
![Page 31: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/31.jpg)
Head Mounted Display HistorySteve Mann
Thad Starner - GaTech!from 1993
Wearable Tech not new! Just more compact and marketed! Battery & CPU finally getting small and cheap enough.
![Page 32: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/32.jpg)
More head mounted options● GlassUP
○ $300 ○ small screen○ tethered
![Page 33: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/33.jpg)
● Recon Jetdesigned for Sporting$600, shipping May?
● Meta Pro○ Does what people think Google Glass does○ built-in depth sensor, potential
Iron Man / Minority report int.○ 40 degree Field of View
(vs 14 degree on Glass)○ $3650, shipping Sept○ tethered controller
![Page 34: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/34.jpg)
CastAR & Spacial AR● Projection on reflective mat● Head tracking via LEDs● Hologram effect● form of Spatial AR
interaction w/ projections
![Page 35: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/35.jpg)
Oculus Rift / Crystal Cove● Primarily for gaming, Virtual Reality● Fully immersive 110 deg Field of View● Head tracking via accel, gyro sensors,
markers on ver2● Some have added cameras to achieve
Augmented Reality effect○ Solves delay problems with see-thru displays
● Motion sickness problem - display must react quickly to head movement.
● Requires fast frame rates to trick brain
Video: Oculus Rift with 2 Cameras achieves Augmented Realityhttp://www.youtube.com/watch?v=Bc_TCLoH2CA
![Page 36: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/36.jpg)
Technology Hurdles● Latency: see-through AR must be faster.
○ Google glass small screen prevents motion sickness, but lag will be more jarring in large screens
● Dimming screen○ per-pixel dimming not viable yet, see-through text can
get washed out. (Video pass-thru helps)● Better, Smaller, Faster (Battery, CPU, screen)● Object & Gestural recognition algorithms
![Page 37: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/37.jpg)
Societal Implications● Always on camera● How do we have a conversation with computer
distractions? Wearable manners?● Engineering - Design in 3D with hands, more natural
gestural control● Job Creation?
○ Reverse of robotic automation - use adept human dexterity ○ Enhance human capabilities via internet, AI or remote assistance.○ Remote diagnostics, field service, DIY repair
![Page 38: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/38.jpg)
Conclusion & Final Thoughts
● Consumer Computers sense and project in 3D space
● Technology more pervasive / always on via wearable implementations
● Content is major bottleneck● If useful, appearance won’t matter
![Page 39: Interfaces of the future now available augmented reality - google glass - 3 d sensors](https://reader034.vdocuments.site/reader034/viewer/2022051611/54b74ee24a795967588b45da/html5/thumbnails/39.jpg)
Contact● Scott Driscoll
○ [email protected]○ plus.google.com/+ScottDriscoll○ blog: ImponderableThings.com ○ youtube: CuriousInventor
● Company Website: EquipCodes.com● Software Tools
○ Vuforia, Unity, Metaio, Google App Engine