2013 426 lecture 2: augmented reality technology
DESCRIPTION
The second lecture in the 426 graduate class on Augmented Reality taught thy Mark Billinghurst at the HIT Lab NZ, University of Canterbury. The class was taught on July 19th 2013TRANSCRIPT
COSC 426: Augmented Reality
Mark Billinghurst
July 19th 2013
Lecture 2: AR Technology
Key Points from Lecture 1
Augmented Reality Definition Defining Characteristics [Azuma 97]
Combines Real and Virtual Images - Both can be seen at the same time
Interactive in real-time - Virtual content can be interacted with
Registered in 3D - Virtual objects appear fixed in space
What is not Augmented Reality?
Location-based services Barcode detection (QR-codes) Augmenting still images Special effects in movies … … but they can be combined with AR!
Milgram’s Reality-Virtuality Continuum
Mixed Reality
Reality - Virtuality (RV) Continuum
Real Environment
Augmented Reality (AR)
Augmented Virtuality (AV)
Virtual Environment
Metaverse
AR History Summary 1960’s – 80’s: Early Experimentation 1980’s – 90’s: Basic Research
Tracking, displays
1995 – 2005: Tools/Applications Interaction, usability, theory
2005 - : Commercial Applications Games, Medical, Industry
Applications
Medicine Manufacturing Information overlay Architecture Museum Marketing Gaming
AR Experience Design
“The product is no longer the basis of value. The
experience is.”
Venkat Ramaswamy The Future of Competition.
experiences
services
products
components
Valu
e
Sony CSL © 2004
Gilmore + Pine: Experience Economy
Function
Emotion
The Value of Good User Experience
20c
50c
$3.50
Good Experience Design Reactrix
Top down projection Camera based input Reactive Graphics No instructions No training
Apple: The Value of Good Design
Good Experience Design Dominates Markets
iPod Sales 2002-2007
Using the N-gage
SideTalking http://www.sidetalkin.com
Interaction Design
“Designing interactive products to support people in their everyday and working lives” Preece, J., (2002). Interaction Design
Design of User Experience with Technology Higher in the value chain than product design
Interaction Design involves answering three questions: What do you do? - How do you affect the world? What do you feel? – What do you sense of the world? What do you know? – What do you learn?
Interaction Design is All About You
Users should be involved throughout the Design Process
Consider all the needs of the user
Interaction Design Process
Gabbard Model for AR Design
1. user task analysis 2. expert guidelines-based evaluation 3. formative user-centered evaluation 4. summative comparative evaluations
Gabbard, J.L.; Swan, J.E.; , "Usability Engineering for Augmented Reality: Employing User-Based Studies to Inform Design,” Visualization and Computer Graphics, IEEE Transactions on, vol.14, no.3, pp.513-525, May-June 2008
Gabbard Model in Context
experiences
applications
tools
components
Building Compelling AR Experiences
Tracking, Display
Authoring
Interaction
Usability
Summary In order to build AR applications you need to
focus on the user experience Great user experience is based on
Low level AR component technology Authoring tools Application/Interaction design User experience texting
AR Technology
experiences
applications
tools
components
Sony CSL © 2004
Building Compelling AR Experiences
Display, Tracking
Core Technologies Combining Real and Virtual Images
• Display technologies Interactive in Real-Time
• Input and interactive technologies Registered in 3D
• Viewpoint tracking technologies Display
Processing
Input Tracking
AR Displays
AR Displays
e.g. window reflections
Virtual Images seen off windows
e.g. Reach-In
Projection CRT Display using beamsplitter
Not Head-Mounted
e.g. Shared Space Magic Book
Liquid Crystal Displays LCDs
Head-Mounted Display (HMD)
Primarily Indoor Environments
e.g. WLVA and IVRD
Cathode Ray Tube (CRT) or Virtual Retinal Display (VRD)
Many Military Applications & Assistive Technologies
Head-Mounted Display (HMD)
e.g. Head-Up Display (HUD)
Projection Display Navigational Aids in Cars
Military Airborne Applications
Not Head Mounted (e.g. vehicle mounted)
Primarily Outdoor (Daylight) Environments
AR Visual Displays
Display Technologies
Types (Bimber/Raskar 2003) Head attached
• Head mounted display/projector Body attached
• Handheld display/projector Spatial
• Spatially aligned projector/monitor
Display Taxonomy
Head Mounted Displays
Head Mounted Displays (HMD) - Display and Optics mounted on Head - May or may not fully occlude real world - Provide full-color images - Considerations
• Cumbersome to wear • Brightness • Low power consumption • Resolution limited • Cost is high?
Key Properties of HMD Field of View
Human eye 95 degrees horizontal, 60/70 degrees vertical
Resolution > 320x240 pixel
Refresh Rate Focus
Fixed/manual
Power Size
Types of Head Mounted Displays
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
The image cannot be displayed. Your computer
Occluded See-thru
Multiplexed
Immersive VR Architecture
Head!Tracker
Host !Processor
Data Base!Model
Rendering!Engine Frame!
Buffer
head position/orientation
to network Display!Driver
Non see-thru!Image source
& optics
virtual object
Virtual World
See-thru AR Architecture
Head!Tracker
Host !Processor
Data Base!Model
Rendering!Engine Frame!
Buffer
head position/orientation
to network Display!Driver
see-thru!combiner
Virtual Image superimposed!over real world object
real world
Image source
Optical see-through head-mounted display
Virtual images from monitors
Real World
Optical Combiners
Optical See-Through HMD
Epson Moverio Display
Stereo see-through display 960 x 540 pixels, 23 degree Field of View, 60 Hz $700 USD Android Powered
View Through Optical See-Through HMD
DigiLens
www.digilens.com
Compact HOE Solid state optics Switchable Bragg Grating Stacked SBG Fast switching Ultra compact
The Virtual Retinal Display
Image scanned onto retina Commercialized through Microvision
Nomad System - www.mvis.com
Strengths of optical see-through AR Simpler (cheaper) Direct view of real world
Full resolution, no time delay (for real world) Safety Lower distortion
No eye displacement but COASTAR video see-through avoids this
Video AR Architecture
Head!Tracker
Host !Processor
Graphics!renderer
Digital!Mixer Frame!
Buffer
head position/orientation
to network Display!Driver
Non see-thru!Image source
& optics
Head-mounted camera aligned to
display optics
Video!Processor
Video image of real world
Virtual image inset into video of real world
Video see-through HMD Video cameras
Monitors
Graphics
Combiner
Video
Video See-Through HMD
Video see-through HMD
MR Laboratory’s COASTAR HMD (Co-Optical Axis See-Through Augmented Reality) Parallax-free video see-through HMD
TriVisio www.trivisio.com Stereo video input
PAL resolution cameras
2 x SVGA displays 30 degree FOV User adjustable convergence
$6,000 USD
View Through a Video See-Through HMD
Vuzix Display
www.vuzix.com Wrap 920 $350 USD Twin 640 x 480 LCD displays 31 degree diagonal field of view Weighs less than three ounces
Strengths of Video See-Through AR True occlusion
Kiyokawa optical display that supports occlusion
Digitized image of real world Flexibility in composition Matchable time delays More registration, calibration strategies
Wide FOV is easier to support
Optical vs. Video AR Summary Both have proponents Video is more popular today?
Likely because lack of available optical products
Depends on application? Manufacturing: optical is cheaper Medical: video for calibration strategies
Eye multiplexed AR Architecture
Head!Tracker
Host !Processor
Data Base!Model
Rendering!Engine Frame!
Buffer
head position/orientation
to network Display!Driver
Virtual Image inset into!real world scene
real world
Opaque!Image source
Virtual Image ‘inset’ into real
Virtual Vision Personal Eyewear
Virtual image inset into real world
Google Glass
View Through Google Glass
What's Inside Google Glass?
Spatial/Projected AR
Spatial Augmented Reality
Project onto irregular surfaces Geometric Registration Projector blending, High dynamic range
Book: Bimber, Rasker “Spatial Augmented Reality”
Projector-based AR
Examples: Raskar, MIT Media Lab Inami, Tachi Lab, U. Tokyo
Projector
Real objects with retroreflective covering
User (possibly head-tracked)
Example of projector-based AR
Ramesh Raskar, UNC, MERL
Example of projector-based AR
Ramesh Raskar, UNC Chapel Hill
The I/O Bulb
Projector + Camera John Underkoffler, Hiroshi Ishii MIT Media Lab
Head Mounted Projector
Head Mounted Projector Jannick Rolland (UCF)
Retro-reflective Material Potentially portable
Head Mounted Projector
NVIS P-50 HMPD 1280x1024/eye Stereoscopic 50 degree FOV www.nvis.com
HMD vs. HMPD
Head Mounted Display Head Mounted Projected Display
Pico Projectors
Microvision - www.mvis.com 3M, Samsung, Philips, etc
MIT Sixth Sense
Body worn camera and projector http://www.pranavmistry.com/projects/sixthsense/
Other AR Displays
Video Monitor AR
Video cameras Monitor
Graphics Combiner
Video
Stereo glasses
Examples
Virtual Showcase
Mirrors on a projection table Head tracked stereo Up to 4 users Merges graphic and real objects Exhibit/museum applications
Fraunhofer Institute (2001) Bimber, Frohlich
Augmented Paleontology
Bimber et. al. IEEE Computer Sept. 2002
Alternate Displays
LCD Panel Laptop PDA
Handheld Displays Mobile Phones
Camera Display Input
Other Types of AR Display Audio
spatial sound ambient audio
Tactile physical sensation
Haptic virtual touch
Haptic Input
AR Haptic Workbench CSIRO 2003 – Adcock et. al.
Phantom
Sensable Technologies (www.sensable.com) 6 DOF Force Feedback Device
AR Haptic Interface
Phantom, ARToolKit, Magellan