426 lecture3: ar tracking
DESCRIPTION
COSC 426 Graduate class in Augmented Reality, lecture on AR tracking. Taught by Mark Billinghurst of the HIT Lab NZ at the University of Canterbury, July 25th 2012TRANSCRIPT
COSC 426: Augmented Reality
Mark Billinghurst
July 25th 2012
Lecture 3: AR Tracking
Tracking Requirements
Augmented Reality Information Display World Stabilized Body Stabilized Head Stabilized
Increasing Tracking Requirements
Head Stabilized Body Stabilized World Stabilized
Tracking Technologies
• Mechanical • Electromagnetic • Optical • Acoustic • Inertial and dead reckoning • GPS • Hybrid
AR Tracking Taxonomy
e.g. AR Toolkit
Low Accuracy at 15-60 Hz
e.g. IVRD
High Accuracy & High Speed
Hybrid Tracking
Limited Range
e.g. HiBall
Many Fiducials in space/time
but no GPS
Extended Range
Indoor Environment
e.g. WLVA
Not Hybridized GPS or
Camera or Compass
Low Accuracy & Not Robust
e.g. BARS
Hybrid Tracking GPS and
Camera and Compass
High Accuracy & Robust
Outdoor Environment
AR TRACKING
Tracking Types
Magnetic Tracker
Inertial Tracker
Ultrasonic Tracker
Optical Tracker
Marker-Based Tracking
Markerless Tracking
Specialized Tracking
Edge-Based Tracking
Template-Based Tracking
Interest Point Tracking
Mechanical Tracker
Tracking Systems Mechanical Tracker Magnetic Tracker Ultrasonic Tracker Inertial Tracker Computer Vision (Optical Tracking)
Specialized (Infrared, Retro-Reflective) Monocular (DVCam, Webcam)
Mechanical Tracker Idea: mechanical arms with joint sensors
++: high accuracy, haptic feedback -- : cumbersome, expensive
Microscribe
Magnetic Tracker Idea: difference between a magnetic transmitter
and a receiver
++: 6DOF, robust -- : wired, sensible to metal, noisy, expensive
Flock of Birds (Ascension)
Magnetic Tracking Error
Ultrasonics Tracker Idea: Time of Flight or Phase-Coherence Sound Waves
++: Small, Cheap -- : 3DOF, Line of Sight, Low resolution, Affected
Environment Conditon (pressure, temperature)
Ultrasonic Logitech IS600
Inertial Tracker Idea: measuring linear and angular orientation rates
(accelerometer/gyroscope)
++: no transmitter, cheap, small, high frequency, wireless -- : drift, hysteris only 3DOF
IS300 (Intersense) Wii Remote
Mobile Sensors Inertial compass
Earth’s magnetic field Measures absolute orientation
Accelerometers Measures acceleration about axis Used for tilt, relative rotation Can drift over time
Global Positioning System (GPS)
Created by US in 1978 Currently 29 satellites
Satellites send position + time GPS Receiver positioning
4 satellites need to be visible Differential time of arrival Triangulation
Accuracy 5-30m+, blocked by weather, buildings etc
Problems with GPS Takes time to get satellite fix
Satellites moving around
Earths atmosphere affects signal Assumes consistent speed (the speed of light). Delay depends where you are on Earth Weather effects
Signal reflection Multi-path reflection off buildings
Signal blocking Trees, buildings, mountains
Satellites send out bad data Misreport their own position
Accurate to < 5cm close to base station (22m/100 km) Expensive - $20-40,000 USD
Assisted-GPS (A-GPS) Use external location server to send GPS signal
GPS receivers on cell towers, etc Sends precise satellite position (Ephemeris)
Speeds up GPS Tracking Makes it faster to search for satellites Provides navigation data (don’t decode on phone)
Other benefits Provides support for indoor positioning Can use cheaper GPS hardware Uses less battery power on device
Assisted GPS
Cell Tower Triangulation Calculate phone position
from signal strength < 50 m in cities > 1 km in rural
WiFi Positioning Estimate location by using WiFi access points
Can use know locations of WiFi access points Triangulate through signal strength
Eg. PlaceEngine (www.placeengine.com) Client software for PC and mobiles SDK returns position
Accuracy 5 – 100m (depends on WiFi density)
WiFi Hotspots in New York
Indoor WiFi Location Sensing Indoor Location
Asset, people tracking
Aeroscout http://aeroscout.com/ WiFi + RFID
Ekahau http://www.ekahau.com/ WiFi + LED tracking
Integrated Systems Combine GPS, Cell tower, WiFi signals Skyhook (www.skyhookwireless.com)
Core Engine
Database of known locations 700 million Wi-Fi access points and cellular towers.
Comparative Accuracies Study testing iPhone 3GS cf. low cost GPS A-GPS
8 m error WiFi
74 m error
Cell Tower Positioning 600 m error
Accuracy of iPhone Locations: A Comparison of Assisted GPS, WiFi, and Cellular Positioning
In GIScience on July 15, 2009 at 8:11 pm By Paul A Zandbergen Transactions in GIS, Volume 13 Issue s1, Pages 5 - 25
Optical Tracking
Optical Tracker Idea: Image Processing and Computer Vision Specialized
Infrared, Retro-Reflective, Stereoscopic
Monocular Based Vision Tracking
ART Hi-Ball
Outside-In vs. Inside-Out Tracking
Optical Tracking Technologies
Scalable active trackers InterSense IS-900, 3rd Tech HiBall
Passive optical computer vision Line of sight, may require landmarks Can be brittle. Computer vision is computationally-intensive
3rd Tech, Inc.
HiBall Tracking System (3rd Tech) Inside-Out Tracker
$50K USD
Scalable over large area Fast update (2000Hz) Latency Less than 1 ms.
Accurate Position 0.4mm RMS Orientation 0.02° RMS
Starting simple: Marker tracking Has been done for more than 10 years Several open source solutions exist Fairly simple to implement
Standard computer vision methods
A rectangular marker provides 4 corner points Enough for pose estimation!
Marker Based Tracking: ARToolKit
http://artoolkit.sourceforge.net/
Coordinate Systems
Tracking Range with Pattern Size
Rule of thumb – range = 10 x pattern width
Tracking Error with Range
Tracking Error with Angle
Tracking challenges in ARToolKit
False positives and inter-marker confusion (image by M. Fiala)
Image noise (e.g. poor lens, block coding /
compression, neon tube)
Unfocused camera, motion blur
Dark/unevenly lit scene, vignetting
Jittering (Photoshop illustration)
Occlusion (image by M. Fiala)
Limitations of ARToolKit Partial occlusions cause tracking failure Affected by lighting and shadows Tracking range depends on marker size Performance depends on number of markers
cf artTag, ARToolKitPlus Pose accuracy depends on distance to marker Pose accuracy depends on angle to marker
Tracking, Tracking, Tracking
Other Marker Tracking Libraries arTag
http://www.artag.net/
ARToolKitPlus [Discontinued] http://studierstube.icg.tu-graz.ac.at/handheld_ar/
artoolkitplus.php stbTracker
http://studierstube.icg.tu-graz.ac.at/handheld_ar/stbtracker.php
MXRToolKit http://sourceforge.net/projects/mxrtoolkit/
Markerless Tracking
Markerless Tracking
Magnetic Tracker Inertial Tracker
Ultrasonic Tracker
Optical Tracker
Marker-Based Tracking
Markerless Tracking
Specialized Tracking
Edge-Based Tracking
Template-Based Tracking
Interest Point Tracking
No more Markers! Markerless Tracking
Natural feature tracking Tracking from features of the surrounding
environment Corners, edges, blobs, ...
Generally more difficult than marker tracking Markers are designed for their purpose The natural environment is not…
Less well-established methods Usually much slower than marker tracking
Natural Feature Tracking Use Natural Cues of Real Elements
Edges Surface Texture Interest Points
Model or Model-Free ++: no visual pollution
Contours
Features Points
Surfaces
Texture Tracking
Edge Based Tracking RAPiD [Drummond et al. 02]
Initialization, Control Points, Pose Prediction (Global Method)
Line Based Tracking Visual Servoing [Comport et al. 2004]
Model Based Tracking Track from 3D model Eg OpenTL - www.opentl.org
General purpose library for model based visual tracking
Marker vs. natural feature tracking Marker tracking
+ Can require no image database to be stored + Markers can be an eye-catcher + Tracking is less demanding - The environment must be instrumented with markers - Markers usually work only when fully in view
Natural feature tracking - A database of keypoints must be stored/downloaded + Natural feature targets might catch the attention less + Natural feature targets are potentially everywhere + Natural feature targets work also if partially in view
Hybrid Tracking
Hybrid Tracking Combining several tracking modalities together
Sensor tracking Used by many “AR browsers” GPS, Compass, Accelerometer, (Gyroscope) Not sufficient alone (drift, interference)
Outdoor Hybrid Tracking Combines
computer vision - natural feature tracking
inertial gyroscope sensors Both correct for each other
Inertial gyro - provides frame to frame prediction of camera orientation
Computer vision - correct for gyro drift
Combining Sensors and Vision Sensors
- Produce noisy output (= jittering augmentations) - Are not sufficiently accurate (= wrongly placed augmentations) - Gives us first information on where we are in the world,
and what we are looking at Vision
- Is more accurate (= stable and correct augmentations) - Requires choosing the correct keypoint database to track from - Requires registering our local coordinate frame (online-
generated model) to the global one (world)
Outdoor AR Tracking System
You, Neumann, Azuma outdoor AR system (1999)
Robust Outdoor Tracking
Hybrid Tracking Computer Vision, GPS, inertial
Going Out Reitmayer & Drummond (Univ. Cambridge)
Handheld Display
Wrap-up Tracking and Registration are key problems Registration error
Measures against static error Measures against dynamic error
AR typically requires multiple tracking technologies Research Areas: Hybrid Markerless Techniques,
Deformable Surface, Mobile, Outdoors
