magic book: enhancing natural feature tracking with the user´s movement context
DESCRIPTION
Magic Book: Enhancing Natural Feature Tracking with the User´s Movement Context. Felix Loew Master Thesis (TU München). 21.01.2005. Outline. Context-Awareness and Augmented Reality Vision Based Tracking: Natural Feature Tracking A „Magic Book“ User Study Software Architecture (DWARF ) - PowerPoint PPT PresentationTRANSCRIPT
Magic Book: Enhancing Natural Feature Tracking with the User´s Movement Context
Felix Loew
Master Thesis (TU München)
21.01.2005
• Context-Awareness and Augmented Reality
• Vision Based Tracking: Natural Feature Tracking
• A „Magic Book“ User Study
• Software Architecture (DWARF )
• Expected results and conclusions
Outline
Context-Awareness and Augmented Reality
• A system has to adopt to the user´s behaviour!
• A system has to learn how it is used!
• New intuitive methods of interaction are required!
• The system has to disappear (Marc Weiser)!
The „Magic Book“ in this context
• User has to „learn“ how it is used
• Very error-prone to movement– Movement of the handheld device is not
considered in the tracking routine– Vision based tracking loses orientation– User has to reinitialize the tracking (looking on
the marker again)– Immersion decreases!
My Idea: Enhancing the Magic Book
Inertial Tracker: Intersense, Inertial Cube3 (~120FPS)
Vision Based Tracking: NFT (~20 FPS)
Movement
• Context-Awareness and Augmented Reality
• Vision Based Tracking: Natural Feature Tracking
• A „Magic Book“ User Study
• Software Architecture (DWARF )
• Summary, Conclusions & Limitations
Outline
Basics: Natural Feature Tracking
Frame 1 Frame 2
Frame 3 Frame 4
• NFTARToolkit (Kato): 2D planar scenes• Estimation of feature position in next frame
– It is not realistic that no movement occurs– Linear estimation– Kalman Filter
• Within a window the “best match” with the template is searched (Template Matching)
• For every pixel within window the similarity with the template is calculated (Normalized Cross Correlation)
Basics: Natural Feature Tracking
Template Matching
• Calculates similarity between template and area around search point– values [-1..1]
• Best match is considered as feature point in current frame (threshold ~0.7)!
• Computational expensive!– O (templateSize2 * searchSize2)
Normalized Cross Correlation
My approach
• Consider movement as well!
• Use additional tracking device (Inertial Tracker / 3DOF)
• Naive approach:much movement large search window
low movement small search window
• Dynamic configuration during runtime
What is much movement? Which movements actually occur?How does a user use such an application?
USER STUDY ON MAGIC BOOK
My approach
• Context-Awareness and Augmented Reality
• Vision Based Tracking: Natural Feature Tracking
• A „Magic Book“ User Study
• Software Architecture (DWARF )
• Expected results and conclusions
Outline
• Goals– Detecting specific movement patterns– Derive Movement model– Estimation of a correlation between changes in
orientation and feature points coordinates
User Study Setup
Orientation (Handheld, 3D)
Position (FP-Coords, 2D)
User Study Setup
• Tracking Orientation and Position (6DOF)– Magnetic Tracker:
Flock of Birds
– Bird1: Magic Book
– Bird2: Handheld device
– Calibration to Magic Book coordinate system (Bird1)
• Magnetic Tracker– Timestamp
– Position P = (px,py,pz)
– Orientation (quaternion) Q = (v,s)
• NFT– Timestamp
– 4 feature points per frame• Id
• Screen-Position S = (sx,sy)
Logging
• Magnetic Tracker
pics\2_20_1.Case2.Flock.xls
• NFT
pics\2_20_1.Case2.Coords.xls
Logging
User Tasks
• Comparable Tasks (Navigation Tasks)– User has to answer certain questions (Overview,
Focus and Search Tasks)• „What is the girls hair color?“
• „How many people do you see?“
• Free Tasks– User can use Magic Book freely
Evaluation
• Ideas for evaluation (Correlation)– Which angular offset (FOB) causes which
coordinate offset (NFT)?– Which angular velocities occur?– Which rotations occur?– Which changes of position occur?– Is there a common movement profile or are
there serious differences between different users?
Evaluation
• Find mapping between:
• Configure Software Architecture with this information
orientation search window
• Context-Awareness and Augmented Reality
• Vision Based Tracking: Natural Feature Tracking
• A „Magic Book“ User Study
• Software Architecture (DWARF)
• Expected results and conclusions
Outline
Software Architecture (DWARF)
NFT ARToolkit
Dynamic Configuration
• Context-Awareness and Augmented Reality
• Vision Based Tracking: Natural Feature Tracking
• A „Magic Book“ User Study
• Software Architecture (DWARF)
• Expected results and conclusions
Outline
Conclusions & Summary
• Hybrid Tracking approach to consider context information as well– Movement context– 2DO: What about user context?
• User Study on Magic Book (work in progress)– Expected conclusions on usage of Magic Book– Find correlation between movement and NFT properties– Process a configuration out of the study– Derive Movement model of the Magic Book– Limitation: Very task specific (features are not moving, only 2D)!
• Software Architecture
• Heaps of work– I won’t cover everything I want to do
• Other “sensor-fusion” approaches might be more suited (Kalman-Filtering)
• Evaluation of User Study will be hard
• Is there an abstraction for NFT based applications?
Limitations & 2DO´s
Thanks
• I would be glad if some of you want to join the study!