charles woodward: mobile mixed reality in diem
DESCRIPTION
TIVIT Results and Business Forum 12 April 2011 Charles Woodward, VTT: Mobile Mixed Reality in DIEMTRANSCRIPT
![Page 1: Charles Woodward: Mobile Mixed Reality in DIEM](https://reader034.vdocuments.site/reader034/viewer/2022052321/54bd526e4a79599e3c8b4571/html5/thumbnails/1.jpg)
Mobile Mixed Reality in DIEM
Tivit Results and Business Seminar12.4.2011
Charles WoodwardVTT
![Page 2: Charles Woodward: Mobile Mixed Reality in DIEM](https://reader034.vdocuments.site/reader034/viewer/2022052321/54bd526e4a79599e3c8b4571/html5/thumbnails/2.jpg)
• General goals– Location based <-> around shopping mall, or city– Close range <-> inside shop, products
Tracking for Mobile Mixed Reality
![Page 3: Charles Woodward: Mobile Mixed Reality in DIEM](https://reader034.vdocuments.site/reader034/viewer/2022052321/54bd526e4a79599e3c8b4571/html5/thumbnails/3.jpg)
Nokia CityScene
![Page 4: Charles Woodward: Mobile Mixed Reality in DIEM](https://reader034.vdocuments.site/reader034/viewer/2022052321/54bd526e4a79599e3c8b4571/html5/thumbnails/4.jpg)
Nokia CityScene (contd.)
![Page 5: Charles Woodward: Mobile Mixed Reality in DIEM](https://reader034.vdocuments.site/reader034/viewer/2022052321/54bd526e4a79599e3c8b4571/html5/thumbnails/5.jpg)
• Department of Media Technology– Research on real-time camera tracking,
joint ISMAR tracking contest entry with VTT– Implemented generic 3D reconstruction
software package– Research on fast and scalable geometric
computer vision algorithms
• Department of Information and Computer Science
– Research on contextual information retrieval and multimodal relevance feedback
– Developed methods for face, object and context recognition
– Studied selection of relevant information based on current context and use history
– Implementation of the second generation platform of the Virtual Laboratory Guide on N900 mobile device
Research at Aalto
![Page 6: Charles Woodward: Mobile Mixed Reality in DIEM](https://reader034.vdocuments.site/reader034/viewer/2022052321/54bd526e4a79599e3c8b4571/html5/thumbnails/6.jpg)
• Annual academic and industrial tracking competition.– A simulated storage room with many objects and tricky features– Reference points (and their global coordinates) are provided– Calibration phase can be done e.g. on a previous day– On contest run, participant will receive the list of object coordinates– The user has to pick the objects in the given order
• Participation by VTT and Aalto's joint solution– Five competitors (incl. Fraunhofer, Metaio, ...)– We had the only mobile phone implementation (1st ever)– We managed to identify all items (results/accuracy unknown)
ISMAR 2010 Tracking Competition
![Page 7: Charles Woodward: Mobile Mixed Reality in DIEM](https://reader034.vdocuments.site/reader034/viewer/2022052321/54bd526e4a79599e3c8b4571/html5/thumbnails/7.jpg)
TrackingServer
ReconstructionServer 1
ReconstructionServer 2
…
ROS
ImageDatabase
– Client sends an image– Server replys with
6-DOF pose and 2D-3D point pairs
– Server can use the imageto improve databases and reconstructions
• Still required– Local recovery based on
feature descriptors– Query server for more
information while tracking– Possibly other sensors
UI for data manipulation
Tracking Solution for ISMAR 2011
• Overview– Server provides global
initialization and recovery
– Client tracks locally
![Page 8: Charles Woodward: Mobile Mixed Reality in DIEM](https://reader034.vdocuments.site/reader034/viewer/2022052321/54bd526e4a79599e3c8b4571/html5/thumbnails/8.jpg)
Mobile Outdoors Tracking
• VTT's background work at AR4BC project– Augmented Reality for Building and Construction– Augmented visualization of 4D Building Information Models
• Now implemented for mobile phone– N900, client/server solution– RIFF algorithm by NRC and Stanford– Speed ups and accuracy improvements by VTT
![Page 9: Charles Woodward: Mobile Mixed Reality in DIEM](https://reader034.vdocuments.site/reader034/viewer/2022052321/54bd526e4a79599e3c8b4571/html5/thumbnails/9.jpg)
Photorealistic Rendering for Augmented Reality
• Challenges, e.g.– Materials, incl. reflections,
transparency– Lighting: direction, intensity,
colors, soft shadows– Accounting for real world lighting
conditions– Matching computer graphics with
webcam video quality
• Comparisons– Left - standard 3D graphics– Right - using VTT methods
![Page 10: Charles Woodward: Mobile Mixed Reality in DIEM](https://reader034.vdocuments.site/reader034/viewer/2022052321/54bd526e4a79599e3c8b4571/html5/thumbnails/10.jpg)
ALVAR Subroutine Library
• "A Library for Virtual and Augmented Reality”– Download at www.vtt.fi/multimedia/alvar.html
• Modules– Basic / Pro -> Desktop (Windows / Linux) – Mobile (Symbian & Maemo, iPhone, …)– Render (Windows, ...)
• Tracking features– Accurate & fast marker pose estimation, error
concealment; marker fields, ; optical flow tracking, hiding of markers, etc.
– Markerless tracking: based on 2D templates, or 3D features ("SLAM"); sensor fusion
• Users– OEM licensing e.g. by Vuzix (data glasses)– Columbia Univ. (Goblin XNA / Microsoft)– MIT (6th Sense), hundreds of others
![Page 11: Charles Woodward: Mobile Mixed Reality in DIEM](https://reader034.vdocuments.site/reader034/viewer/2022052321/54bd526e4a79599e3c8b4571/html5/thumbnails/11.jpg)
MMR – Current & Next Steps @ TUT/IHTE
• Online survey for people using existing AR applications on iPhone or Android (ongoing)
• E.g. Layar, WikiTude, Junaio, WhereMark, TattooAR, StickyBits, MagiCam …• Focus: benefits & usefulness of AR, user experience, UI & functionality
issues• Targeting several hundred respondents
• Pilot study of the NRC’s CityScene panorama-based mixed reality client (Summer 2011)
• A 2-week study in U.S. with various user groups creating content on the mixed reality panoramas and reporting their experiences
• In collaboration with VTT & Aalto, developing an AR pilot in verkkokauppa.com shop in Ruoholahti (Spring 2011)
• Goal: to concept and implement a functional MAR service during 3rd project period of DIEM
• Concepting design implementation user experience evaluation with end-users
11
TUT / Human-centered technology 4/11/2011
![Page 12: Charles Woodward: Mobile Mixed Reality in DIEM](https://reader034.vdocuments.site/reader034/viewer/2022052321/54bd526e4a79599e3c8b4571/html5/thumbnails/12.jpg)
Contact:
Prof. Charles WoodwardVTT Technical Research Centre of Finland
Vuorimiehentie 3, Espoo, FinlandTel: +358 40 500 1514
[email protected]://www.vtt.fi/multimedia