Image-based Techniques for the Creation

and Display of Photorealistic Interactive Virtual Environments

George Drettakis1, Maria Roussou3,4, Nicolas Tsingos1, Alex Reche1,2, Emmanuel Gallo1,2

1REVES/INRIA, Sophia-Antipolis, France2CSTB, France

3University College London, UK4Foundation of the Hellenic World, Hellas (Greece)

overall goals Realistic virtual environments (VE’s)

– simplify model creation– enhance realism of display– interactivity

Education and design– enhance learning process– help design and decision making

previous approaches Limited level of realism in VE’s

– creation of environments slow, costly and with limited realism

– display of environments with low realism

missing shadows and vegetation, flat textures

missing sound no life (crowds, traffic etc.)

previous approaches Many developments in Computer

Graphics, but rarely transferred to VE’s– considered too expensive in resources– engineering aspect overwhelming

Examples of CG techniques– image-based modeling [Debevec 96/98]– point-based rendering [Stamminger 01]– perspective shadow maps [Stamminger 02]

overview of our approach Project context

– CREATE IST project Three components

– creating and displaying a realistic VE– interactivity– user-centered design and evaluation

data acquisition for CREATE

Two applications areas– cultural heritage and education– urban planning and design

Two corresponding sites– Ancient Messene in Greece

(archeology)– Place Massena & Garibaldi in Nice

(tramway project)

model creation Image-based modeling

– shoot panoramas, stitch and render cube maps

resulting model After calibration, modelling &

texturing

synthetic image

video Creation of model using Stitcher

and ImageModeler (improved versions)

view-dependent 3D models

One common model, multiple textures– use blending for view-dependent display

For complex scenes use layering– perform geometric sort to create image

layers– use a view-dependent texture per layer

view-dependent 3D models

Results of Enhanced VE’s

urban planning models

synthetic image

enhanced virtual environment

enhanced virtual environment

video Place Garibaldi simulation

– shadows– sound

interactivity Cultural heritage

– manipulation of architectural members

– reconstruction of temple Urban planning

– design and construct “urban features” such as benches or umbrellas

user-centered design and evaluation

User task analysis identified expert and novice users:– Cultural heritage (C/H)

experts: archeologists novice users: general public, children

– Urban planning (U/P) experts: architects novice users: general public, decision

makers

user task analysis Interviews with users, analysis of

existing task flow– augment the current tools– add realism and interactivity to

enhance productivity Preliminary studies

– cultural heritage with children– preparation for urban case

preliminary formative studies

Children in a 3-task experiment (C/H)– 7 to 12 years old– CAVE environment

Preparation of usability evaluation with architects (U/P)– direct observation– description of needs– scenario-based evaluation

current work Completion of enhanced VE tools

– more realistic trees– integration of crowds and sound

Real-task experiments– architects in a design task for urban

planning (under way). Will be evaluated for effectiveness, efficiency, ease of use, and satisfaction.

video Interactivity

– design and placement of benches and umbrellas

conclusions Image based techniques

– improve realism of VE’s improved capture & display

– combined with interactivity and user-centered design show promise for improved usage of VE’s

acknowledgements CREATE is a 3-year RTD project funded by the

EU’s FP5 IST Programme (IST-2001-34231), http://www.cs.ucl.ac.uk/create. The partners of CREATE are University College London (UK), Foundation of the Hellenic World (GR), INRIA (FR), REALVIZ (FR), University of Cyprus (CY), PERCRO (I) and CSTB (FR)

Alias for their donation of Maya Alexandre Olivier-Mangon for all the image-

modeling and virtual modeling, and the videos Alex Reche and Emmanuel Gallo are supported

by CIFRE doctoral fellowships INRIA-CSTB

thank youhttp://www-sop.inria.fr/reves

http://www.cstb.frhttp://www.cs.ucl.ac.uk/create