Creating 3D Projection on Tangible Objects

Pekka Nisula*, Jussi Kangasoja and Minna Karukka Oulu University of Applied Sciences

ROTATING GLOBE DEMONSTRATORProjected multimedia installations or learning material are part of everyday life in museums, exhibitions and schools. Earlier research has introduced relatively much work on projection systems, where images are shown on arbitrary surfaces, see e.g. [1, 2]. We present our on-going work to take a step further in creating more immersive and engaging multimedia presentations, and change from conventional 2D media to 3D media by projecting the visual content on a non-planar physical object – a multi-projector set-up, which we aim to use for exhibition, teaching, design and early prototyping purposes.

In the demonstration described in this paper we use non-planar surface, a sphere shaped object and project a visualization of rotating Earth on it. Earlier research has experimented with sphere shaped interactive displays. At Microsoft, Benko et al. built and experimented with a spherical multi touch screen Sphere [3], and in [4], a spherical volumetric display with touch input is described. Our demonstration does not currently have input functionality, but it is considered as the potential direction for future work. Bimber and Raskar [5] presents techniques involving both hardware and software to implement spatial augmented reality (SAR) installations.

This demonstration presents a tangible 3D projection that we aim to develop and use for exhibition and teaching purposes. In our demonstration we used a sphere shaped object and projected a visualization of rotating Earth on it. Our 3D setup contains five projectors, four in the sides of the projection surface and one is at the top of the surface as seen in figure 1.

All projectors are 4 feet away from the sphere and the whole set-up takes 10 x 10 feet space. Four of the projectors are at 90-degree angles from each other surrounding the sphere and one projector is mounted 4 feet above the sphere.

* e-mail: pekka.nisula@oamk.fi

The simulation in the virtual 3D world contains five cameras according the physical world setup, and each camera represents a projector. The virtual 3D world contains a 3D model of the sphere, which has textures from satellite images. It animates clouds as well as the change of the night and day as seen in figure 2. Our setup is made using open source libraries such as Open Scene Graph and Projection Designer.

We have done a study where we compared two different set-ups, one presented animated earth projected on a 2D display and another one was our 3D projection of earth [6]. The results show that the 3D installation was found more interesting and engaging than 2D presentation. The results were encouraging, and we will continue our work with the subject.

Figure 1. First rotating globe demonstrator set-up.

Figure 2. Textures projected on sphere shaped physical object

187

IEEE Virtual Reality 201316 - 20 March, Orlando, FL, USA978-1-4673-4796-9/13/$31.00 ©2013 IEEE

In the future we will focus on enabling functional interactions and automatic projector calibrations to make projections more useful and interesting e.g. for teaching purposes.

ACKNOWLEDGEMENTSThe authors wish to thank Jonna Häkkilä, Center for Internet Excellence (CIE), Oulu University.

REFERENCES[1] Raskar, R., van Baar, J., Beardsley, P., Willwacher, T., Rao, S.,

Forlines, C. iLamps: Geometrically Aware and Self-Configuring Projectors, SIGGRAPH 2003, ACM, 809-818

[2] Sun, W. Yang, X., Xiao, S., Hu, W. Robust Checkerboard Recognition for Efficient Nonplanar Geometry Registration in Projector-camera Systems. In Proc. PROCAMS 2008, ACM 2008.

[3] Benko, H., Wilson, A. D. Balakrishnan, R. Sphere: multi-touch interactions on a spherical display. In Proc. UIST 2008

[4] Grossman, T., Wigdor, D., Balakrishnan, R. Multi-finger gestural interaction with 3d volumetric displays. In Proc. UIST’04, 61-70, 2004.

[5] Bimber, O., Raskar R. Spatial Augmented Reality : Merging Real and Virtual Worlds (2005). p. 7, 213-216. A K Peters Ltd., MA, USA. ISBN 1-56881-230-2

[6] Karukka, M., Nisula P., Häkkilä, J., & Kangasoja, J. Charting the Audience Perceptions of Projected 3D Media Installations. Poster in Proc. MUM2012, 3-6. December 2012, Ulm, Germany.

Creating 3D Projection on Tangible Objects

Pekka Nisula*, Jussi Kangasoja and Minna Karukka Oulu University of Applied Sciences

DEMONSTRATOR SETUP

Fig. 1: Illustration of Rotating Globe Demonstrator set-up from above

18

Fig. 2: Side-view illustration of Rotating Globe Demonstrator

Mixed Reality Research Lab

GENERAL INFORMATION

Oulu University of Applied Sciences (OUAS), one of the largest universities of applied sciences in Finland with approximately 9,000 students, operates in one of the fastest growing areas of the country - the Oulu region. The focus areas of research and development are: business and marketing competence and development of services supporting business, strong internationalisation, well-functioning and effective innovation system and processes and an innovative and internationally competitive operating environment

Use of the Internet and& ICT is playing a big role in almost all business areas. We focus on new ways to utilize Internet & ICT, such as 3D technologies, Virtual and Augmented Reality, Stereoscopic 3D,

collaborative production models or internet as a brand new business ecosystem and distribution channel. Emerging and available technologies together with open-minded people and deep understanding of business opportunities create services and experiences that are far beyond our current imagination. OUAS provides world-class excellence and welcomes new openings and collaboration possibilities.

For the Internet & ICT we execute currently R&D projects with the total budget of 5,5M€ whereas average project length is 24 months and there are around 11 simultaneously ongoing projects on several specialiation fields: i) Utilization of 3D in products and services, ii) Living Lab –activities and iii) Internet Content Production

RESEARCH LAB

18

Mixed Reality is an extensive research, development and innovation initiative, which facilitates emerging MR technologies, practical needs from regional companies and business knowledge by developing new service concepts and application demos. It provides new possibilities and activities for the labor markets by evaluating existing technologies, services and opportunities and developing new service concepts, application demos and information systems.

One of the key objectives of the initiative is to provide information and knowledge of the availability, applicability and effectiveness of MR technologies in different types of application and operating environments.

Biggest set-up constructed by Mixed Reality initiative is Seamless Design Space (SDS)1, CAVE (Cave Automatic Virtual Environment) -alike immersive virtual environment (IVE) set-up, in order to create a seamless design space for 3D user experiences. The user experience goes beyond the utilitarian aspects, and we wish to examine the balance between different factors that affect the holistic experience. In our approach, the Seamless Design Space combines means for high-quality immersion and presentation. This set-up allows us to investigate different

1 http://cave.oamk.fi

components of 3D content creation, 3D UX in narratives and storytelling, as well as aesthetics and engagement when interacting in immersive 3D environments.

Mixed Reality research team was founded in early 2012 as former Research Leader of Nokia Research Center (NRC) Minna Karukka joined OUAS. The team consists of a half dozen researchers with 3D and UX as a common interest for all.

KEY PERSONNEL

Jussi Kangasoja, Master of Engineering in Wireless Technologies and B.Sc in Telecommunications by education is 3D specialist and he has worked for the OUAS, School of Engineering, since 2004 in a number of demanding research and development initiatives such as eOTE-OAMK, Neo Arena, Virtual Campus for Digital Students (ViCaDiS), ICDNET and UbiGO with the work profile of a technical expert, consult and project coordinator. He has also been in charge of research and consulting tasks of new mobile service concepts, as well as project planning at national (ERDF, ESR) and international (FP, ITEA) level. Currently he is project manager in Mixed Reality.

Pekka Nisula, Bachelor of Science in Information Technology and telecommunications by education is software developer and software architect designer. He is currently working at MotiMind (startup company) as Chief Architect and OUAS, School of Engineering as Chief Designer. He has been worked for OUAS since 2009 in UbiGO and Mixed Reality projects.

1

Minna Karukka, Master of Arts from the University of Lapland as well as a B.Sc. from the Tampere University of Applied Sciences. She is co-founder of Soul4Design (private company) and former Research Leader at Nokia Research Center (NRC). Currently she is working as a project designer at Oulu University of Applied Sciences School of Music, Dance and Media. More than 10 years of experience in working with mobile phone industry, especially on the area of user experience, user centric design and usability. Also participated to multiple research projects. Most of research activities relate to media domain, for example, media consumption, social applications and 3D. Further credits include acting as an entrepreneur.

1