Virtual Reality to Go:

A USC ICT Mixed Reality Lab Demonstration

David M. Krum∗

USC Institute forCreative Technologies

Evan A. Suma†

USC Institute forCreative Technologies

Mark Bolas‡

USC School of Cinematic ArtsUSC Institute for

Creative Technologies

ABSTRACT

Our demonstration will exhibit a number of low cost virtual realitysystems built using smartphones, inexpensive optics, and game en-gine software. These systems will demonstrate how VR researchersand developers can leverage current trends in commodity hardwareand software to bring virtual reality to a wider audience. Thisdemonstration dovetails with the Workshop on Off-the-Shelf Vir-tual Reality (OTSVR), which we will also be organizing at the VRconference.

The Mixed Reality Lab at the USC Institute for Creative Tech-nologies works to advance the design and practice of mixed realityand virtual reality. The lab is particularly focused on the researchand development of new technologies and techniques to enhanceimmersion and interaction for learning and training.

Index Terms: H.5.1 [Information Interfaces and Presentation(I.7)]: Multimedia Information Systems—Artificial, augmented,and virtual realities B.4.2 [Input/Output and Data Communica-tions]: Input/Output Devices—Image display

1 DEMONSTRATION

Early virtual reality systems required substantial technical and fi-nancial resources due to the limited computing capability of thetime. Accordingly, image generation, tracking, and content creationwere costly and difficult. Moore’s Law and other trends in gamingand mobile devices have dramatically changed the landscapes ofcomputing and virtual reality.

The Mixed Reality Lab at the University of Southern California’sInstitute for Creative Technologies (USC ICT) has embarked on anumber of projects which appropriate off-the-shelf technologies tolower the cost of virtual reality systems. Examples of our work inthis area includes a gesture control toolkit that utilizes the Kinectdepth camera [7] and a smartphone based head mounted displayusing LEEP optics [4] (see Figure 4).

We have continued to develop a number of low cost virtual re-ality systems that utilize commodity hardware and software, suchas smartphones, inexpensive 3D stereoscopic optics, and commod-ity game engines. This has yielded systems such as a smartphonebased virtual reality system using Hasbro My3D optics [1].

At Virtual Reality 2012, we propose to demonstrate several ofthese systems using both Google Android and Apple iOS baseddevices, in conjuction with our workshop on Off-the-Shelf VirtualReality (OTSVR). These devices utilize off-the-shelf optics, likethe Hasbro my3D add-on for Apple iOS devices, and inexpensive,custom optics designed by our lab, for Android devices.

We have created a number of immersive training games anddemonstration environments using the Unity game engine which

∗e-mail:krum@ict.usc.edu†e-mail:suma@ict.usc.edu‡e-mail:bolas@ict.usc.edu

Figure 1: A commodity smartphone based virtual reality system us-ing Hasbro my3D optics.

Figure 2: Screen images from the smartphone based virtual realitysystem displaying stereo pairs from a memory training game.

showcase the capabilities of these smartphone based virtual realitysystems. These include a memory game that tests a user’s ability toobserve changes in an environment (see Figure 2) and 360 degreestereoscopic photo panoramas (see Figure 3). Users interact withthese virtual scenes using head orientation and position (sensed ei-ther by built-in gyroscopes or external motion capture systems).Bluetooth keyboards allow additional navigation and object selec-tion capabilities.

Visitors to our demo will have the opportunity to try our differentlow cost systems and learn about how they were constructed. Wewill have software and technical specifications available for down-load. We plan to have folding optical kits available in envelopes andsoftware available for download so that researchers and developerscan immediately try our demos with their own phones and developtheir own applications.

Figure 3: A complete 360 degree stereoscopic photo panorama canbe created for smartphone based virtual reality systems. The panora-mas are processed from stereo photo pairs taken with fisheye lenses.

179

IEEE Virtual Reality 20124-8 March, Orange County, CA, USA978-1-4673-1246-2/12/$31.00 ©2012 IEEE

Figure 4: A low cost, experimental head mounted display with LEEPlenses in place and removed, revealing the two smartphones thatrender and display imagery.

2 THE ICT MIXED REALITY LAB

The Mixed Reality Lab (http://mxrlab.com) at the USC Institute forCreative Technologies develops new techniques and technologiesto advance immersive learning and training. The lab is led by Pro-fessor Mark Bolas, the Associate Director for Immersion at ICTand an Associate Professor of Interactive Media at the USC Schoolof Cinematic Arts. Professor Bolas is an early pioneer of virtualreality. His thesis work “Design and Virtual Environments” wasamong the first efforts to map the breadth of virtual reality as a newmedium and led toward a basic model for immersive experiencedesign. He co-founded Fakespace Inc. which pioneered many tech-nologies used by the VR and scientific visualization communities.

Figure 5: A head mounted projector developed by the lab which pro-vides personal perspective correct imagery on retroreflective screenswithout bulky optics in front of the user’s eyes.

The Mixed Reality Lab has developed and contributed to the de-sign of several innovative displays included head mounted projec-tors (see Figure 5) [3], the lightfield display [2], and the Wide5 headmounted display (see Figure 6), which can provide a horizontalfield of view of approximately 150 degrees. Other research projectsat the Mixed Reality Lab include investigations of redirected walk-ing [5, 6], displays for virtual humans, and audio.

The Mixed Reality Lab is part of the USC Institute for Cre-ative Technologies. The ICT is a university affliated research center(UARC), founded in 1999 by the University of Southern Californiawith a multi-year contract from the US Army. The institute bringstogether academia and the creative talents of Hollywood to advancedigital media for learning, simulation, and training. The goal is tocreate synthetic experiences so compelling that participants react asif they are real.

REFERENCES

[1] P. Hoberman, D. M. Krum, E. A. Suma, and M. Bolas. Immersive

training games for smartphone-based head mounted displays. In IEEE

Virtual Reality, 2012.

[2] A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec. Ren-

dering for an interactive 360 degree light field display. ACM Trans.

Graph., 26, July 2007.

[3] D. Krum, E. Suma, and M. Bolas. Augmented reality using personal

projection and retroreflection. Personal and Ubiquitous Computing,

16(1):17–26, 2012. 10.1007/s00779-011-0374-4.

Figure 6: The Wide5 head mounted display provides a horizontalfield of view of about 150 degrees. It is used in studies of redirectedwalking and virtual human interaction.

[4] J. Olson, D. Krum, E. Suma, and M. Bolas. A design for a smartphone-

based head mounted display. In IEEE Virtual Reality, pages 233–234,

March 2011.

[5] E. A. Suma, S. Clark, S. Finkelstein, Z. Wartell, D. Krum, and M. Bo-

las. Leveraging change blindness for redirection in virtual environ-

ments. In IEEE Virtual Reality, pages 159–166, 2011.

[6] E. A. Suma, D. Krum, S. Finkelstein, and M. Bolas. Effects of redi-

rection on spatial orientation in real and virtual environments. In IEEE

Symposium on 3D User Interfaces, pages 35–38, 2011.

[7] E. A. Suma, B. Lange, A. Rizzo, D. Krum, and M. Bolas. FAAST: the

flexible action and articulated skeleton toolkit. In IEEE Virtual Reality,

pages 247–248, 2011.

180