Floating Integral Photography Using Fresnel Mirror

Kazuhisa Yanaka Masahiko Yoda Terumichi Iizuka

Kanagawa Institute of Technology

ABSTRACT We previously developed an integral photography (IP) system in which a 3D image with both horizontal and vertical parallax looks as if it is floating in the air, and we have now expanded it so that animated images can be displayed. No special glasses are necessary with this system, which consists of a 3D display subsystem and a Fresnel mirror. The light emitted from the 3D display subsystem is reflected by the Fresnel mirror, and the image of a floating object is then formed in space.

KEYWORDS: 3D display, integral photography, Fresnel mirror

1 INTRODUCTION Integral photography (IP) is an ideal 3D display method because viewers can see full-color moving 3D images with both horizontal and vertical parallax. Moreover, as shown in Fig. 1, it requires only simple hardware consisting of a fly’s eye lens and a liquid crystal display (LCD) on which a special image, called an “IP image”, is displayed.

Figure 1. Simple IP system using LCD and fly’s eye lens.

In the past, IP was very expensive because an extremely high resolution LCD was required. Moreover, the lens pitch of the fly’s eye lens had to be the integer multiple of the pixel pitch of the LCD, as shown in Fig. 2 (a), in which the integer was two. This meant that the fly’s eye lens had to be custom-made in accordance with the pixel pitch of the LCD, usually by using a very expensive metal mold.

With the extended fractional view (EFV) method [1][2][3], however, the ratio of pixel pitch to lens pitch can be an arbitrary real number, as shown in Fig. 2 (b). This condition is satisfied even if any ready-made fly’s eye lens and any ready-made LCD are combined. Very inexpensive IP systems are thus possible.

(a) Conventional IP method (b) EFV method

Figure 2. Conventional IP method and EFV method.

In the EFV method, the algorithm used for synthesizing IP images is unique. The ray that is emitted from each LCD pixel, refracted at the fly’s eye lens, and captured by one of an array of

cameras is traced, and the corresponding pixel value of the camera is set to the pixel value of the LCD. In this process, the difference between the lens pitch of the fly’s eye lens and the pixel pitch of the LCD is absorbed. An animation is essentially a series of still pictures, so the EFV method is applicable to not only still pictures but also animations [2][3]. The EFV method can also be applied to live-action moving images [4].

We expect IP to play a key role as the gateway between real space and virtual space in the near future, but before that can happen, there is one problem that must be solved. In conventional IP systems, the observed 3D objects may look as if they are floating or sinking, as shown in Fig. 3 (a), but they are usually not very far away from the screen. It is quite difficult to actually float objects in the air far away from the screen, as shown in Fig. 3 (b), and in this study, we overcame this restriction by combining the conventional display method with a Fresnel mirror.

Figure 3. IP technologies.

2 PRINCIPLE OF THE SYSTEM Fresnel mirrors are produced by depositing aluminum onto

acrylic Fresnel lenses in a vacuum. The Fresnel mirror used in the proposed system has basically the same function as a conventional concave mirror in that it converges incident light to a focal point, as shown in Fig. 4 (a). However, Fresnel mirrors are quite different from standard concave mirrors: they are far thinner, lighter, and easier to install because they do not have a curved surface. Moreover, large Fresnel mirrors can be manufactured easily. When there is a point light, the real image of it is formed, as shown in Fig. 4 (b). In this case, the lens formula

1/a + 1/b = 1/f

is applied, where f denotes the focal length and a and b denote the distance from the mirror. Because a and b are approximately anti-proportional, the real image of an arbitrary real object is formed, as shown in Fig. 4 (a).

This leads to a very interesting question: What will happen when the real object is replaced by a vertical object produced by IP? The answer is shown in Fig. 5 (b). Essentially, virtual objects created by IP technology can be floated in the air by using the Fresnel mirror. Note that up and down has been reversed due to the mirroring effect. Even in this case, the depth axis of the

displayed with IP

LCD

(a) Conventional IP technology

(b) Desired IP technology

Fly’s eye lens

Virtual object Virtual object displayed with IP

Fly’s eye lens LCD

135

IEEE Virtual Reality 20124-8 March, Orange County, CA, USA978-1-4673-1246-2/12/$31.00 ©2012 IEEE

floating object is not reversed. Therefore, the real image can be seen quite naturally (except that it is mirrored). As for the still images, it has previously been reported that IP can display 3D images when using this method [5]. In this study, we applied the proposed method to an animation produced with CG.

Figure 4. Real images created by parallel incident light and point light.

Figure 5. Real images created by real and virtual objects.

3 EXPERIMENT In this experiment, we used an IP subsystem consisting of an ordinary 12.1-inch laptop PC (Fujitsu FMV-R8250, number of pixels: 1280 H × 800 V, dot pitch: 0.204 mm) with an LCD covered by a fly's eye lens (Fresnel Technologies No. 360, with enlarged external size, lens pitch: 1 mm), and a 300-mm square Fresnel mirror (Fig. 6 (a)).

An animation in which a character named “Simon” walks forth was created using Poser software. The background was black. Each frame was rendered from 32 × 32 viewpoints using Shade to synthesize an IP image with our software, after which a video file was made from the IP images and played back on the laptop PC.

This procedure is basically the same as that described in our previous papers except that mirrored images were used. A high quality 3D animation with full parallax (as if it were floating in the air) was obtained (Fig. 6 (b)). Currently, the position at which a user can observe the 3D image is mostly restricted to the front of the Fresnel mirror. Moreover, slightly visible concentric rings of the Fresnel mirror seem to degrade the image quality. These issues can be solved by using a larger and thinner Fresnel mirror that has narrower grooves. At this point, further consideration is required in terms of what type of content or which applications can make the best use of this technology.

(a) Top view

(b) Front view

Figure 6. Experimental system.

4 CONCLUSION A 3D display technology that combines IP and a Fresnel mirror

was proposed and demonstrated to be effective for displaying floating moving images with full parallax. It is expected to be useful for displaying virtual 3D objects in mixed reality systems.

REFERENCES [1] Kazuhisa Yanaka. Integral photography using hexagonal fly's eye

lens and fractional view. SPIE Proceedings Vol. 6803, Stereoscopic Displays and Applications XIX, Andrew J. Woods, Nicolas S. Holliman, and John O. Merritt, Editors, 68031K, pp. 1–8, 2008.

[2] Masahiko Yoda, Akifumi Momose, and Kazuhisa Yanaka. Moving integral photography using a common digital photo frame and fly's eye lens. SIGGRAPH ASIA 2009 (Yokohama, Japan, December 17–19, 2009) Poster, 2009.

[3] Kazuhisa Yanaka, Masahiko Yoda, and Akifumi Momose. Integral Imaging of 3-D CG Animation Using a Laptop PC and General-purpose Fly's Eye Lens. SID Symposium Digest of Technical Papers May 2010, Volume 41, Issue 1, pp. 1256–1259, 2010.

[4] Masahiko Yoda and Kazuhisa Yanaka. Real-Time Integral Photography Using the Extended Fractional View Method. SIGGRAPH ASIA 2010 (Seoul, Korea, December 15–18, 2010) Poster, 2010.

[5] Kazuhisa Yanaka and Masahiko Yoda. Generation of Image Perceived as Floating Using Concave Fresnel Mirror. ICIPT2011 (Bangkok, Thailand, August 17–20, 2011) pp. 96–101, 2011.

Fresnel mirror

Focal point

(a) Parallel incidence light and its real image

f

Fresnel mirror

Fresnel mirror

b

a

(b) Point light source and its real image

Real image

Point light

Real image of virtual object

Real object (a) Real image created by real object

(b) Real image created by virtual object

Real image of real object

Virtual object displayed with IP

Fly’s eye lens on LCD

Floating full parallax 3D moving image

136