A mixed reality room following the generalized peephole

metaphor

Andreas Butz, University of Munich, Germany, butz@ifi.lmu.deAntonio Kruger, University of Muenster, Germany,

antonio.krueger@uni-muenster.de

Abstract

We present the generalized peephole metaphor, a model of interaction for ubiquitous com-puting and instrumented environments. The metaphor provides a way of organizing andstructuring ubiquitous in- and output facilities in instrumented environments consisting ofseveral distributed but coordinated sensors and displays. Its main idea is to look at the en-vironment as one large display and sensor continuum, in which peepholes provide localizedand user-specific windows between the physical environment and a virtual information layer.The metaphor nicely matches models of human perception, for example the fact that humansmake use of external representation in their environments and access information by guidingtheir attention to specific locations.

We then present a specific MR room and show how a number of in- and output activitiescan be described in terms of the peephole metaphor. We discuss how the metaphor can copewith scalability and access control and how it supports a family of interaction styles and pre-sentation methods in instrumented environments. We analyze the technological requirementsfor implementing the peephole metaphor and show that it works very well with the limitedhardware already available today, such as projector-camera units, wall-mounted displays andportable screens. Looking beyond today’s technological limitations, we argue that peepholeswill be a particularly helpful metaphor for structuring interaction with hardware which mightbe available in the future, such as room-covering interactive displays in the form of e-inkwallpapers and ubiquitous gesture recognition.

Keywords: H.5.1.b Artificial, augmented, and virtual realities, H.5.2.i Interaction styles

1 Introduction

The vision of ubiquitous computing as introduced by Mark Weiser describes a world in which manyof our daily objects are equipped with processing power and networked with their environmentsin order to provide helpful services. Early research in the field has concentrated on displays andthe ability to provide services depending on the user’s [12] and the display’s [7] location. Whileinteraction with a single small display has become a familiar situation with the appearance oftoday’s mobile computing devices, interaction across multiple displays and interaction with largedisplays is the topic of ongoing research.

With the introduction of IBM’s Everywhere Displays projector [9] it became possible to createdisplays on everyday surfaces and to move them under the control of the environment. The originalED projector created stationary desktop interfaces in given locations. Later applications used thetechnology more flexibly to label real world objects and generally create visual elements in thephysical world. At this point, we might just as well assume a different standpoint and describe asteerable projector as a single time-multiplexed room-sized display. All surfaces in the room havethe capability to display information, but not necessarily at the same time, and in a relatively lowresolution. Conventional displays then provide islands of higher resolution, better interactivityand temporally unrestricted availability in this display continuum. This effectively adds a virtuallayer on top of the physical environment, turning it into a Mixed Reality environment.

1

If we try to envision the technological development, it might well be possible some day, to createnon-multiplexed room-sized displays with the help of, e.g., wallpapers and floor tiling using e-inktechnology. This would provide the environment with the capability of simultaneously displayingvisual elements on all of its surfaces. Thoughtlessly used, such an environment could easily becomeconfusing and annoying by visually polluting the otherwise calm surroundings. In the followingsections we will discuss how such visual pollution can be prevented by restricting visual displayto small peepholes into an omnipresent virtual layer, which itself is spatially continuous. We willshow that with the current technology of steerable projectors, a room-sized MR environment canbe built which nicely implements this peephole metaphor by its pure working principle, and wewill introduce a number of simple applications we built in this room in order to demonstratethe viability of the peephole metaphor. The structure of this article follows the same road map:After a brief discussion of related work and work we took inspiration from we will introduce thepeephole metaphor, discuss its technical and cognitive aspects, as well as some implications ofit. In the next part we will then introduce an instrumented environment which we built usingthis metaphor. Then follows a list of applications which were implemented in this environmentin chronological order. At the end of each application we will briefly highlight what aspect ofthe metaphor was stressed there. After these application descriptions, we will give an outlook onconceptual extensions of the metaphor and close the paper with a summary.

2 Related work

From a technological side, recent work on steerable projector-camera units, such as Crowley’sportable display screen (PDS) is relevant to our research. The PDS provides a tracked cardboardsurface onto which content is projected in such a way, that it appears stabilized on the cardboard.This involves real time tracking and rectification of the projected image and basically emulatesa very lightweight and bright display, which can be moved freely. From a conceptual side, ourwork draws on the established ideas and interface concepts of spatially aware displays [7] andthe notion of magic lenses and toolglasses [2]. These provide peepholes in the form of specializedviews into a virtual layer and allow to naturally navigate this layer by physically moving thelens, glass or display. More recently, Yee [13] nicely defined the notion of peepholes into a virtuallayer by describing interaction techniques for a PDA display in an imaginary information plane.Baudisch kept the imaginary virtual layer visible in his work on focus and context displays [1],where he created an island of high resolution in a larger display of low resolution. Finally, wewill see that the peephole metaphor as described here implies a number of properties which wererecognized as being important in other work. Streitz describes in his roomware work a family ofdisplays integrated into furniture. Two of these displays can be spatially composed to form onelarger spatially continuous display. The importance of spatial continuity is also emphasized inRekimoto’s work on multiple display environments.

3 The peephole metaphor

The peephole metaphor was first introduced in [5], where we discussed how its basic idea was builton a generalization of earlier work and where we described its properties on a conceptual level. Inthe meantime we have built a number of applications which in their own way implement or useaspects of the peephole metaphor, and which we will discuss in this article.

3.1 Base metaphor

The core idea of the peephole metaphor is a virtual layer superimposed to our physical environment.This layer is normally not perceptible, but can be made visible, hearable, or otherwise sensible byopening a peephole from our physical world into it. Such a peephole can be opened by a display(visual peephole), a loudspeaker (acoustic peephole) or other devices. A display will, for example,show an item of information which is spatially located at the display’s position.

2

Basic properties of the virtual layer are its spatial continuity, its temporal persistence, and itsconsistency across different peepholes. If we move two different portable display devices to thesame position in our environment, they will display the same information, if not necessarily inthe same graphical representation. In the special case, in which the display is a tracked, head-worn display, the virtual layer becomes a virtual world superimposed to the real world whereverwe look, and we obtain the general paradigm of spatial augmented reality (AR). In the specialcase in which the display is a position-tracked handheld computer, our metaphor describes Yee’speephole interfaces [13] and Fitzmaurice’s Chamaeleon [7] adequately. Both implement a spatiallycontinuous virtual layer which is only visible through the peephole opened by the tracked display.Peepholes in our environment work much in the same way as magic lenses on a 2D screen. If weadd interactivity, they behave like toolglasses [2].

3.1.1 User- and system-initiated peepholes

Peepholes can be created both by the user and by the system. If they are actively created andmanipulated by the user, we call them user-initiated peepholes. With the use of a display underthe environment’s control, such as a steerable projector, the environment can actively open system-initiated peepholes. The latter can be used, for example, to hint the user at certain content, toprovide alarms or to otherwise guide the user’s attention to a certain position in space. We proposethis terminology to replace our own former active and passive, which carried too many unwantedconnotations and was particularly misleading under a user-centered perspective.

3.1.2 Input and output peepholes

Peepholes can provide a view for users into the virtual layer, usually via situated (i.e. location-aware) displays. These peepholes are called output peepholes. If we think of cameras or otherinput channels, they open a ”view” into the physical world from the virtual layer and can thus becharacterized as input peepholes. Other examples of input peepholes are touch-sensitive surfaces.A location-aware touch-screen display therefore implements a two-way peephole which allows in-teraction (input and output in the same place). Input peepholes don’t need to spatially coincidewith output peepholes, if we think of steerable cameras and steerable projectors mounted in (andpointing to) different locations.

3.2 Technical aspects

The peephole metaphor is ideally suited for time multiplexed displays, such as steerable projectors,since they only allow simultaneous display in a very restricted area anyway. The most straight-forward solution for image rectification in a steerable projector consists of a virtual camera whichis moved synchronously with the physical projector[9]. The projector just shows what the camerarecords, so if items in the virtual layer are visually represented in the 3D world surrounding the vir-tual camera (which has to match the physical room geometrically for correct image rectification),the physical projector creates a peephole into the virtual layer by its pure working principle.

Tracked handheld displays, such as the Chameleon [7] or the vampire mirror[3] also createpeepholes in a very direct way. They show a geometric 3D representation of the virtual layer,which in their case just contains 3D objects. This situation can be generalized for virtual layerscontaining arbitrary items, such as documents or media files. If they are each assigned a certainposition in space, they can be seen (and manipulated) on a tracked handheld display as soon asthe display is located close enough to this position. The representation of the object on the displaydoesn’t need to be 3D. It can be a regular desktop icon, a full view of a text document or image,or any other visual representation suitable for the display at hand. If we use a mobile phone asthe display, it might display an icon or a text line with the object’s name. If we move a PDA tothe same location, it might display a preview version of the document, and if we use a tablet PC,it might show us a full representation of a text or image, which we can edit or manipulate withthe pen.

3

3.3 Cognitive aspects

While traditional models of visual attention assume that everything we see is accumulated ina visual buffer and that all subsequent cognitive processing relies on this buffer, newer modelsdeny the existence of such a continuously detailed buffer. Experiments have shown that visualdisturbances in an image, which attract the visual attention of observers, make them fail tonotice big changes in other places of the image at the same time. This effect, known as changeblindness, provides strong arguments against the existence of a cognitive visual buffer providinghighly detailed information everywhere. The model described by Rensink in [10] explains therole of attention to establish the visual coherence between objects. By guiding our attention toa certain object, we retrieve highly detailed visual information, which is lost when we guide ourattention to other parts of the environment. A much coarser representation of the object remainsin memory, which allows, for example, to remember the object’s position in space.

This view on the cognitive representation for visual processes has some striking similaritieswith the peephole metaphor described above. Instead of displaying every visual information allthe time, we rely on guided attention to (re-)establish spatio-visual consistency between objects inthe environment. According to Rensink’s model, only those objects are represented in high detail,which are needed for the visual task at hand. For our peephole view we argue in the same way:only the areas which are needed for the user’s actual task will display virtual objects. In the sameway the brain is saving processing power and memory, the instrumented environment is savingsimilar resources when presenting and obtaining information to and from users. The implicationsof Rensink’s model suggest that users will still be able to interact with the virtual layer effectively,although only a small part of it is visible in high detail.

Using peepholes, applications can even maintain an explicit model of the user’s attention, whichwill improve the match between the area of attention of users and the locations where informationis presented. Furthermore, simultaneous interaction of several users in the same instrumentedspace can be supported by well-designed peepholes and the exploitation of the human ability toconcentrate on a specific area in spite of disturbances caused by other individuals in the room (theso-called cocktail-party effect.

3.4 Implied interaction techniques

In a spatially continuous layer, moving objects between displays is logically reduced to moving theobjects in space, if we assume a room-stabilized virtual layer and that the displays don’t move atthe same time. Spatial composition of displays thus comes for free. We can take an object with usby making it stick to the display that currently shows it. If we release it somewhere else, we havetransported it to that location. Objects attached to a display device change their spatial positionsynchronously with the device’s position. For transportation, technically they are moved from theglobal coordinate system into the local coordinate system of the device, e.g., screen coordinates.

Better scalability and access control to files can easily be implemented by a simple extensionof the metaphor. Many people are familiar to some extent with the concept of optical filters orcurtains. We know that they prevent us from seeing something behind them or at least filter oralter the visual appearance. If we think of situated displays as peepholes into a virtual layer,this additional step is quite natural. By thinking of the peephole containing a filter, we caneasily understand that it will only show certain objects, e.g., the ones which are relevant for usunder some criteria, or the ones we are allowed to see. We also know that filters and their effectscan be combined. A logical ”and” between filter effects means putting the filters one after eachother, a logical ”or” means putting them in parallel, which has to be supported by correspondingvisual hints. The concept of filtered peepholes can be used to handle worlds with almost arbitraryamounts of data in the virtual layer by filtering out just relevant objects. It can be used to handleaccess controls by providing, for example, ”read-only” filters or ”existence” filters which wouldonly show that there is a certain object at this location, but not what the object actually is.

4

3.5 Managing Peepholes

On the conceptual side, the peephole metaphor provides a terminology, a mental model, and anelegant way to describe different types of interactions in instrumented environments. On thetechnical side, it suggests the concept of a peephole manager which orchestrates interaction andallows application designers to build applications more easily. We have implemented a simpleversion of such a peephole manager in our instrumented environment. In analogy to a windowmanager, which takes care of window placement as well as input and output focus, the peepholemanager organizes and administrates the peepholes of the environment and the interaction withthem. The peephole manager takes, for example, care of positioning peepholes. Position andsize of peepholes and virtual objects can be specified in different frames of reference. On theroom level, peepholes are placed at 3D-coordinates in the coordinate system of the room. Sincepeepholes created by displays or surfaces are inherently two-dimensional, positions can also bespecified as 2D-coordinates in the local coordinate system of these peepholes. The same positionspecifications are used to specify the position of content in the virtual information layer of theenvironment. While the virtual layer is superimposed and thus homomorphous to the physicalthree-dimensional space, it can contain an arbitrary number of additional dimensions describingdifferent properties of information associated with a 3D room coordinate. One dimension can, forexample, be ownership, another temporal validity of data. Filters can then be used to select therelevant subspaces, such as all files belonging to one owner, or all information newer than one day.

Besides opening and closing static peepholes, the peephole manager can also control gradualchanges of the peephole properties over time, e.g., to realize moving peepholes. This is done bymeans of interpolators instead of fixed coordinate values. If an object is positioned in the localcoordinate system of a system-initiated peephole and this peephole is moved across the room, theobject will always remain visible and visibly travel from one place to another.

4 A mixed reality room as a display continuum

Figure 1: A conceptual sketch of our room-sized display continuum. Apart from the steerableprojector on the ceiling it contains a projective desk and a number of displays of various sizes aswell as a set of speakers for spatial audio.

5

To test and develop our ideas related to the peephole paradigm, we are using an instrumentedlaboratory at Saarland University, the former affiliation of both authors. The Saarland UniversityPervasive Instrumented Environment (SUPIE) provides a heterogenous assembly of displays andsensors. As can bee seen in the sketch of SUPIE in figure 1, the available displays range in sizefrom a fixed 50-inch plasma screen to much smaller, but portable tablet PC and PDA screens.The central component in the room is a ceiling-mounted steerable projector (see also figure 2 left),which can be used to display information on every visible surface in SUPIE. The steerable unit iscomplemented by an additional fixed projector which is used to augment the surface of an officedesk. When designing SUPIE, we tried to blend all technology into the room’s architecture in orderto avoid distraction by the obvious presence of technology and to enhance the user’s impression tointeract with information objects rather than with computing hardware. Although the steerableprojector could only be partially integrated into the ceiling, we were able to integrate the plasmascreen into one of the walls and to place the fixed projector behind another one, using a mirror todirect the projector’s beam through a hole in the wall into the SUPIE (see figure 2). The overallinstallation allows to look at the entire SUPIE as a display continuum with varying physical (i.e.resolution and color depth) and temporal properties (i.e. ability to display information at a certainpoint in time in a certain area of the room).

Figure 2: The steerable projector used in the SUPIE environment (left), a hole in the wall with amirror for the projector beam over the instrumented desk (middle), and one of several additionalsteerable cameras (right).

Several types of sensors have been deployed to track portable screens and users in the envi-ronment. Four networked cameras are installed in the room to track user positions, gestures andoptical markers. Two of those cameras have been installed in two corners of the room. Thesecameras are steerable and have an optical zoom, allowing them to focus on an arbitrary locationin the room. One camera has been mounted above the desktop surface to identify objects andhand gestures and another camera has been installed on the steerable projector, which allows tointeract with the information projected to a surface. Active RFID-tags provide further informa-tion of tagged objects and users in the room. Although the precision of these sensors is rathercoarse (1-3 m), it helps to focus the cameras on an area of the room where objects or users areexpected, thus supporting the optical tracking procedures.

In analogy to the visual display continuum, SUPIE also provides an audio continuum. Aneight channel surround sound system allows to place sound sources at arbitrary room coordinates.This enables the environment, for example, to provide spatial audio cues in order to guide theuser’s attention to interesting areas of the visual display continuum. This is especially useful foruser notifications if information has appeared outside the user’s field of view.

5 Example applications using peepholes

5.1 Searchlight

The searchlight application described in [6] implements system-initiated input peeping for scanningthe environment for physical objects and it uses system-initiated peephole output for guiding the

6

Figure 3: The results of a query are highlighted by projecting a bright spot around the corre-sponding objects in the physical environment. In these examples, a book is found on the shelf(left) or lying on the window sill (right).

user’s attention to a certain physical object. In an initial scanning phase, the steerable projectortakes a series of overlapping pictures with its attached camera covering the whole room. In thesepictures, optical markers attached to objects are found using the AR Toolkit library and theirposition is stored in a database. This represents a system-initiated input peephole under thecontrol of the environment. Upon a query, the environment can then actively highlight the objectsby opening a system-initiated output peephole in their location and projecting a bright spot aroundthem (figure 3). As described in [6], this whole procedure can even be implemented without anexplicit 3D model of the environment, but as soon as other input technologies (such as tracking ofobjects by RFID tags) are used, a full spatial model is needed. The SearchLight application wasour first prototype using system-initiated input and output peepholes.

5.2 WipeIt

We have demonstrated an interaction technique called wiping for moving objects to distant displayswhich might not be spatially accessible to the user. Objects on a desk surface are displayed inan output peephole created by a steerable projector (see figure 4 left). A camera attached tothe steerable projector opens the corresponding input peephole and observes object motions in itsframe by analyzing difference images. If an object such as a hand is recognized to move consistentlywith a certain speed behavior in the location of the displayed object, this is recognized as a wipinggesture (see figure 4 right).

The objects displayed on the desk surface can thus be wiped from the desk in different direc-tions. In our demo setup the display closest to the wiping direction would ”catch” the objects.They could thus be wiped to the monitor standing on the desk or to the white projection screenin the background of figure 4 left. The wiping gesture is a very direct implementation of movingobjects spatially. Wiping a virtual object will make it move to a different location in the virtuallayer. Catching the object by a different peephole in the direction of wiping is important for visualfeedback and also makes the otherwise quite imprecise wiping direction and speed ”snap” to asensible value. Objects can just be wiped to other peepholes and don’t get lost in the virtual layerthis way. WipeIt was our first prototype exploring interaction across several peepholes.

5.3 Annotation of physical objects by projection

Another example application we have built is a labelling function for physical objects. Thislabelling function starts from the 3D model of a physical object and its sub-objects and computesgood and unambiguous positions where labels which can be put to explain details about the object.

7

Figure 4: A steerable projector displays visual objects on a desk surface (left). A camera attachedto it determines from difference images whether a hand (or any other object) moves across theframe in a consistent direction and recognizes this as a wiping gesture (right).

Figure 5: A geometrically complex physical object whose subparts are labelled with projected text

The labels can be text, graphics or video objects and are placed in the computed positions aroundthe object (see figure 5). The steerable projector is then used to open a peephole around theobject and just displays all annotation objects which have been placed in the virtual layer. Ifthe environment contained large amounts of information in the virtual layer, the peephole wouldnaturally also make other items visible, which happened to be close by the annotated physicalobject. To prevent this, a filter was used with the annotation peephole, which just displayed theannotation objects. The annotation application was our first prototype using filters in conjunctionwith peepholes.

8

5.4 A virtual room inhabitant

One of the most pressing research problems of ubiquitous computing is finding appropriate interfacemodels for users to manage the complex machinery behind the scenes. In one of our prototypes, wehave investigated the use of a life-like anthropomorphic character who can be addressed by the user.Earlier studies with desktop systems have shown that interfaces incorporating life-like charactersare subjectively rated better than standard interfaces[11]. We are convinced that the advantages oflife-like characters will multiply in instrumented environments with complex technical equipment.Being in such an environment, delegating tasks to a virtual character (also called Virtual RoomInhabitant, VRI) seems a natural way for users to achieve their goals. The VRI can accompanythe user and provide personalized assistance under consideration of user preferences and availableresources.

Figure 6: The virtual room inhabitant, projected next to a large plasma screen on the wall

Conceptually, the VRI is a visual and audible manifestation of the technology in the environ-ment. From an implementation point of view, it is a special information item which is used tocommunicate planned or scripted content. The VRI makes itself visible by creating a system-initiated output peephole. In [8], a VRI is projected by means of the SUPIE steerable projector.It can guide the user’s attention to other peepholes for specific viewing situations. Being able tomove freely through the environment, the VRI can, for example, signal the user to follow him to alarge display with high resolution in order to display detailed content. This effectively representsa peephole switch, since the user can now be expected to pay attention to the new peephole thathas been opened on the large screen. It is therefore safe to close the initial peephole for the VRIand use the technical resources (here: the steerable projector) for other purposes (i.e. for newpeepholes).

5.5 Spatial audio notification

In addition to the optical peepholes we discussed, it is also possible to create acoustic peepholes.The SUPIE instrumented environment contains a speaker setup which can create spatialized audio

9

in real time, using vector based amplitude panning. It can position sounds such that they appearto come from a certain direction or to move along a certain path. In [4] we describe a scheme fornotifying users by embedding optional parts into a composition which is playing as the backgroundmusic in our environment.

Users choose their preferred instrument or rhythmic pattern which will serve as their personalnotifier. The core part of the background music is normally playing without using this particularinstrument or pattern. As soon as information for the user appears somewhere in the room, theuser’s instrument or pattern is added to the overall mix and appears to come from the directionwhere the information appears. This is an acoustic form of system-initiated peeping. The en-vironment opens an acoustic peephole in the same spatial position as the peephole in which theinformation resides, to which the notification refers. The notification sound is simply put in thesame position as the information item and thus directs the user’s attention in this direction.

6 Ongoing and future work

6.1 Peephole detection

One important technical question is how to retrieve possible locations of peepholes. For buildingprototypes, it is possible to explicitly model those locations and manually maintain a consistentmodel of the environment. While this is plausible for test purposes and for instrumented spaceswith low complexity, it does not scale to technically complex and dynamic instrumented environ-ments. Hence, it would be desirable to allow these environments to detect possible locations ofpeepholes automatically. We are currently looking at possibilities to easily detect the location ofdisplays in such a space. For this purpose we are investigating a two-step procedure involving onlythe displays and cameras present in the environment.

In a first step the displays start to blink in a characteristic color and sequence in order tobecome easily detectable for the cameras around them. Once their presence is registered by oneor more cameras, the cameras will be able to zoom in on the display to retrieve images of higherresolution. In a second step the displays present geometrical patterns of a given size and shape,thereby allowing the cameras to determine their relative location and orientation. This schemenicely corresponds to a two-tier social protocol used in human communication to preserve andfocus the scarce resource of attention: If we want to communicate to a person who is currentlybusy with something else, we first knock, wave or utter an indistinct sound to get her attention,and only then say what we want to communicate. In an instrumented environment containingmultiple cameras and displays, displays can follow this scheme in order to inform the environmentabout their own position. Since the procedure matches the human social protocol so closely, theenvironment might even use it to gain user’s attention in the same way.

6.2 Wormholes

If we are looking for ways of implementing the ubiquitous computing vision, we will encountervery large environments containing a virtual layer of sorts. Peepholes provide a convenient wayof structuring our interaction with information in the virtual layer of our visible environment,for example, at the scale of a single room. In a world with many of these rooms we might wantto use a similar metaphor to structure interaction between several of these rooms, and to createcommunication channels between distant rooms. A convenient metaphor from physics are theso-called Einstein-Rosen bridges which science fiction has made quite popular under the nameof wormholes. In Physics, wormholes create a transportation channel between regions which aredistant in space or time. In a metaphorical sense, wormholes create a channel from our localenvironment to a distant one. In their simplest form they connect an input peephole on one sideto an output peephole on the other side and vice versa. For practical matters, these could be acamera and a display on both sides, which then provide a bridge for video conferencing. If we addan acoustic wormhole, users in the two distant environments can see and hear each other and have

10

a conversation. We have thus connected the two physical environments. If we want to connectthe virtual layers of two environments, we can build virtual wormholes, which basically reflect thecontent of the virtual layer in a specific area on one side to a specific area on the other. Thesevirtual wormholes can be used to implement shared workspaces onto which users on both sidescan open peepholes to manipulate the same objects. The same filtering mechanisms can then beapplied as for peepholes, thus creating a metaphor for access restrictions and privacy. We arecurrently setting up two additional instrumented environments in Munich and Muenster. Thesewill be connected to the instrumented environment in Saarbrucken via wormholes of various sortsand we hope to get additional insights from these practical trials.

7 Summary

We have introduced the generalized peephole metaphor as a model for interaction with Mixed Re-ality and instrumented environments. The particular instrumented environment, which we haveintroduced, represents a room-sized, spatially continuous, but time-multiplexed display. Users canmove around inside this display continuum and interact with information through peepholes into avirtual information space which is overlaid to the physical room. While this metaphor makes gooduse of technology available today, such as steerable projectors, it will also be helpful for structur-ing room-sized displays available in the future, although these might not have today’s technicallimitations, and might, in principle, allow simultaneous, ubiquitous display of information.

From studies of human perception we have learned that humans don’t perceive their wholeenvironment at an equally detailed level. The focus of attention defines which part of the en-vironment is perceived in high detail, and which parts remain in the periphery. It is thereforelogical, to just display information in such focus areas and leave the rest of the environment calm.This strategy will save technical resources, such as power or computing capacity, but it will alsocontribute to create calm environments and avoid visual pollution in a world, in which displaysmight be available in arbitrary sizes and qualities.

8 Acknowledgments

The work described in this article was funded by ”Deutsche Forschungsgemeinschaft” under theyoung investigator award ”Fluidum” and the special collaboration project SFB 378. We would liketo thank our anonymous reviewers for valuable suggestions and all of our colleagues and studentswho helped implementing various aspects of the work described. In particular, we thank ChristophEndres, Ralf Jung, Michael Kruppa, Christian Schmitz, Michael Schmitz, Michael Schneider andMira Spassova.

References

[1] P. Baudisch and N. Good. Focus plus context screens: Visual context and immersion on thedesktop. In SIGGRAPH 2002 (Emerging Technologies demo paper), 2002.

[2] Eric A. Bier, Maureen C. Stone, Ken Pier, William Buxton, and Tony D. DeRose. Toolglassand magic lenses: The see-through interface. Computer Graphics, 27(Annual ConferenceSeries):73–80, 1993.

[3] Andreas Butz, Tobias Hollerer, Steven Feiner, Blair MacIntyre, and Clifford Beshers. En-veloping users and computers in a collaborative 3D augmented reality. In Proceedings of theInternational Workshop on Augmented Reality IWAR ’99. IEEE Computer Society Press,1999.

11

[4] Andreas Butz and Ralf Jung. Seamless user notification in ambient soundscapes. In Proceed-ings of the International Conference on Intelligent User Interfaces (IUI 2005) in San Diego,California, January 9 - 12, 2005.

[5] Andreas Butz and Antonio Kruger. A generalized peephole metaphor for augmented realityand instrumented environments. In Proceedings of The International Workshop on SoftwareTechnology for Augmented Reality Systems (STARS), Oct. 7, 2003 in Tokyo, Japan, 2003.

[6] Andreas Butz, Michael Schneider, and Mira Spassova. Searchlight - a lightweight searchfunction for pervasive environments. In Proceedings of Pervasive 2004. Springer LNCS, 2003.

[7] George W. Fitzmaurice. Situated information spaces and spatially aware palmtop computers.CACM, 36(7):38–49, July 1993.

[8] Michael Kruppa, Mira Spassova, and Michael Schmitz. The virtual room inhabitant. InMU3I: Multi-User and Ubiquitous User Interfaces. workshop held in conjunction with IUI2005, 2005.

[9] Claudio Pinhanez. Using a steerable projector and a camera to transform surfaces intointeractive displays. In Proc. ACM CHI 2001, Seattle, Washington, USA, March 31 - April5 2001. ACM Press.

[10] R. Rensink. Internal vs external information in visual representation. In Proceedings of SmartGraphics 2002, pages 63–70. ACM Press, 2002.

[11] Susanne van Mulken, Elisabeth Andre, and Jochen Mueller. The persona effect: How sub-stantial is it. In Proc. Human Computer Interaction Conference, pages 53–58. Springer, 1998.

[12] Roy Want, Bill Schilit, Norman Adams, Rich Gold, Karin Petersen, John Ellis, David Gold-berg, and Mark Weiser. The parctab ubiquitous computing experiment. Technical ReportCSL-95-1, Xerox Palo Alto Research Center, mar 1995.

[13] Ka-Ping Yee. Peephole displays: Pen interaction on spatially aware handheld computers. InProceedings of CHI 2003. ACM Press, 2003.

12