New ways of worldmaking: the Alterne platform for VR art

TeesRep - Teesside'sResearch Repository

Item type Meetings and Proceedings; Book Chapter

Authors Cavazza, M. O. (Marc); Lugrin, J-L. (Jean-Luc); Hartley,S. (Simon); Libardi, P. (Paolo); Barnes, M. J. (Matthew);Le Bras, M. (Mikael); Le Renard, M. (Marc); Bec, L.(Louis); Nandi, A. (Alok)

Citation Cavazza, M. O. et. al. (2004) 'New ways of worldmaking:the Alterne platform for VR art', 12th annual ACMinternational conference on multimedia, 2004, New York,USA, October 10 - 16, in Schulzrinne, H. and Dimitrova,N. (eds) Proceedings of the 12th annual ACM internationalconference on multimedia. New York: ACM, pp.80-87.

DOI 10.1145/1027527.1027542

Publisher ACM

Rights ACM allows authors' version of their own ACMcopyrighted work on their personal server or on serversbelonging to their employers. For full details seehtp://www.acm.org/publications/policies/RightsResponsibilities [Accessed 04/06/2010]

Downloaded 8-Jun-2018 19:15:09

Link to item http://hdl.handle.net/10149/100301

TeesRep - Teesside University's Research Repository - https://tees.openrepository.com/tees

TeesRep: Teesside University's Research Repository http://tees.openrepository.com/tees/

This full text version, available on TeesRep, is the post-print (final version prior to publication) of:

Cavazza, M. O. et. al. (2004) 'New ways of worldmaking: the Alterne platform for VR

art', 12th annual ACM international conference on multimedia, 2004, New York,

USA, October 10 - 16, in Schulzrinne, H. and Dimitrova, N. (eds) Proceedings of the

12th annual ACM international conference on multimedia. New York: ACM, pp.80-

87.

"© ACM, 2007. This is the author's version of the work. It is [posted] included here by permission of ACM

for your personal use. Not for redistribution. The definitive version was published in Advances in

Computer Entertainment Technology, 2008

http://dx.doi.org/10.1145/1027527.1027542

When citing this source, please use the final published version as above.

This document was downloaded from http://tees.openrepository.com/tees/handle/10149/100301

Please do not use this version for citation purposes.

All items in TeesRep are protected by copyright, with all rights reserved, unless otherwise indicated.

New Ways of Worldmaking: the Alterne Platform for VR Art Marc Cavazza1, Jean-Luc Lugrin1, Simon Hartley1, Paolo Libardi1, Matthew J. Barnes1, Mikael

Le Bras1, Marc Le Renard2, Louis Bec3 and Alok Nandi4 1 School of Computing, University of Teesside, Middlesbrough TS1 3BA, United Kingdom

m.o.cavazza@tees.ac.uk 2 CLARTE, 6 rue Léonard de Vinci, BP 0102, 53001 Laval CEDEX, France

3 CYPRES, Friche de la Belle de Mai, 41 rue Jobin, 13003, Marseille, France 4 Commediastra, 182, av. W. Churchill, 1180 Brussels, Belgium

Abstract We introduce a novel approach to the creation of Virtual Reality Art installations, which supports the design of alternative worlds, in which laws of Physics can be redefined to induce new user experiences. To implement this concept of "Alternative Reality", we have used Artificial Intelligence techniques to support the definition of the virtual environment behaviour, an approach inspired by Qualitative Reasoning systems. Besides the redefinition of physical laws, we have developed mechanisms for eliciting causal relations between events, as causality plays an important part in users' perception of virtual worlds. Our pilot installation is a CAVETM-like system incorporating a state-of-the-art computer game engine as visualisation software, which has-been ported to this immersive display. The event-based system underlying the game engine is used to bypass the native Physics engine and replace it with our Alternative Reality software. A first prototype has been fully implemented, the Alternative Reality modules totaling over 100,000 lines of C++ code. We present early results obtained with this approach, illustrated with examples taken from two artistic briefs, developed by digital artists associated to this research.

Categories and Subject Descriptors H5.1 [Multimedia Information Systems] Artificial, Augmented and Virtual Reality - Virtual Reality for Art and Entertainment.

General Terms Theory, Algorithms, Design and Experimentation.

Keywords Digital Arts, Intelligent Virtual Environments, Qualitative Physics, Causality.

1. Introduction Virtual Reality Art [8] [13] is at the forefront of Digital Arts as it explores at the same time visual aesthetics, the construction of alternative universes and user interactive experiences. To a large extent, VR Art has kept in touch with the original ideas of VR pioneers, who advocated VR as an endeavour to produce artificial

realities and psychedelic experiences [10], not just as a simulation tool aiming to recreate our common sense reality. In that sense, there is a tradition in VR Art to construct alternative worlds, e.g. in Char Davies’ OsmoseTM environment, Louis Bec’s artificial creatures [1] or Maurice Benayoun’s Quarxs™, invisible creatures that bend the rules of Physics. In this context, the objective of the Alterne platform is to facilitate the creation of Virtual Worlds whose physical behaviour departs from our common sense experience, enabling new kinds of virtual explorations and the development of “alternative realities”. One of the challenges is to improve the conceptual continuity between the creative stages and their technical implementation. This should support the creation of alternative realities from first principles, rather than by the ad hoc scripting of pre-defined effects. Our central idea consists in using Artificial Intelligence techniques to support the definition and implementation of virtual worlds behaviour, an approach inspired by Qualitative Reasoning systems. These have multiple advantages. From a representational perspective, the existence of an explicit symbolic description has the potential to facilitate discussions between artists and scientists during the specification phase (epistemological continuity). But from an implementation perspective, the discretised approach makes it possible to simulate more complex phenomena by increasing the level of abstraction of the simulation. In the next sections, after a brief overview of the system architecture, we introduce our approach to Alternative Reality, which is based on Intelligent Virtual Environments, i.e. the inclusion of AI-based simulation underlying traditional VR approaches. Throughout the presentation, we will illustrate the system behaviour with examples from artistic briefs which support the first demonstrators of the platform.

2. System Architecture: the VR Installation The system aims at constituting a complete platform for (immersive) VR Art, which includes visualisation hardware and software. The specific behaviour module which supports alternative reality as defined above, is an additional layer built on top of the visualisation software. The first requirement is to offer a good immersive experience, which will properly support the perception of alternative physical phenomena. This is why we operate within a CAVE-like environment, the SAS Cube™, which is a PC-based immersive visualisation system (Figure 1). We use a state-of-the-art game engine, Unreal Tournament 2003 (UT 2003), as a visualisation engine. Games engines provide

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MM’04, October 10–16, 2004, New York, New York, USA. Copyright 2004 ACM 1-58113-893-8/04/0010...$5.00.

Figure 2: System Architecture

Figure 1: Immersive Visualisation in the SAS-Cube™

sophisticated visualisation features and most importantly constitute a software development environment in which further components can be integrated. The latter explains that Game engines are increasingly used in VR research [11]. The former has made game engines a popular choice for 3D artists as well (e.g. Tobias Bernstrup; Matthias Fuchs and Sylvia Eckermann). Our choice of UT 2003™ is based on the above criteria, plus our own previous experience in using it for other research applications. In particular, it comprises a sophisticated Physics event-based system which is an important API into which our developments have been integrated, bypassing the native Physics engine. Our Alternative Reality module, whose components will de detailed in further sections, represents a total of 100,000 lines of C++ code, communicating with the UT 2003 engine via UDP protocol. The overall software architecture is represented on Figure 2. In addition, UT 2003 has been ported to immersive environments through the CaveUT™ system [9]. We have adapted the latest version of the CaveUT™ system to the SAS-Cube™ and have extended its feature to include stereo visualisation with head and wand tracking.

The CaveUT™ software is based on two components:

• A mutator (an Unreal-specific script which enables to modify the behaviour of objects in the 3D engine) modifying camera placement, as the native “spectator” camera has to be controlled via the CaveUT™ system.

• An OpenGL driver, re-compiled to modify the frustum of the camera in order to get a correct perspective view when the user is not facing the centre of the screen.

We have developed a specific interface module supporting stereoscopic display in the SAS Cube™, which is to be included in the latest CaveUT™ release (known as CaveUT2k3™). In order to implement stereoscopy, we have opted for the use of two PC per screen, generating separate images for the left & right eyes. The SAS Cube™, like similar immersive displays, is equipped with a Crystal Eyes system. These are coupled with genlocked video cards, thus forming an active stereoscopy system. Four groups of two PCs are used to produce an active stereoscopy video signal (one group for each side of the SAS Cube™). For each PC, we need to modify the configuration file used by CaveUT™, in order to get a correct perspective. In addition, the two video signals need to be “mixed” via a hardware solution, which in our case makes use of an ORAD™ PC cluster. Within this cluster, each PC is genlocked with the others (to obtain the same refresh rate, with the same phase). They are linked via a LAN, and their video input/output are chained. Tracking is performed through an IS900 Intersense™ tracker, integrated in the cluster using a VRPN server. Figure 1 gives a view of a spectator experiencing one of our artistic briefs in the SAS Cube™.

3. The Experimental Artistic Briefs We have developed our research very much as an “Art + Science” [16] project, in which artistic concepts can echo scientific enquiry and conversely, new technical approaches can support new forms of experience in artistic installations. Throughout this paper, we illustrate our Alternative Reality technology with data from two experimental artistic briefs, which are being produced as part of the Alterne project, to support demonstrations of the Alternative Reality platform. In this section, we give a short overview of these briefs, introducing the behavioural aspects that are relevant to the concept of Alternative Reality. These two example briefs are “Ego.Geo Graphies” (Alok Nandi) and “Arapuca” (Louis Bec).

3.1 “Ego.Geo Graphies” This brief is situated in an imaginary world governed by alternative laws of physics. It features the behaviour of physical objects as actors, thus blurring the line between objects and actors in a narrative. This brief features a certain number of object-actors (spheres) being emitted by certain areas of the landscape (Figure 3). These acquire the status of autonomous agents, whose main interaction mode with other agents (or with elements of the virtual world) is physical collision. The consequences of these multiple collisions are made to vary according to the narrative context, which is itself influenced by the history of user interaction, somehow representing the “empathy” demonstrated by the user towards certain object-actors. On one hand, this brief is an exploration of the notion of context through the variable behaviour of the environment which itself responds to the user

involvement. But on the other hand, it constitutes an exploration of causality. As such, it requires mechanisms varying the physical effects of collisions (bouncing, merging, bursting, exploding, altering neighbouring objects…), taking into account the semantics of the environment. These mechanisms, implemented through a “causal engine” will be described in section 4.

3.2 “Arapuca” Arapuca is the latest development of Louis Bec’s artwork based on artificial life. This brief aims at enhancing this work by supporting the creation of artificial organisms living in artificial worlds, thus producing a virtual ecosystem (Figure 4). It introduces real principles for the behaviour of virtual creatures by modelling their internal physiology with Qualitative Reasoning techniques that are formally similar to Qualitative Physics. In a similar fashion, various physical processes can be modelled for the creatures’ environment thus supporting the creation of an artificial ecosystem. This makes possible to simultaneously experiment with imaginary creatures and imaginary worlds where physical laws can be redefined.

Figure 4: The “Arapuca” World

4. The Elements of (Alternative) Reality In VR, the sense of reality derives from the experience of interacting with the virtual environment. When the world is rich enough to feature a variety of physical objects, which are also allowed dynamic behaviour, the apprehension of the world by the

user depends on its perceived regularities. Users will form mental models of the world, thus developing a “common sense” physics for the virtual environment as a whole. From this empirical, yet cognitive, perspective, we can identify two essential components of the construction of reality. The first one corresponds to the laws of physics that govern the behaviour of objects, materials and structures, which accounts for an understanding of mechanistic behaviours. The second one, causality, relates events to one another. Causality has both predictive and explanatory aspects. It is an important concept for making sense of the world, hence also a privileged target to create illusion and surprise. Although physical processes rely on causal propagation, causality itself cannot be reduced to physical phenomena, as i) it can relate events outside the scope of a single physical process and ii) it can operate at various levels of description. In the next sections, we describe the mechanisms we have implemented to support experimentation with these two aspects of reality, illustrated through examples from artistic works in progress.

4.1 Defining (Alternative) Laws of Physics with Qualitative Simulation Physical simulation is an important aspect of VR realism, albeit, from a computing perspective, a resource-intensive one. Most visualisation engines, including game engines, resort to a semi-discretised approach, based on the notion of events. Events play an interface role between those processes that are physically simulated (such as motion trajectories) and those who have been pre-calculated (deformations, fractures, explosions). These event-based systems also provide an opportunity to tamper with physical behaviour at various stages of the simulation. However, rather than departing from physical laws through a variety of “cheats”, altering basic physical interactions such as collision mechanisms or gravity, the real objective should be to support the principled definition of alternative laws of Physics. These would be part of the world-making process, in which the artistic brief for VR will extend itself to encompass aspects of the world Physics. To that effect, we have adopted a simulation technique called Qualitative Process Theory [7] [3]. Its main advantage is to be centred on the actual physical processes in a way which is closest to common sense reasoning about physical systems. It also naturally embeds the causal nature of some physical phenomena (i.e. the fact that a flow of liquid will fill a container). It would be highly impractical to invent a consistent set of novel equations of Physics based on the intention of an artistic brief. Rather, because QPT is based on a high-level conceptualisation of the physical laws it facilitates the knowledge transfer between a plain language brief, or simple schematic representations and the formalism itself. Qualitative Process Theory is centred around the notion of process, which represents integrated Physical phenomena, in a way which is characteristic of our common sense experience, e.g. boiling, evaporating, flowing, filling, etc. Within these processes, the laws of Physics are represented via qualitative equations relating discrete variables. These equations describe the causal relations between qualitative variables. Figure 5 illustrates the simulation of liquid flows in the context of the Ego.Geo Graphies brief with one of the corresponding QP.

Figure 3: The “Ego.Geo Graphies” World

The QP formalism comprises pre-conditions for its activation, in this case the alignment between a liquid source and a “recipient”. The key equations governing the QP simulation are called influence equations and express the variation of qualitative variables through time. In this case, the fact that liquid (?sub liquid) accumulates in the pit (?dst) as a function of the flow rate can be formalised through such an influence equation, which reads:

QPT provides a complete set of mechanisms to solve these equations thus determining the variation of qualitative variables. In this context, there are many advantages to the use of QPT. One is that qualitative modelling of abstract processes is much more efficient than the accurate numerical integration of the corresponding equations, which will not be compatible with real-time interaction on a large-scale environment. But the most significant one is that QPT naturally supports the definition of alternative processes based on alternative laws of physics [2], thus supporting the principled description, from the artistic brief, of the physical laws of fantasy worlds, such as containers accepting infinite amount of liquid without overflowing, yet increasing their mass in proportion.

The Qualitative Physics engine is a C++ software component integrated into the visualisation engine communicating with it through UDP protocol. The main principle consists in bypassing the native Physics engine for a specific class of objects, called QP objects. When QP objects are interacted with, interaction events trigger the QP engine instead of the native Physics engine. This is based on an Event Interception System (described in [2]), which serves as an interface layer between the visualisation engine and our behavioural systems. The QP Objects can be defined through qualitative variables, which are the quantities that are involved within a processes qualitative proportionalities and influence equations. The initialisation of the QP Engine begins by a data driven pattern-matching procedure, which ensures that every object that can take part in QPs is associated with the QP events for their activation. Thus, the events that satisfy a QP’s preconditions will activate that instance within the QP engine. The quantities within the associated QP objects are then analysed and, if these quantity conditions are passed, the QP engine will activate the corresponding process. In the above example (figure 5), when a liquid-flow is aligned with a container, the pre-conditions of a filling QP are satisfied. The QP engine then proceeds to check the quantities with the QP Objects involved and then allows the triggering of the corresponding process simulation. In the course of a simulation, the encapsulated qualitative variables (e.g.

volume or amount of liquid) are updated in real-time and their values are visualised in the virtual environment through dynamic changes in objects’ appearances (using animations, dynamic textures or particle systems). The QP engine is principally involved with the resolution of influence equations. The algorithm for the resolution of influences within the QP engine begins by testing the influences on the quantity. If the quantity is already being influenced, either directly or indirectly, its derivative value is calculated by the summation of all the influences. If the quantity is being singly influenced we calculate the derivative directly from the influence equation. Since, the QPT formalism expresses a process’ direct influences as causing a change in the first parameter at a rate specified by the second parameter. In the example influence equation above, the influences are interpreted as: the change in the qualitative value “amount-of” is given by increasing the value at the rate in the qualitative value “flow rate” for the QP Object “pit” (?dst). However, if the quantity is not being singly influenced the first stage in the calculation for a quantity's derivative is to test whether all the influences have the same sign. If all the influences have the same sign, the engine can determine readily how the quantity will change. Ambiguities for the influences arise when a quantity has influences of different signs or has a quantity that is both directly and indirectly influenced. The resolution of this situation is by an order of magnitude approach. The QP engine updates values as the qualitative simulation passes the landmarks that have been associated to the objects (it uses qualitative proportionalities where appropriate). When the QP engine updates a value within an object it also tests whether the change triggers the event for passing the landmark. An example of a landmark value is the boiling temperature for water when this is passed an event is generated to begin the steaming effect. The qualitative engine then generates appropriate events in the unreal engine that trigger corresponding changes in objects’ states. The QP engine activates relevant QP (those which pre-conditions are valid in the current world state) and simulates the evolution of variables through the QP. In the above example, when a liquid-flow is aligned with a container, the pre-conditions of a filling QP are satisfied, triggering the corresponding process simulation. In the course of a simulation, the encapsulated qualitative variables (e.g. volume or amount of liquid) are updated in real-time and their values are visualised in the virtual environment through dynamic changes in objects’ appearances (using animations, dynamic textures or particle systems). Another example is represented on Figure 6, which shows how physical processes determine interaction between a virtual creature and its environment in the “Arapuca” brief. Physical

Figure 5: Qualitative Simulation of Liquid Flows

Figure 6: Qualitative Simulation of Environmental Effects

processes in the environment such as those governing the behaviour of the liquid medium affect the stability of the Diaphaplanomena. The QP described in the figure controls the appearance of turbulences in the environment, whose strength is a function of temperature gradients. These can impact on the virtual creature and prompt it to respond by a compensatory behaviour maintaining its buoyancy and stability.

4.2 Causal Attribution in Virtual Worlds: the “Causal Engine” Causal perception is one of the important phenomena through which we construct our reality from experience. There are many aspects to causal reasoning, which constitutes a topic widely debated in Cognitive Science, Psychology, Philosophy, AI [14], and … Digital Arts alike [15]. However, causal perception refers to the apprehension of real-time physical events. Hence the idea that inducing causal perception could be a powerful vehicle for experimental VR. The creation of causal impressions in real-time is a technical challenge that first needs to rest on a foundation of human causal perception. There is an abundant literature on the subject since the experiments of Michotte [12]. From a pragmatic perspective, it is considered that human subjects derive causal relations from co-occurring events, seen in retrospect as action and reaction. Research on the role of agency in causal attribution suggests that certain expectations are triggered when the action in originated by the user; however this is not an absolute pre-requisite, as subjects are able to perceive causal relations between events they have not initiated. In an artistic context, causal impressions can be an important aspect of user experience. The difficulty lies in being able to “programme” causality on the basis of the artistic intentions: this requires mechanisms for the explicit handling of causality. Alternative causality is based on the artificial creation of co-occurrences, that is to say temporal relations between events. Specific events detected by the user tend to generate expectations, such as the collision between two objects (e.g. two spheres in the Ego.Geo Graphies world). To create an artificial co-occurrence, the normal consequence should be prevented from happening and instead replaced with another effect in close temporal succession from the original action. The substitution of new effects to the normal consequences of occurring events is performed by a “causal engine” which intercepts real-time events and modifies them using semantic information. The causal engine operates in the following fashion. High-level events, called Context Events (CE) are instantiated from the flow of low-level basic events obtained from elementary mechanisms in the game engine (collisions, entering volumes,

etc.). The system also intercepts the native consequence events normally associated to them. Each CE thus represents a frozen cause-effect relationship formalised using a STRIPS-like representation [5], in which the post-conditions represent the normal consequences. For instance, Figure 5 shows the CE for a bouncing effect between two spheres. The causal engine constantly samples the environment for such events at a given sampling rate. Within a given sample, the collection of CE will be modified to generate new co-occurrences. This modification is performed by the application of a set of Macro-Operators (MOp), which alter the post-conditions of the frozen CEs. These alterations can consist in changes to the kind of effects or to the objects to which these effects apply. For an example of effect modification, a sphere colliding against a wall will not burst but instead generate more spheres, or change the colour of the wall. They can also change the object instance to which this effect is applied, so that if two spheres collide, none of these will explode but instead some elements in their immediate surroundings will do. A key aspect of the process is that these MOp can be classified according to the type of transformations they perform and that this classification can be used by the causal engine to search the set of MOp to be applied (by using a cost-bound search algorithm operating in real-time: an important feature of the causal engine is that, even though it is based on a heuristic search algorithm, its response time always lies within the allotted sampling rate (of 50-100 ms)). We can now illustrate this through several examples involving collisions between object-actors in the Ego.Geo Graphies brief. It can be noted that (although the brief was in no way influenced by this fact) collision between moving objects is considered as a standard example in studies on causal attribution [12] [5]. In the world of Ego.Geo Graphies, sphere-shaped object-actors may collide with one another or with elements of the landscape. The effects of a collision between spheres is normally expected to be felt on the spheres themselves and the nature of the effect will depend on visual cues as to their physical properties (i.e. soft/hard, deformable, etc.), which can be conveyed to some extent by their textures and animations. The default effect of two spheres colliding is considered to be a soft bounce due to their overall visual appearance and speed. This is represented as the baseline CE for sphere-sphere collision (Figure 7-A). The causal engine can apply various transformations to this baseline CE. It can for instance replace the bouncing effect with the explosion of both spheres (by applying a “change effect” MOp) or with a fusion of the colliding spheres into a single larger one (Figure 7-B-C). Another way of inducing causal perception is to propagate effects to elements of the landscape itself (a specific

class of MOp exists for propagating effects). In that instance, the collision between two spheres will result in the explosion of landscape elements (Figure 7-D). In VR Art, believability, rather than realism, is the key requirement, precisely because of the potentially unfamiliar nature of the virtual worlds created. In the specific case of alternative reality installations, user experience is determined by two principal factors: one is the nature of user interaction required to experiment alternative reality and the other is the response time of the alternative behaviour mechanisms. The first aspect is largely determined by artistic decisions, as the user interfaces in the immersive environment (head tracker and wand tracker) do not differ from those used in other CAVE-based installations. The authoring process should thus determine those affordances that will trigger interactions revealing the alternative nature of the environment. The second one, response time, actually conditions the believability of the environment as a whole. We have gathered psychological data on one specific aspect, causal perception, which are largely compatible with the response times obtained with the causal engine (i.e. < 100 ms).

5. Knowledge Engineering as Authoring The approach we have introduced, besides supporting new VR experiences, may also have an impact on the authoring process. We have already mentioned that the use of a symbolic representation should facilitate the discussion at a more abstract, conceptual level, which is closer to initial ideas contained in many artistic briefs. To a large extent, this can be said to shift the development process from software engineering to knowledge

engineering. The latter situation is characterised by the use of symbolic representation to encode knowledge described in natural language, acquired through interviews and/or discussions. However, while the explicitation of content in these representations facilitates the discussion between the scientists and the artists, the use of specific AI formalisms still prevents direct authoring. We can however, at this stage, describe the authoring process used for the current briefs for both components of Alternative Reality (causality and artificial physics) as they already open some interesting perspectives. The description of a novel/alternative physical law involves identifying the causal relations between variables within a process and the writing of QP. The description of an alternative physical process in a brief is most often derived from an existing one, explaining which elements of alternative behaviour should replace the traditional ones. This means that in terms of implementation as well, alternative physical processes can be derived from ordinary ones (such as filling, burning, freezing, melting, bouncing, etc.). The groundwork for these processes is already available through an ontology of QP. The knowledge engineering step can then be limited to the redefinition of specific Influence Equations accommodating the alternative behaviour stated in the brief. The formalism underlying alternative causality, which as we have seen is derived from planning formalisms, is also of a difficult handling in terms of knowledge representation. However, it does

Figure 7: Alternative Causality in the “Ego.Geo Graphies” Brief

not constitute a target formalism for artistic authoring. Within an artistic brief, the definition of alternative causality can start with a list of possible effects affecting elements of the world (which have to be visually implemented). The subsequent step is to specify the conditions under which certain co-occurrences will take place. In the “Ego.Geo Graphies” brief for instance, the causal effects have been “storyboarded” by initially listing cause-effects relationships. This was subsequently refined by identifying a list of possible consequences for each event. What the causal engine should implement is firstly the possibility of generating alternative consequences and, secondly, some form of control, in line with the artistic intentions. The generation of alternative consequences is supported by MOp (modifying frozen Context Events), whose nature can in many cases be related to elements of the brief (Figure 8). This is the case because most MOp have a clear semantics, such as the propagation of effects to other target objects, or the substitution of new target objects to the original ones. The second aspect governs the heuristic selection of transformations: it should be related to high-level concepts such as “level of change”, “level of surprise”, “level of disruption”. To some extent, the heuristics governing MOp selection can also be adapted to reflect these concepts. Overall, the most important aspect is that these representations support quick prototyping and interactive experimentation, which are well adapted to the artistic development process.

6. Conclusions The creation of VR installations has until recently essentially been based on visual rendering and pre-defined modes of interaction, in which context many technical problems used to be given design-based solutions. What our approach offers is the

possibility to aim at more sophisticated worldmaking by re-incorporating high-level concepts describing the dynamic behaviour of the virtual world into the design process. Creating a common symbolic level facilitates collaboration for artistic projects with a strong epistemological stance. This has the potential to shift the implementation phase of VR Artworks from pure software engineering to knowledge engineering, which in turn should not only facilitate development but also potentially improve the creation of abstract building blocks and their re-use within certain classes of applications. Our research in simulation techniques that could underpin alternative reality was originally driven by the finding that many VR Art briefs were concerned to some extent with physical phenomena, without always properly formalising them. However, it is legitimate to discuss under which conditions digital artists, especially those not originally associated with the project, would benefit from these technical developments in terms of artistic creation. We address this issue from two complementary perspectives: one is the actual artistic motivation for adopting these tools and the second is their practical usability.

The motivation derives from many observed trends in VR Art, such as interdisciplinary interest (involving Physics) and the rapid adoption of game engines as a support for VR Art. We also received positive feedback from artists during previous project presentations [4], where these tools were seen as emancipating VR Art from the traditional presuppositions of simulation. In terms of their usability, as the discussion on authoring has illustrated, the ALTERNE tools support the definition of complex alternative worlds, while being still compatible with a storyboard approach for the early phases of creation, thanks to its declarative formalisms (see Figure 9).

Figure 8: Generation of Event Co-occurrences by the Causal Engine

Figure 9: Symbolic Representation Layers in the Alternative Reality Platform

Finally, this approach offers new perspectives on VR Art, and has a potential to improve collaboration from an Art + Science [16] perspective.

7. Acknowledgements The ALTERNE project (IST-38575) is funded in part by the European Commission, under the IST initiative (Cross-Programme Action 15). Jeffrey Jacobson developed the original CaveUT2003™ system and is thanked for his assistance in adapting it to the SAS-Cube™.

8. References [1] Bec, L. Elements d'Epistemologie Fabulatoire, in: C.

Langton, C. Taylor, J.D. Farmer, & S. Rasmussen (Eds.), Artificial Life II, SFI Studies in the Sciences of Complexity, Proc. Vol. X. Redwood City, CA: Addison-Wesley, 1991

[2] Cavazza, M., Hartley, S., Lugrin, J-L. and Le Bras, M. Alternative Reality: A New Platform for Digital Arts. Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST 2003) (Osaka, Japan, October 1-3, 2003). ACM 2003, ISBN 1-58113-569-6, 2003, 100-108.

[3] Cavazza, M., Hartley, S., Lugrin J-L. and Le Bras, M. Qualitative Physics in Virtual Environments. Proceedings of the 2004 International Conference on Intelligent User Interfaces (IUI ’04) (Funchal, Madeira, Portugal, January 13-16, 2004). ACM 2004, ISBN 1-58113-8, 2004, 54-61.

[4] Cavazza, M., Lugrin, J-L. Hartley, S., Libardi P, Barnes M, Le Bras M, Le Renard M, Bec L and Nandi A. ALTERNE: Intelligent Virtual Environments for Virtual Reality Art. Proceeding of the 4th International Symposium on Smart Graphics (SG 2004) (Banff Centre, Canada, May 23-25, 2004). Lecture Notes in Computer Science 3031 Springer 2004, ISBN 3-540-21977-3, 2004, 21-30.

[5] Chaput, H. H. & Cohen, L. B. A model of infant causal perception and its development. Proceedings of the Cognitive Science Society Conference. Edinburgh, 2001.

[6] Fikes, R. E. and Nilsson, N. J. STRIPS: a new approach to the application of theorem proving to problem solving. Artificial Intelligence, 2 (3-4), 1971, 189-208.

[7] Forbus, K.D. Qualitative Process Theory. Artificial Intelligence, 24, 1-3, 1984, 85-168.

[8] Grau, O. Virtual Art: from Illusion to Immersion. MIT Press, 2003.

[9] Jacobson, J. and Hwang, Z. Unreal Tournament for Immersive Interactive Theater. Communications of the ACM, Vol. 45, 1, 2002, 39-42.

[10] Leary, T. Chaos and Cyberculture, Ronin Press, 1994. [11] Lewis, M and Jacobson. Games Engines in Scientific

Research. Communications of ACM, Vol. 45, No. I, 2002, 27-31.

[12] Michotte, A. The perception of causality. New York: Basic Books. Translated from the French by T. R. and E.Miles, 1963.

[13] Moser, M.A. (Ed.) Immersed in Technology: Art and Virtual Environments. Cambridge (Massachussets), MIT Press, 1996.

[14] Pearl, J. Causality. Cambridge University Press, 2000. [15] Sato, M., and Makiura, N. Amplitude of Chance: the Horizon

of Occurrences. Kinyosya Printing Co., Kawasaki, Japan, 2001.

[16] Sommerer, C. and Mignonneau, L. (Eds.) Art @ Science. New York: Springer Verlag, 1998.