Augmented Places and Mixed Realities

communicating history through mobile media and physical space

Staging and Practicing Places: Site-specific Designs,

Aarhus University October 11th 2011

Jacob Knudsen, udviklingskonsulent, Videnscenter for Integration, Vejle Kommune Kjetil Sandvik, associate professor, Media, Cognition and Communication,

University of Copenhagen

Museums, digital media and a perpetual beta way of communicating

• Case: “The 23 sculls – a conspiracy tail”

• communicating cultural heritage in the age of mobile media, experience economy and new learning formats:

• the history of Vejle (DK) communicated through a mixed reality game using mobile phones, web 2.0 mash-ups, a playable conspiracy plot, and the city as game universe

The main point!• Communicating cultural heritage is

about telling stories

• It is about engaging the users in the storytelling process (participation)

• It is about creating a storytelling device which can be played with, manipulated, changed (co-creation)

• A constructivist approach towards knowledge and learning

Storytelling classic

• A chain of events in time and space• Told by someone (a narrator) to

somebody else (a reader or spectator)

• Through a specific media (novel, movie, TV series…)

• And in a specific discourse (a genre defining the structure of the plot/storyline)

Storytelling 2.0: participation

• The ingredients are the same, but the role of the recipient has changed

• The story hands out the possibility for interaction: – to influence the course of events– to gain control over one or more

characters – to play a part in the storyline

• Storytelling Storydwelling

Labyrinthine story structure

Myst 2: Riven

Storytelling 2.0: co-creation

• Added the possibility for the participants to take part in creating the story, – adding new parts to it, – adding new characters, – adding new narrative spaces and so

on

• Storytelling Storyprocessing

Storyspace

World of Warcraft

Sandbox game

Design tools

Concept

Second Life

Environment

Augmentation

• an informational, aesthetical and/or emotional enhancement of our sense and experience of place by use of various framing strategies (e.g. Wallander’s Ystad) and mobile media technologies (e.g. audio guides).

Understanding places through media

• We understand places through media (e.g. Lonely Planet, Google Earth, travel literature and so on),

• We use media to construct places (using cameras, mobile phones, GPS, and so on),

• Media shapes our experience of places (guided tours, theme parks, computer simulated worlds, and so on).

• Augmentation in this context implies that the actual place is blended with mediated representations and media-inflected perceptions of the place.

Augmented places

• Places which has gotten a certain surplus of meaning, a certain kind of narrative embedded into it.

• The characteristics of these places have been enhanced in that a certain mode, atmosphere or story has been added to them as an extra layer of meaning.

Augmentation of places

• A process of storytelling • the place constitutes a scene for the

performance or staged re-enactment of ‘true’ stories,

• actual events which at some point have taken place at the specific location.

• Traversing the specific location and at the same time having a specific story linked to the location narrated

The staging of the Jack the Ripper-story runs through and organizes urban space of today’s London and thus changing it into a specific place with a specific atmosphere and a specific plot: the scene of the crime.

Augmentation of places

• Storytelling

• A process of fictionalization

• the actual place is working as a setting for fictions, which again influence our perception of the place.

• May be used for augmented reality experiences like murder walks

When tourists embark on one of this tours, they are taken on a guided walk through parts of the actual towns working as ‘scenes of the crime’ (Milan, Paris…) in Brown’s novel, but following the trails laid out not by some historical person or chain of historical events (like in the case of the Jack the Ripper-tour ) but by fictional characters and their actions and thus the actual places have become augmented as a result of fictionalization.

Augmentation of places

• Storytelling

• Fictionalization

• Construction of a kind of mixed reality

• the place has a status both as an actual location in the physical world and as a storyspace

• blend of fact and fiction.

Split reality vs Mixed reality

• Split reality: switching between mediated space (e.g. inside the mobile phone) and physical space

• Mixed reality: blending between mediated and physical space (e.g. looking at physical space through an ‘augumented reality browser’ on the mobile phone)

• Mixed reality implies a certain way of telling stories connecting the actual and the fictional space (this is where Hikuin’s Vendetta goes wrong)

The 23 sculls

Participation:• The players are put in the role as

journalists investigating the disappearance of a museum inspector.

• He has left behind a lot of notes and disturbing video clips on YouTube about a conspiracy engineered by powerful men throughout the history of Vejle city.

• And he has left maps, trails and layers of information about various places, various building, various persons – everything accessible on a variety of web 2.0-services as the player navigates through the physical town.

The 23 sculls

Co-creation• As journalists the players are not just to

uncover a hidden story• They are too piece things together, make their

own interpretations and conclusions and to write their own tale of the conspiracy.

• The storyspace is structured in a way that the players can create their own stories and at the same time securing that there is a narrative progression:

• As they send their texts and pictures to the news paper editor (the game as system) new levels with new hints and clues are revealed

There are real human sculls in there!

The 23 sculls• The game has a traditional level structure with

each level being themed in a way so that there is a narrative progression:

• Level 1’s theme is “the city is filled with clues”, while level 2’s theme is “it is all about power” and finally the theme of level 3 is simply “everything is connected”.

• There are a lot of clues, hints, story fragments to be found both in the physical city and in its mediated online counterpart

• Pictures, videos, trails or routes may be found on Flikr, YouTube and Google Maps may be displayed as augmenting information layers over the various physical locations when they are observed through the camera lens and the augmented reality browser (Layar).

The 23 sculls

• For the players the city has been altered, it has been augmented by the interplay between and interweaving of the mediated city and the actual physical city.

• As such this new, mixed reality version of Vejle, which the players now are to investigate, functions as an augmented place.

Physical space as media• The physical space is to some degree

functioning as media communicating specific types of information, specific types of stories.

• the city quarters with its streets, alleys, buildings, ornamentations such as statues, gargoyles and so on function as a narrative architecture like a theme/themed park like Disneyland including buildings and landscapes known from the catalog of Disney fairytales

• Several parts of the city of Vejle used as location for the 23 sculls game have these qualities of being media in themselves, as carriers of the story of Vejle.

The city seal depicting the river and the bridge: the two main infrastructural components constituting Vejle as a city of commerce (they play also other important roles in the history of Vejle)

Physical space as media

• With the use of mobile phones equipped with navigation tools and augmented reality browsers this information residing in the very architecture and infrastructure of the city may be pulled forth and made visible, accessible and interactive from the perspective of communicating history and cultural heritage and facilitating learning processes.

Storytelling 2.0 structure in The 23 sculls

Story-space

Story-space

Story-space

Theme 2hints, clues

Level 1 Level 2 Level 3Theme 3hints, clues

Theme 1hints, clues

task 1 solved task 2 solved task 3 solved

Co-creationParticipation

The tale of the

conspi-racy

Summing up so far…• Augmentation makes us see things in

new ways: • Buildings are not just buildings, streets

are not just streets – the carry stories, they carry cultural meaning which the players through the gameplay and the interplay between the physical space of the city and the mobile media may acquire, discuss, investigate and relate to.

• By applying mobile media we may challenge and develop our way of communicating knowledge and cultural heritage through augmented reality game systems based on participation and co-creation.

Can you do it your self?

Well, we did…

---and it is getting easier all the time.

Mashup Webpage

Mashup Layar

A local project that you can test for your self…

In the very near future

Almost ”everybody” can make their own AR (With a little help from a friend):

• Games

• On line learning tools

• Location based storytelling (Why read a book, when you can take a walk)

• Art

• Etc.