instructors: jay bolter, lcc blair macintyre, coc some...

MR Design

Some Media ConceptsOntology, Aura, Presence, Liveness,

Instructors: Jay Bolter, LCC Blair MacIntyre, CoC

Thursday, April 26, 12Revised March 2009

Media Concepts: Ontology, Presence 122

Why consider media history?

History of media contributes to understanding new media(Digital design affected by evolving cultural assumptions)

Media issue of the 20th and 21st centuries

medium’s ability to reproduce realityphotographyfilmtelevision

VRAR/MR

Thursday, April 26, 12

Issues of media theory and their relationship to our AR design projects.

trying to consider the meaning for digital design of these earlier media.


Ontology of the Image

Andre Bazin, Ontology of the Photographic Image What does Bazin mean by “ontology”? (not what semantic web researchers mean) means the status as real, resemblance to the real

Striving for realism in painting: Renaissance perspective Camera obscura of Renaissance --> camera of Niepce two ambitions of painting spiritual reality psychological reality - illusionism

Thursday, April 26, 12belief in the reality of the photograph - goes back to 19th century. Fox Talbott The Pencil of Nature. William J. Mitchell. The Reconfigured Eye. this would now be considered the naive view: why? Post-modern claims that arts can't resemble the real because the real has disappeared. common neoMarxist view that arts are ideological - reflect ruling class view, not reality (if there is such a thing).

What happened in the Renaissance with regard to painting.Renaissance painting as new kind of realism. new balance between between artist's desire for spiritual expression and optical illusion (or realism - note how the terms illusion and reality conflate in this kind of discussion.[what is perspective painting - claim to mathematical solution to experience of viewing. projection of 3D onto 2D plane. is it "natural"? - what the viewer "really" sees? does not reveal as much as certain other forms of representation that are less realistic.


Renaissance and linear perspective

Massacio Dürer


Massacio Dürer and renaissance perspective




Camera Obscura


Camera obscura realization of the camera (later becomes the photographic camera) but here in Renaissance it was one way of getting a projection that was more or less mathematically true.



Techniques of linear perspective


Renaissance painting as new kind of realism. new balance between between artist's desire for spiritual expression and optical illusion (or realism - note how the terms illusion and reality conflate in this kind of discussion.[what is perspective painting - claim to geometrical solution to experience of viewing. projection of 3D onto 2D plane. is it "natural"? - what the viewer "really" sees? does not reveal as much as certain other forms of representation that are less realistic.


Mathematics of projection


Steadman’s 3D model: belief that Vermeer actually painted using a camera obscura



Photorealism before photography: Canelleto


Caneletto in 18th century


Photorealism before photography: Ingres


Not the best example. but realism of portrait is getting. Ingres early 19th century. These portraits got largely replaced by photography.


Photography as photorealism


Not the best example. but realism of portrait is getting. Ingres early 19th century. These portraits got largely replaced by photography.


Pencil of nature


Not that old a picture - but dates from 19th cetury so it will do for this puprose -belief in the reality of the photograph - goes back to 19th century. Fox Talbott The Pencil of Nature. William J. Mitchell. The Reconfigured Eye.


Painting after photography


What happened to painting in the aftermath of the creation of photography?


Painting after photography


This is Kandinsky from the 1930?



Andre Bazin, Ontology of the Photographic Image Perspective as “original sin” of Western painting redeemed by Niepce and Lumière (by photography and film)

Photography is automatic and “objective” principle of transparency: photograph as transparent record

Cinema is objectivity in time (adds temporal dimension to reproduction of the real)

Photography “freed” Western painting from realism and allowed it to recover aesthetic autonomy


Ontology of Photographic Image



insert a quicktime of a classic Hollywood film - again the issue of realistic representation

next to an avant-garde piece - could do chien d’andalou

actually do excerpt from The Ring - the ring video

talk about true avant-garde


We talked before about the Hollywood style and how it follows Bazin - Ontology of the Image




what is this? the death video from the Ring. Most of the Ring is done in Hollywood style -

talk about the avant-garde




Comparison of popular vision of avant-garde and “true” avant-garde Chien d’andalou



Two representational strategies in 20th century:

transparent hypermediated (representation) looking through looking at (user)

mainstream? unusual? (cultural status) avant-garde



Ontology of the Image: analog/digital

Is any medium really transparent?


W.J.Mitchell - manipulation of images does not depend just on digital. How we believe newspapers.


Ontology of the Image: analog/digital

Is any medium really transparent?


How do we know that Geitner and Obama were really together in the same room a couple of days ago?


Photorealism


Is this a photograph or a photorealistic computer graphic, neither - it is a painting. Ralph Goings


Photorealism: Goings


Is this a photograph or a photorealistic computer graphic, neither - it is a painting. Ralph Goings


Ontology of the Image: Computer Graphics


Computer graphics - perfect (at least potentially) mathematization of the image because the image is never taken from the light at all. entirely constructed. So it is totally realistic according to principles of Renaissance and yet totally fabricated. Like animated film - it is just a construction.


Ontology of the Image: Computer Graphics


Computer graphics - of course more than just projection - also texturing and shading to achieve photorealism.



Two representational strategies in digital technology:

transparent hypermediated (representation) looking through looking at (user)

mainstream? unusual? (cultural status)

Virtual Reality Mixed/Augmented Realityphotorealism CG


these strategies still operate today in the digital realm



Photographic realism: VR and AR

Ontological status of VR? uses both perspective and motion but NOT a record of what happened “in the light” yet VR subscribes to principal of transparency employs first-person POV

Ontological status of MR/AR shows user the physical world supports multiplicity of representational modes physical world + computer graphics can employ first-person POV aura of place


How does this relate to VR or VE? what is the ontological status of VR?What is the status of VR or VE given this notion of photography as scientifically assured by the light? because VR rejects the light. yet uses both perspective and motion. movies of things that never were. but what about issues of perspective. VR uses perspective projection to give user sense of being in the scene.[what happens if you try other projections in VR or computer graphics generally? - in particular oblique affine projection. where there is no point of view, but there is a direction of projection. is this a God's eye view?]

yet VR subscribes to the same belief in transparency. goal is to create perfect, seamless illusion. VR is all about point of view.

what about AR? - by definition does not necessarily subscribe to the myth of transparency, since the computer graphics are added to the scene. most AR applications assume that the user distinguishes between the graphics and the real world, but not necessarily. We also have the notion of AR creating a world where the user could not distinguish between the computer graphic part and the physical part.

however, many AR applications are multiple - support heterogeneity. notice how connected to the world AR is. notice the applications like Mackay and Ishii - how these are insisting on the connection between paper or other artifacts and the computer. and therefore heterogeneous.


Aura: Mona Lisa


Now switch to a psychological dimension of media rather than purely perceptual dimension. talk about aura. Why do people line up in Louvre to “see” Mona Lisa


Aura: Mona Lisa (oil reproduction)


Oil reproduction is available for sale - $70 on ebay.


Aura: Mona Lisa


giclee repoduction method.


Aura

Walter Benjamin, The Work of Art in the Age of Mechanical Reproduction

Background The Frankfurt School: Marxist critique of culture Horkheimer, Adorno Benjamin loosely associated striving for revolutionary politics of art 1933: moves to Paris 1936: writes “Work of Art”


Now switch to a psychological dimension of media rather than purely perceptual dimension. talk about aura


“Work of Art in the Age of Mechanical Reproduction” history of reproducibility in art mechanical reproduction: photography, film reproducibility allows art work to leave its place but work loses aura aura - presence is time and space uniqueness of object historical circumstances help determine modes of perception decay of aura <-> change in perception

Aura

Thursday, April 26, 12Issues of media theory and their relationship to our AR design projects.


“Work of Art in the Age of Mechanical Reproduction” (cont.)

uniqueness of art: embedded in tradition

art originated in ritual cult object always remains distant, no matter how near same quality for aura of object

in era of mechanical reproduction exhibition value replaces cult value happened early in history of photography 19th century debate over photography as art

Aura


we discussed issue of photography versus art last time



comparison of theater actor and film actor film actor loses aura, unlike a stage actor

film responds to shriveling of aura by building up cult of personality of film stars POV of camera

painters maintains distance film camera is immediate, penetrates reality

Aura


But what about the aura of the Hollywood star system -- this to Benjamin is apparently a kind of false or ersatz aura.


Chaplin


But what about the aura of the Hollywood star system -- this to Benjamin is apparently a kind of false or ersatz aura. Yet he thought that film would destroy aura. Notion of penetration and filmic space.


Aura: really ever lost?


Film was often acknowledged as the popular modernist medium



Film is art of the masses unlike avant-garde painting Dada sought to destroy aura, make contemplation impossible Film achieves on popular scale what Dada set out to do

Epilogue on Fascism aesthetization of war (Marinetti) communism responds by politicizing art

Aura

Thursday, April 26, 12Film was often acknowledged as the popular modernist medium


Aesthetization of Politics

Thursday, April 26, 12Film was often acknowledged as the popular modernist medium


Aura and (digital) media

aura: associated object or place cultural and personal significance for user VR, like film, seems to diminish aura because CG world is complete, seamless all places and objects are mediated yet some aura may remain aura of place can be maintained in AR because user experiences place as unmediated

Aura


VR and screen-based experiences like videogames seem to diminish aura


Oakland Cemetery Experience: which has (most) aura?

Prose description (guidebook) of Cemetery

Photograph of Cemetery

Film (TV) presentation of Cemetery (like Burn’s Civil War) VR experience of Cemetery AR experience in Cemetery

Human-guided tour in Cemetery

Aura

Thursday, April 26, 12It would seem that the AR experience and the human-guided tour would have the most aura, unless these two failed to take advantage of the inherent possibilities


Photograph of Cemetery (aura because of “poor” quality)

Aura


Grainy b/w photograph has more aura than a contemporary color picture or a computer graphic


Audio tour

Aura


Showed you this before: Auratic or not?


Aura in traditional artHollywood cinema

Aura in Locative media: museum toursFlash mobsAlternate reality games?

Aura as a design parameter


None of you are doing narrative experiences - get them to consider this? None reallly using the aura of place either. Why not?

what about the social dimension


Presence in VR Definition (Sheridan, Steuer etc.): “the feeling of being there” Can presence be measured subjectively (user reports) Slater et al. objectively (galvanic skin response etc.) Brooks et al at UNC

Presence and Aura



Presence in VR, AR Definition (Sheridan, Steuer etc.): “the feeling of being there” Alternative definition (Lombard, etc.): “absence of sense of mediation” forgetting about the VR apparatus

Can presence be measured subjectively (user reports) Slater et al. objectively (galvanic skin response etc.) Brooks et al at UNC

Gandy and MacIntyre at GT

Presence and Aura



Lombard and Ditton, “At the Heart of It All: The Concept of Presence” “Although the conceptualizations discussed above vary considerably,

they share a central idea. Each represents one or more aspects of what we define here formally as presence: the perceptual illusion of nonmediation. The term "perceptual" indicates that this phenomenon involves continuous (real time) responses of the human sensory, cognitive, and affective processing systems to objects and entities in a person's environment.”

Presence and Aura



Lombard and Ditton, “At the Heart of It All: The Concept of Presence” “The illusion of nonmediation can occur in two distinct ways: (a) the

medium can appear to be invisible or transparent and function as would a large open window, with the medium user and the medium content (objects and entities) sharing the same physical environment; and (b) the medium can appear to be transformed into something other than a medium, a social entity.”

Presence and Aura



Gandy and MacIntyre et al: “Experiences with an AR Evaluation Test Bed: Presence, Performance, and Physiological Measurement”

This paper discusses an experiment carried out in an AR test bed called "the pit". Inspired by the well-known VR acrophobia study of Meehan et al. [18], the experimental goals were to explore whether VR presence instruments were useful in AR (and to modify them where appropriate), to compare additional measures to these well-researched techniques, and to determine if findings from VR evaluations can be transferred to AR. An experimental protocol appropriate for AR was developed....

The Pit




...The initial experimental findings concern varying immersion factors (frame rate) and their effect on feelings of presence, user performance and behavior. Unlike the VR study, which found differing frame rates to affect presence measures, there were few differences in the five frame rate modes in our study as measured by the qualitative and quantitative instruments, which included physiological responses, a custom presence questionnaire, task performance, and user behavior. The AR presence questionnaire indicated users experienced a high feeling of presence in all frame rate modes...

The Pit




...Behavior, performance, and interview results indicated the participants felt anxiety in the pit environment. However, the physiological data did not reflect this anxiety due to factors of user experience and experiment design. Efforts to develop a useful AR test bed and to identify results from a large data set has produced a body of knowledge related to AR evaluation that can inform others seeking to create AR experiments.

The Pit



Presence as sense of “being there” should be easy (almost automatic) in AR experience because user IS present in the place

Presence as absence of mediation, transparency? not usually striving for absence of mediation

Aura may or may not be operative in MR and ARpossible in historic locationsor with narrative experiences

not with heterogeneous mediated experiences and games

Aura and Presence in MR and AR?


What is the relationship to AR


Nerdherder

What are the experience design issues in this game?

Based on the video, how could the game be elaborated and refined?

Nerdherder




Video actors in Argon

Imagine one or more experiences that could use this technique effectively?

When would video be better than animated 3D? When would animated 3D be better?

Video (vs. 3D0




Project Glass

Are google glasses the future?

Are they AR?

What are the interface and experience issues that are highlighted in the video?

What issues are ignored or downplayed?

Google glasses



Media Theory and MR Design

Ontology of the ImageAuraPresence...

Liveness


So we have looked at three media concepts and now look at a fourth. the concept of liveness and its relationship to MR design


Theory and Practice

How does media history and theory relate to design? Larger question of relationship of theory to practice Critical Theory Film theory Cultural studies of visual culture Communications theory etc.

Contrast this to “productive” theory e.g. graphic design theory or color theory or HCI


Issues of media theory and their relationship to our AR design projects. The larger question is the relationship of all forms of critical theory to the practice of new media design. Engage that larger question briefly. Critical theory in the humanities and social sciences differs from productive theories which aim to improve practice. Critical theory may seek to show the dangers of mass media or new media without necessarily proposing anything that could improve the practice. For example, may believe that new media are hopelessly enmeshed in global capitalism.

We will not address these questions in this course. but do what to see whether some background on a few key terms can help us.

Historicity of our assumptions. because these assumptions are changes are


Presence and Liveness papers

the sense of “being there” is the original definition

Presence as Transparency


Brings together the concept of presence and the concept of the ontology of the photographic image. Use Matrix as allegory. the allegory the desert of the real.

Matrix and notion of transparency. VR world is reversed.

instructors: jay bolter, lcc blair macintyre, coc some...

Documents