-~-~~

/ ~

J~" .~\

~~~=~G~~~~~~)~_~-

^ ^~~^~

, `

~ ~

^, `~~~~«~~ Q~~~~

~~ ~~'

~eux~ \u/r~~

- ~~/~`F~^

"~~l~~,~~Y~p~~ ~~~~

s L1

1~~~~'~ ~

~

` ` . ~ ~~~~

\

~~ ~ ~~ ~~ ' ~ ~ ~ ~

~ ~ ~ ~

CINEMA AND THE CODE

Gene YoungbloodCommunication Arts DepartmentThe College of Santa FeSt . Michael's DriveSanta Fe, New Mexico 87501 USA

ABSTRACT : The author and his colleagues suggest a criterion forevaluating artistic achievement in the medium of the digital,moving image as distinct from other forms of cinema . Thiscriterion is the extent to which the formal possibilities ofdigital imaging are employed as syntactical or linguisticelements, not simply as "special effects ." Four digitalimaging techniques are discussed as possibilities for a newsyntax and, hence, for the expansion of cinematic language .

What are the implications of digital imaging for the evolution of

cinematic language? Since 1986, Peter Weibel, Steina and Woody

Vasulka and I have been meeting to discuss that question .[1] We

thought our talks might become a book, whose subject Weibel

conceived as "the evolution of the image through the digital

image ." What follows is a superficial and incomplete outline of

our conversations, hastily assembled for this publication from two

hundred pages of transcript . It is in every sense a first draft, a

working paper . We are quite aware of the problematic nature of our

discourse, especially in the cursory form presented here . Every

conclusion is vulnerable to criticism, which we welcome . We are

certain of only one thing : that these questions are important and

need to be explored .

YOUNGBLOOD/CINEMA AND THE CODE

PAGE 2

The subject of "digital imaging," we agreed, exists in the context

of both video and computer (different only in the source of the

image and the possibility of real time operation), and covers the

generic areas of image processing, image synthesis, and writing or

organizing digital code in a procedural or linguistic fashion . [2]

But in every case, when we are referring to the phenomenology of

the moving image we call it cinema . For us it is important to

separate cinema from its medium, just as we separate music from

particular instruments . Cinema is the art of organizing a stream

of audiovisual events in time . It's an event-stream, like

music .[3] There are at least four media through which we can

practice cinema --- film, video, holography, and structured

digital code --- just as there are many instruments through which

we can practice music . Of course each has distinct properties and

contributes differently to the theory of cinema, expands our

knowledge of what cinema can be and do . Each new medium modifies

and extends the linguistic possibilities of the moving image,

subsumes the syntaxes of previous media without negating them .

Thus, the basic phenomenology of the moving image --- what Vasulka

calls "the performance of the image on the surface of the screen"

--- remains historically continuous across all media . Digital code,

for example, has radically altered the epistemology and ontology

of the moving image, but has not fundamentally changed its

phenomenology . There are no digital images that have not been

/CINEMA AND THE CODE

PAGE 3

prefigured in painting, film and video . With the code we can only

summarize them, elaborate and unfold them, exercise modalities .

Vasulka calls the code a variation machine . There are no new

classes of images, there are only new variations, and new

epistemological and ontological conditions for generating and

witnessing those variations . Each new medium of the future, said

Vasulka, can only "play host to the phenomenology of the moving

image," which will evolve through that medium to the next,

accumulating the language of each .

Weibel put it this way : a medium is "a corpus of aesthetic

strategies" inherited from previous media . He recalled that in

the 1920s mathematicians had attacked the problem of foundations :

What was pure logic? What was an axiom? Today the answers to

those questions are implemented in the computer . Logical concepts

have become instrumental, they have become parts of machines . And

any machine element, said Weibel, is nothing but a physical.

implementation of a formal device . It implements mental strategies

into something physical . (This is what Buckminster Fuller meant

when he defined technology as "instrumented or documented

intellect .") Similarly, aesthetic strategies invented one hundred

years ago in photography and cinema -- scaling, perspective,

positive/negative reversals, wipes, mattes, -- have now become

machine elements whose operations are trivially invoked through

the preset button . It is a question of primitives . The code is a

metamedium ; through it, high-level aesthetic constructs from

YOUNGBLOOD/CINEMA AND THE CODE

PAGE 4

previous media become the primitives of the new medium . This

influences which aesthetic strategies will be emphasized . When a

strategy that was possible but difficult in film becomes a preset

button in video, or a command in computer graphics, it tends to be

used more frequently . But that doesn't make it more meaningful .

The challenge is to turn "effects" into expressions, into

syntactical units of meaning .

This raises the question, how has the corpus of aesthetic

strategies inherited in a medium like photography or film gone

over to electronic media, and especially to the code? Things are

possible in the code that were not possible, or at least not

easy, in film and video . Only by comparing formal devices

developed in one medium to other devices developed in other media

can we arrive at criteria for evaluating artistic achievement .

Have the syntactical and linguistic possibilities of the digital

image been identified and elaborated in practice? We think not

--- at least, not very often . We rarely find them in work that is

otherwise admired in the name of the medium . People praise a

particular work of "video" or of "computer art," and yet we find

in this work no definitory elements of video or of the code . It'

may be great cinema but it is not great plggtEggig cinema . We are

not arguing for exclusivity or essence . We are not trying to be

the Clement Greenberg of the code . The phenomenology of the moving

image remains constant across all media, but each new medium

brings about a shift of emphasis or accent . Through the code, we

YOUNGBLOOD/CINEMA AND THE CODE

PAGE 5

can unfold the potential of formal strategies that were possibile

but limited in previous media, thereby expanding the richness of

cinematic language .

Vasulka asked, "Who creates the language of a medium?" Weibel

responded by quoting Heidegger : "Man is but a guest in the house

of language ." Vasulka agreed . All possibilities of a system, he

said, are contained within that system . We are not free to invent

the language of film, video or computer . The language already

exists in the system . Our task is to discover it, identify it,

draw it out and name it, put a nomenclature on it . Vasulka built

his machines in order to discover "the language" in them, which

could be found only through dialogue with the machines . He points

out that this is not unique to electronic cinema . Film language

also arose from a similar systemic understanding . As a syntactic

device, the cut, the edit, is machine-bound . It is the only way to

splice film . The most important figures in the history of film are

those who elaborated its syntactic or linguistic potential . This

is our criterion for artistic achievement in the new medium : to

what extent does the artist articulate and develop the formal

possibilities of the system as syntactical or linguistic elements?

To what extent does the artist transform effects into expressions?

YOUNGBLOOD/CINEMA AND THE CODE

PAGE 6

It is a

ion not only of the evolution of cinematic language,

but of human perception itself . Human vision, Weibel pointed out,

has always been "machine assisted ." The invention of perspective,

for example, was machine-dependent . It was derived from optical

instruments . Qurer"s boxes were in this sense "machines ." They

implemented physically what then became formal strategies . With

the help of this machine we could invent perspective . (Weibel

thought this curious . Why did it take so long?) Similarly,

Vermeer, under the influence of Spinoza and the science of optics

in the 17th century, created paintings that were not initially

seen as poetic . They were regarded more like scientific research .

(In the 19th century, Proust, influenced by photography,

"rediscovered" Vermeer, now regarded as a poet . The computer is to

the artist of today as the lens was to Vermeer .) The

Impressionists too were following theories, not subjective

experience . Impressionism was based on color theory : three

different colors produce a fourth impression . Am optical theory of

color, said Weibel, is also a machine, a mental machine, like a

Turing machine . So we have substantial evidence that the evolution

of vision is dependent on machines, either mental or physical .

Coming to such an end that it was not possible any more to

suppress the machine part of it : first the camera, now the

computer . This is significant, Weibel thought, because art always

/CINEMA AND THE CODE

PAGE 7

tries to suppress the influence of the machine element in the work

itself . It's not art if the technology is too apparent . But the

issue here isn't art, it is language and perception . They co-

evolve only to the extent that the syntactic possibilities of

technological systems are made the subject of aesthetic inquiry .

The following formal possibilities of digital imaging are

available for articulation as syntactic elements or linguistic

primitives : (1) image transformation, (2) parallel event-streams,

(3) temporal perspective, and (4) the image as object .

Image Transformation

If mechanical cinema is the art of transition, electronic cinema

is the art of transformation . Film grammar is based on transitions

between fully-formed photographic objects called frames . It is

done primarily through that collision of frames called the cut,

but also through wipes and dissolves . In electronic cinema the

frame is not an object but a time segment of a continuous signal .

This makes possible a syntax based on transformation, not

transition . Analog image processing is one vehicle of this

particular art --- for example, scan processors . But it becomes

even more significant in digital image synthesis, where the image

is a database . One can begin to imagine a movie composed of

thousands of scenes but with no cuts, wipes or dissolves, each

image metamorphosing into the next .

YOUNGBLOOD/CINEMA AND THE CODE

PAGE 8

A cut is a cut, but a transforming or metamorphosing operation is

open-ended . There are infinite possibilities, each with unlimited

emotional and psychological consequences . Metamorphosis is not

unique to digital imaging ; it is a familiar strategy in hand-drawn

animation . What is unique is the special case of pbQgE1gl

metamorphosis . It's one thing for a line drawing or fantasy

painting to metamorphose, quite another for a photographically

"real" object to do so . Yet even this is theoretically possible in

mechanical cinema . It has been prefigured, but never fully

realized, in hand-drawn animation, where it would be so difficult

and time consuming that it is, for all practical purposes,

impossible . It is possible digitally, because the code allows us

to combine the subjectivity of painting, the objectivity of

photography and the gravity-free motion of hand-drawn animation .

Steina pointed out that there are two kinds of transitions based

on the cut, and these will require different kinds of

metamorphoses . One moves us to a different point of view in the

same space/time, the other moves us to a different space and/or

time . In flashbacks (cinematic memory), a matte is used within the

frame, or else the whole frame dissolves . With the code, a part of

the frame can metamorphose . This implies an expanded cinematic

language of simultaneity .

/CINEMA AND THE CODE

PAGE 9

Parallel Event-Streams

With the arrival of electronic cinema it became apparent that film

grammar was limited in what might be called its vocabulary of

tenses . It was always mostly "meanwhile" or "after ." For example,

simultaneous events are traditionally signified through cross-

cutting, or what is known as parallel montage . But, Weibel noted,

there was never a formal distinction between a cut to a different

position in space/time (say, between people in conversation) and a

cut between different spaces or times . The distinction has always

been logical or inferential (as in parallel montage), never

formal . Digital code offers formal solutions to the "tense''

limitations of mechanical cinema . Past, present and future can be

spoken in the same frame at once .

There are at least three possibilities : superimposition (overlay),

or simultaneous but spatiallty separate event-streams that are

either framed or unframed . Superimposition has been explored

extensively in experimental film, notably by Stan Brakhage . His

work is the closest cinema has come to the Joycean text . In such

work it is not always possible to consciously identify each image-

stream ; just as it is often impossible to distinguish every voice

in a musical composition . One is disturbed by this only if one is

pmfanmmiXiar with it . Once one learns to read it, the dense text is

a pleasure . Digital code offers possibilities of image-overlay

whose linguistic potential we have not begun to explore .

YOUNGBLOOD/CINEMA AND THE CODE

PAGE 1 0

The second possibility is more familiar : framed parallel event-

streams, such as split screens in film (optical printing) or

floating image-planes in video, done with digital effects devices

like ADO or Quantel . But there is also the possibility of g[lf[gMQ

parallel events occupying different areas of a single image . This

can be seen in the work of the Vasulkas, for example, where

pointilist textures move independently in separate areas of the

frame . Different zones of the image are activated in different

ways in parallel . The Vasulkas accomplish this through digital

image processing . But image* synthesis, through a variation on

metamorphosis, would provide unlimited possibilities for umframed

but separate parallel event-streams in a single frame .

I shall have more to say about parallel event-streams momentarily,

in a discussion of the image as object . Meanwhile, consider that

simultaneity enlarges our concept of a cinematic event . Weibel put

it this way : whereas first we had the industry of the moving

image, today we have the industry of the accelerated image . If

there are three image-planes instead of one, the information

conveyed within the overall frame is tripled, and, furthermore,

each succeeding image destroys the meaning of the previous one .

The information is accelerated so much, in perspective and all

other ways, that the value of "the image" is replaced by the value

of the image-gestalt or image-field .

D/CINEMA AND THE CODE

PAGE 1 1

Temporal Perspective

"The history of every art form," wrote Walter Benjamin, "shows

critical epochs in which a certain art form aspires to effects

which could be fully obtained only with a changed technical

standard, that is to say, in a new art form ."[4] Weibel pursued

this logic in reverse, working backward from the digital image to

find desire for its powers in art history . He began by noting that

Renaissance perspective was always at eye level, always one point

of view, one vanishing point . By 1850, photographers were climbing

shooting down into streets . Twenty

painted a balloon-suspended eye moving

was no longer bound to a static point

the same period, the

painter Kaspar David Friedrich painted mountain

at an angle different (that is, displaced in time)

from that of the impinging sunlight . Other examples are El

Lissitshy and the cubo-futurist movement . Painting, influenced by

photography and cinema, introduced multiple points of view and

implied time .

onto Parisian rooftops and

years later, Qdillon Red«xn

up to the sun . Perspective

of view . It had

German Romantic

shadows falling

become free-floating . In

And what did cinema do with perspective? Not much . Bound to

psychological realism, it exploited it only spatially, mainly

through deep focus (Eisenstein, Welles, Renoir), never temporally .

Only in experimental cinema was temporal perspective explored in

any serious way at all --- the outstanding example being the work

/CINEMA AND THE CODE

PAGE 1 2

of Michael Snow, such as Lg ReoigU Cgnt[glg and Bgqk and

.

But with the advent of the code, the emphasis on perspective

returns . Moving image art can now embrace it in an emphatic way .

When the image is a three-dimensional database, perspective

becomes a temporal as well as spatial phenomenon . It's a strategy

that is intrinsic to the code . Painters, photographers and

filmmakers could not realize the full potential of this desire .

But now we can unfold and elaborate that which could only be

indicated in earlier media .

Vasulka noted that if we remove the two cinematic vectors from

earth to space and establish the principle of a point in space,

arrive at two possibilities : first, cinema looks from one point

infinity in a spherical point of view . That's one vector, shall

say . The other is the opposite: you look from each point in space

into a single point . If all these points are in motion around one

point, that's the space in which ideal cinema operates . But as

long as we are talking about psychological realism we will be

bound to an eye-level cinema .

The Image As Object

There are three technologies through which the image can become an

image synthesis and three dimensional

(stereoptic) or holographic . The code is

two and may be partially involved in the

object : image processing,

display, either binocular

responsible for the first

third . This is another aspect of parallel event-streams . We

we

to

we

WIy4EMA AND THE CODE

PAGE 13

recognize cinema as frame-bound and frame-unbound . Mechanical

cinema is characterized primarily by its reliance on the frame . It

can't leave the frame unless a special effort is made through

optical printing . But with code it becomes a trivial matter to

remove the image from the frame and treat it as an object . In

digital video it's still only a two-dimensional object, an image-

plane, because those tools have no capacity to deal with the

geometry of the image itself, only with its location or position

(its "address") within the larger frame . The use of framed

parallel events points to new narrative possibilities, new

semiotic strategies . For example, the possibility of a previous or

future event appearing spatially behind or in front of a current

event within the same frame . There is always a pending image .

Editing can be avoided entirely --- as Vasulka did in his 1987

work AQ gf MeMeCy . He points out that through hierarchies of

image planes in particular arrangements "in a mental space,"

future and past tenses may be suggested . As already mentioned in

the discussion of parallel event-streams, conventional film

language is rather inarticulate in this respect . There's no

temporal eloquency in film . But digital video suggests the

possibility of establishing one image-plane as "present" with

other time-planes visible simultaneously within the frame . This

would extend the possibility of transfiguration (metamorphosis)

YOUNGBLO8D/CINEMA AND THE CODE

PAGE 1 4

into a narrative space composed of layers of time, either as

moving or still images . Ed Emshwiller's Sgngtgnp was one of the

first works to explore these possibilities . In it the image

becomes object, and it has both framed and unframed parallel

event-streams .

When image becomes object in a stream of parallel events, the

realm of psychological realism or photographic truth is abandoned .

The frame-bound photographic image brings us truth . But three

image-planes within a frame lose what Vasulka calls "the aura of

truth ." We detach ourselves from them psychologically . Will it be

possible to construct a psychological space in a language of

frame-unbound parallel event-streams?

For Weibel, all this raises a fundamental challenge to the

metynomic nature of cinematic language . He invoked the name of

Roman Jakc»bson, who argues that there are only two fundamental

operations in language : metaphor and metynomy . And the language of

cinema is not metaphoric, it is metynomic . It is the language of

the part for the whole . All cinematic images are contingent . The

frame, said Jakobson, is always part of an unseen whole. At its

fundamental syntactic level --- the level of cutting, of editing,

of bringing spaces together --- the filmic language game is

metynomic . In the service of psychological realism, conventional

editing reconstructs "real" time and "real" space, following'

logical causal chains by metynomic association . Experiments like

YOUNGBLOOD/CIWEMA AND THE CODE

PAGE 1 5

Lalt QgC gt Mar

were attempts to transcend that limitation

within psychological narrative . But in the electronic image

there's no need to make a MgEigQQd, because it's so clear that we

don't have that constancy of time and space any more . Once you put

an image-object against a reference, the metynomic tension is

lost . Objectifying the image within the frame puts it in a

different time zone . Metynomy becomes problematic . On one hand,

such constructs are not metynomic because the space they occupy

isn't "natural ." The image-object is not part of the whole, no

longer contingent . But it's not metaphoric, either . It's something

new . W** don't know what it is . It might still function

metynomically, but in a different way . This is an important area

that is wide open for aesthetic exploration .

The second level of the image as object is achieved through

digital image synthesis . Here, because it is a three-dimensional

database, we can control not only the location of the image-object

within the frame but also its perspective, its angle of view, its

geometry . As a result, the synthesized image thus becomes truly an

object, the witness becomes a "user," and the relation between

them becomes not observation but interaction . Jean-Louis Baudry

argues that in the cinema of psychological realism, the primary

identification of the spectator is not with the characters but

with the camera itself .E53 But in interactive image synthesis, the

spectator il the camera . Since it is not separate from the scene

it surveys, the virtual camera is neither a voyeur nor an

YOUNGBLOOD/CINEMA AND THE CODE

PAGE 1 6

instrument of surveillance . "It is a point of view that is active

qithbI the scene," writes Catherine Richards . "Not only can this

camera (the user) direct its own looking, it can be sensed,

responded to, and represented in the scene : it sees and is

seen ."[6]

The third level of the objectification of the image is realized

through three-dimensional display . Whether through holography or

binocular (stereoptic) technology, cinema is moving from the two-

dimensional image on a screen to the three-dimensional object in

space . Today cinema represents reality ; tomorrow it will Q

reality . Already with stereoptic technology the image becomes an

object . And in Scott Fisher's virtual environment project for NASA

(combining a three-dimensional database with stereo vision in a

wraparound head-mounted display), cinematic space becomes a place

to live . An unframed image isn't an image, Vasulka pointed out,

it's am object in space : "It forces you to deal with air ." It is

no longer a representation but the thing itself . Vasulka noted

that different understandings of reality and truth are implied by

the representational image and by an object in space, no matter

how insubstantial that object may be . Three-space cinema, he

suggested, is more like theatre. In (two-space) cinema there is

truth but no reality . In theatre there is reality but no truth .

-t., .

/CINEMA AND THE CODE

PAGE 17

4w ~ /

Notes

1 . Peter Weibonl is a videomaker, mathematician, art historian,writer and professor of art and electronic media in Austria andthe U .S . Steina and Woody Vasulka are internationally knownvideo artists who founded The Kitchen in New York City in 1970as one of the world's first presentation centers for electronicart . Gene Youngblood is author of EKgg[ldgd CingM, (1970), thefirst book about video as an artistic medium .

2 . Both real-time video machines and computers operate on the samestructure of digital code . An ADO or Quantel or Fairlight aredigital computers . The only difference is that they take their"model" from camera input, and they operate in real time . Withthe exception of extremely fast computers, most digital imagesynthesis, or "computer graphics" is not done in real time .Other than this we make no distinction between them, except inreference to the source or model of the organization of theimage --- one through camera input, the other throughalgorithms . Also, we regard the process of writing orstructuring the code as part of the digital imaging procedure .It's the craft of digital imaging in computer graphics . Youdon't "write the image" in video .

3 . My colleagues found the concept of the "event-stream"problematic . Vasulka defined it as "every scheduled change ." Hepointed out that there is always an invisible technologicallevel to every perceived event, like the event of line-formingin video, or computations and logical operations in imagesynthesis . The key is to realize that the event does not haveto be consciously perceived . In music, for example, a listenerwould be incapable of naming each sonic event, but music isnevertheless a system of parallel event-streams .

4 . Walter Benjamin, "The Work of Art in the Age of MechanicalReproduction," IllgMiggtignp, New York, 1978, p . 237 .

5 . Jean-Louis Baudry, "Ideological Effects of the BasicCinematographic Apparatus," Film MACKAY, vol . 28, no . 2(Winter, 1974-75) . Quoted in Christian Metz,"History/Discourse : a note on two voyeurisms," in ThggEigg gfAUtbgCahin, ed . John Caughie, London, 1981, p . 231 .

6 . Catherine Richards, "Virtual Worlds, Digital Images," in thecatalog of the 1987 AMgQggq FilM Institgtq Vidgg Fggtiygl, LosAngeles, 1987 .