connectionism and ontological...

Peter Gaffney / PHF 2009-2010

Connectionism and Ontological Realism

(Or “This is a tank.”)

The appearance of the mirror already introduced into the world of perception an ironical effect of trompe-l’oeil, and we know what malefice was attached to the appearance of doubles. But this is also true of all the images which surround us: in general, they are analyzed according to their value as representations, as media of presence and meaning. The immense majority of present day photographic, cinematic and television images are thought to bear witness to the world with a naïve resemblance and a touching fidelity. We have spontaneous confidence in their realism. We are wrong. They only seem to resemble things, to resemble reality, events, faces. Or rather, they really do conform, but their conformity is diabolical.

Jean Baudrillard, “The Evil Demon of Images” (1987)

Representations are bodies too!

Gilles Deleuze and Félix Guattari, A Thousand Plateaus (1987)

PART I: Computer vision and the problem of “investiture”

The problem

When you ask somebody in the field of computer vision about the limitations of neural

networks, they are likely to tell you this story1: In the 1980s, the Pentagon implemented a plan to

equip each of its tanks with a device that would scan the visual field for enemy tanks and alert

the driver if it saw anything suspicious. The plan looked feasible enough as far as hardware goes

(a camera connected to a computer), but when it came to writing a program capable of

deciphering the complexity of the visual field in real time, they found that conventional

algorithm-based data processing was ill suited to the task. Making matters worse, the device

would have to be capable of detecting enemy tanks hiding behind trees or other objects in the

visual field. So they opted for an artificial neural network, a type of program that is designed to

mimic the way biological neurons approach the task of pattern recognition (and other perceptual-

cognitive functions). We will see in a minute how this differs from algorithmic symbol-based

programs, and how neural networks gain a key advantage when employed in computer vision. In

the case of the tank-detection program, it is important to note that neural networks, unlike

conventional programs, do not start out with a complete understanding of how to execute a

particular task; they have to be trained. So the programmers took 100 photos of tanks hiding

behind trees and another 100 of trees without tanks. They fed half of all the photos into the

neural net and put the other half aside for testing the program when training was complete. Each

time a photo was presented to the net, it was asked if there was a tank hiding behind a tree.

1 I based my version of the story on the article “Neural Network Follies” by Google software engineer Neil Fraser. The same article is cited in Priddy and Keller, Artificial Neural Networks: An Introduction (Bellingham, Washington: The International Society for Optical Engineering, 2005), and Crochat and Franklin, “Back-propagation neural network tutorial” (http://ieee.uow.edu.au/~daniel/software/libneural).

Gaffney, "Connectionism and Ontological Realism" Page 2

Initially, the program was only able to produce random guesses, since it did not even know what

it was supposed to do, let alone what features might make a tank visually distinct from a tree.

Distinctions like these (the task itself, in essence) must be “taught” to the neural net. This is done

by changing the way excitement flows from one “neuron” (or unit) to another⎯that is, by

modifying the “weights” of all connections within the network until it consistently produces the

desired distribution of electrical excitement for a given input. Eventually, the program was able

to identify every photo in which a tank was hiding behind a tree. It performed equally well when

it was given the 100 photos initially set aside (testing stage). But the programmers wanted to be

sure, so they returned to the field and took 100 more photos. This time, however, the answers

that the neural net generated appeared to be completely random. What the programmers had

failed to consider is that the initial 100 photos with tanks had been taken on a sunny day and all

the others when it was cloudy. Guided by no symbolic representation on which to base its

deductions during the training stage, the neural net had been looking all along at the color of the

sky. As Neil Fraser puts it, “The military was now the proud owner of a multi-million dollar

mainframe computer that could tell you if it was sunny or not.”

What does this story tell us? At first glance, it seems to illustrate a fundamental flaw with

neural nets: we just do not know what they learn.2 But what about the programmers? They had

neglected to control a key variable in the training period (weather) that just happened to coincide

with the target pattern (hidden tanks). This may seem like an obvious error⎯though, apparently,

not an uncommon one3⎯, but we can imagine other, subtler patterns that a programmer might

2 It is worth looking at Neil Fraser’s exact words here: “This story…is a perfect illustration of the biggest problem behind neural networks. Any automatically trained net with more than a few dozen neurons is virtually impossible to analyze and understand. One can’t tell if a net has memorized inputs, or is ‘cheating’ in some other way.” 3 Rolf Lakaemper, a computer vision and robotics expert who first brought this story to my attention, added that programmer errors of this magnitude happen all the time.


inadvertently feed to a neural network during the training period, patterns that would not

necessarily be as obvious as the weather. We can even imagine patterns that emerge as a

determining factor in the regular behavior of a neural net without ever becoming detectable to

the programmer⎯until some catastrophic later moment when the two patterns suddenly diverge.

It is plausible, for example, that birds partially hidden in the photos of trees were not present in

the ones with tanks. Such a coincidence might initially help a neural net produce the desired

output, until the day when birds are no longer afraid of tanks, or some combination of season and

terrain means there are no birds to begin with.

But this is precisely what is so striking about neural nets. At no point in the training

period can we say that they learn to recognize the target object⎯indeed, we cannot say they

learn to recognize any object at all, not even the color of the sky, so long as we define these

things first and foremost as symbolic representations, each one with a finite sets of attributes.

What a neural network lacks is not the capacity to react consistently to certain patterns, but the

power to treat intersecting patterns as discrete entities. We have already gone too far when we

say that the tank-detection program can tell us if it is sunny or not, since it could just as well be

some other pattern that intersects with fair weather, one that is generally present⎯but not limited

to⎯the collection of attributes that we call a “sunny sky.” The most we can say about a neural

network, in this sense, is that its numerous connections can be modified to distribute electrical

impulses in a certain pattern every time it encounters some other pattern (but which?) in the real

world. By the same token, we cannot say that responses produced by the tank-detection program

during the final phase of testing were random in any sense of the word. They only appear random

from the point of view of a programmer who is tracking the features of a different object.


From basic neural nets to adaptive-subspace self-organizing maps (ASSOM)

In their basic conception, neural networks designed for pattern-recognition do not differ

greatly from conventional feature extraction and mapping. Early designs for these networks

provided a new platform for processing complex data, one that led to a new paradigm in

cognitive science and artificial intelligence, namely connectionism. But it was not until the early

1980s that Teuvo Kohonen, a Finnish physicist working on the problem of adaptive learning in

neural nets, developed a method for taking full advantage of its novel characteristics.

Connectionism gets its name from the numerous synaptic-like connections that join one layer of

units to another in a neural net. In a simple “feed-forward” network (fig. 2), units on an input

layer are activated in a certain pattern (+1, -1, -1, +1). Each one of these units sends a charge to

each other unit on the output layer, but the strength of that charge (the amount of activation)

depends on the weight of each connection. It is by modifying these weights that a programmer

can “train” a neural net to produce a desired pattern when it is given a particular input. Since it is

the programmer who changes these weights (according to a technique called “backpropagation”),


it is not clear at first why the proper weights are not simply calculated in advance and

incorporated into the design of the machine. But this can be explained in part by the fact that

many connectionists consider their research as a basis for bridging the conceptual gap between

artificial intelligence and cognitive science. Neural nets provide a functional model that explains

some of the same adaptive properties as real neurons in the brain, and therefore bring new insight

to key issues in both fields.

But is there any practical advantage to be gained by reaching the appropriate

configuration of weights over many (sometimes thousands) of training cycles or “epochs”? One

answer has to do with the capacity for neural networks to memorize and implement many

different patterns. A simple two-layered network designed by James McClelland and David

Rumelhart was able to match four eight-digit patterns using eight input units and eight output

units (Bechtel and Abrahamsen 1991, 2002: 93). This is remarkable considering the similarities

between patterns, which consisted only in positive and negative integer activation values, as in

the example above. Even more impressive is the network’s ability to cope with distorted input

data. William Bechtel and Adele Abrahamsen built a two-layered network but simulated

distortion by adding a random value between 0.5 and -0.5 to the activation of each input. In only

50 epochs, the network produced the appropriate output for each of the four patterns with a

margin of error of less than 0.2. In other experiments, they reversed the sign of one of the units

or gave it an input pattern it had never seen before. In both cases, the neural net performed well,

with nearly error-free output in the first case and plausible generalized output in the second (i.e.

it chose the pattern from the four it had learned that most resembled the new pattern).

Examples like these demonstrate how complex pattern-recognition tasks can be carried

out with relatively few units simply by shifting the work of computation to the connections


between them. But it is difficult to imagine how this principle could be directly applied to real-

life pattern-recognition devices like the one designed to detect tanks. For one, we have been

looking at models for neural networks in which the units are all uniform, and uniformly

distributed. How would such a network approach the task of selecting out those features that

belong to a tank or any other object, so that it could distinguish it from other objects in the visual

field? The easiest solution is to design each input neuron to detect a different feature and then

relay it to another layer where it can be incorporated into higher representations: this is called

“feature extraction.” In 1959, Oliver Selfridge designed a feature-extraction and processing

principle called “pandemonium” that is a forerunner of modern Optical Character Recognition

(OCR) software. In the diagram below (fig. 3), we see how competition among the feature

“demons” leads to correct output.


As mentioned earlier, Kehonen’s approach to pattern-recognition in neural nets, called

adaptive-subspace self-organizing maps (ASSOM), differs in several ways from the examples

above:

(1) Input units are not set up in advance to serve as feature-specific (invariant)

detectors or “demons”; in their initial state, these units are variable and

uniform (like the ones in fig. 2). It is only after many epochs in the training

period that the neural net begins to acquire the capacity to detect micro-

features in a visual field.

(2) Training is unsupervised; this means that programmers have no part in changing

the weights of connections; instead, ASSOM nets are designed with several

layers of self-organizing units whose only job is to record patterns they receive

from the input layer. Weights are changed as a result of competition among

these subspace representations.

(3) This “winner takes all” approach to competitive learning also requires that units

have the capacity to exert an inhibitory effect on other groups of subspace

units. This habituation to regular patterns in its environment allows the neural

network to build successively more complex maps, but it also means that some

patterns will be filtered out by more competitive representations. This is meant

to simulate the way perceptual-cognitive faculties are able to identify the same

patterns when they are partially hidden or transformed with respect to

contingencies in the visual field. (Kehonen 1995, 2000, Kehonen 2003)

Kohonen’s explicit purpose in implementing the principle of self-organization is to avoid any

model in which the mechanism for transforming patterns into representations might be based on


a priori representations: “What we aim at is a model of experimental data, not of the process

variables” (2003: 1178). This may sound like a subtle distinction, but we have seen that it can

mean the difference between a device that detects enemy tanks and one that just connects

ineffably with the world.

The symbolic approach

It will be useful to consider how a conventional symbol-based processor would handle the

same pattern-recognition task as the tank-detection net before moving on to some conclusions

about what this means for a new “ontological” definition of perception. There are two questions

I’d like to pose in this part of my paper: (1) How do symbol-based processors work?; (2) What

can conventional principles of pattern recognition tell us about the (still ambiguous) notion of

symbolic representations, and what does this have to do with what we will have reason to call

“investiture and decoding” (or Baudrillard’s “diabolical conformity”)? In its most basic

conception, a conventional computer program functions by manipulating symbols according to

an algorithm or sequence of instructions. In the case of a program designed to identify an enemy

tank, such an algorithm might proceed from one proposition to another following the decision

tree shown below (fig. 4). From the initial condition (transmission of new data from camera to

computer), data flows through a series of “if…then…” propositions that model the object of a

tank and lead the program to the appropriate safe/alert response. The diagram in figure 4 is a

rather simplified version of what might be required to complete the task of tank detection, but we

could always add more propositions to the sequence.


Rather than leading each part of the data through a series of commands, it might be better to

cluster it into different tasks. Some of these might compare new features to previous maps in

order to make subtler deductions. The diagram below (fig. 5) shows how an actual program

enables a mobile robot to build maps of its environment autonomously. This feature is called

“simultaneous localization and mapping” or SLAM:


What is important in the first example is that any instruction added to the algorithm to further

qualify a “Yes” statement must be inserted between two instructions in the sequence. Similarly,

an instruction or series of instructions used to qualify a “No” would either proceed to a “Safe”

output or back to the next instruction in the main sequence. Propositional logic thus shares its

essential structure with the syllogism, such as the famous “All men are mortal; Socrates is a man;

therefore Socrates is mortal.” So long as the constituent propositions are true, the final deduction

must also be true; it is a “truth-preserving” device. Of course, we don’t need a data processor to

deduce that Socrates is mortal; this is already built into our higher representations of Socrates, if

not immediately given in the attribute which tells us that Socrates is already dead. But not all

syllogisms are so straightforward. We can add as many premises to the syllogism as we want,


creating more stipulations in a chain of reasoning (one that goes beyond the simple task of

pattern-recognition). Here’s a more complicated example:4

Instruction in natural language variables substituted If industries are to be kept going, if B there must be a steady supply of oil: then C If the United States is to prosper, if A its industries must be kept going: then B therefore if the United States is to prosper (therefore) if A there must be a steady supply of oil. then C In order to secure a steady supply of oil, if C the United States must go to war: then D therefore if the United States is to prosper, therefore if A it must go to war. then D therefore if it does not go to war, therefore if not D the United States will not prosper. then not A

SLAM is another algorithm-based program, but the flowchart above (fig. 5) does not

describe the attribution of symbolic values to objects in the visual field. It only shows how the

robot extrapolates its position by comparing new visual data to an existing map (localization),

and to modify the map whenever it diverges from the extrapolated position (mapping). But let’s

say we were hired to modify this robot for an urban combat environment where it must identify

and destroy enemy combatants. We would be in the same position as the tank-detection

programmers, but with a much greater imperative to assign and manipulate the proper symbolic

values for each target object. So long as we were using algorithm-based processing, we would

need to develop a series of compounding stipulations similar to the one above (if A and B and C

and D and E… then X, but only if Y = Z, etc.). This time, there would be a much greater burden

4 This example is a modified version of the one found in Chapter 83 of R. W. Jepson’s “Clear Thinking” (http://www.ourcivilisation.com/smartboard/shop/jepsonrw/chap83.htm).


on the side of the programmers, if not on the program itself, to determine the thresholds between

targeted and non-targeted objects. Is an object in the visual field acting hostile? Is it hiding a

weapon? Is it harboring insurgents? What is at issue here is not the effectiveness of feature

extraction vis-à-vis some partially obscured object in the visual field, but the capacity for

programmers to invest an image with whatever representations will be necessary to complete a

given task. The real question is about human interest and context, not computer vision.

In War in the Age of Intelligent Machines (1991), Manuel DeLanda explores some of the

troubling implications of this issue in the case of military automata:

The PROWLER [fig. 6]…is a small terrestrial armed vehicle, equipped with a primitive

form of “machine vision” (the capability to analyze the contents of a video frame) that

allows it to maneuver around a battlefield and distinguish friends from enemies. Or at

least this is the aim of the robot’s designers. In reality, the PROWLER still has difficulty

negotiating sharp turns or maneuvering over rough terrain, and it also has been deployed

only for very simple tasks, such as patrolling a military installation along a predefined

path. We do not know whether the PROWLER has ever opened fire on an intruder

without human supervision, but it is doubtful that as currently designed this robot has

been authorized to kill humans on its own…For now, the robot simply makes the job of

its human remote-controller easier by preprocessing some of the information itself, or

even by making and then relaying a preliminary assessment of events within its visual

field. (1991: 1)

In its practical application, the computer vision of PROWLER and other automata of its era is

little more than a surveillance camera. But DeLanda goes on to point out that in the military


application of artificial intelligence, “it is precisely the distinction between advisory and

executive capabilities that is being blurred” (2). He gives the example of a war game designed to

help the Pentagon learn which sequence of actions comprise the winning strategy; the problem is

that “SAM and IVAN [the simulated players of the game]…do not have any problem triggering

World War III” (2).

Of course, this problem is not limited to algorithm-based programs, nor does it first

emerge at the level of programming. As we have already seen, the design of any pattern-

recognition device has its starting point in a network of representations that belong to the

designers, to people further up the chain of command (those who conceive and finance the

project), and eventually to a heterogeneous field of social codes which determine the flows of

knowledge, investment capital, raw materials, and so on. Likewise, computer-generated output is

only meaningful in relation to a broader set of representations and to those who are interested in

its results. There is no such thing as a computer that sees, a computer that knows, but this has


more to do with the agency behind it than anything in its design. Such is the predicament John

Searle indicates when he writes: “What [a computer] does is manipulate formal symbols. The

fact that the programmer and the interpreter of the computer output use the symbols to stand for

objects in the world is totally beyond the scope of the computer” (1980: 437). Searle takes the

more extreme view that a computer program is not capable of processing actual images, only

other, non-computer symbol systems: the designations coded in a natural language, for example,

together with the underlying cultural matrix that sanctions how these designations are made.

What we have been calling pattern recognition thus consists in projecting meaningful content

onto a visual field and then interacting with a social and physical environment on that basis.

Programmers may very well have reason to be optimistic about the growing power and

efficiency of computer vision technology. But this coincides with growing concern among Searle

and others that there are ideological consequences of symbol-manipulating technology. By

confusing intelligence and “investiture,” the symbolist approach to cognitive science serves only

to reify our representations by pretending to find them “in the outside world.”

Seeking and finding: the notion of “investiture”

It is precisely in this sense that Victor Burgin5 speaks of a “photographic paradox”

(Thinking Photography). It is worth quoting these texts at length here, since they illustrate how

closely the practical problems we have been discussing resemble⎯because they derive

from⎯the more elusive mechanism of investiture:

5 It should be mentioned that many of Jean Baudrillard’s works (The Evil Demon of Images, The Gulf War Did Not Take Place, among others) trace the same development of a “diabolical conformity” in the mediatization of the event. I have chosen to follow Victor Burgin’s statement of the problem because it so closely resembles the pattern-recognition problem.


The structure of representation⎯point-of-view and frame⎯is intimately implicated in

the reproduction of ideology (the “frame of mind” of our “points-of-view”). More than

any other textual system, the photograph presents itself as “an offer you can’t refuse.”

The characteristics of the photographic apparatus position the subject in such a way that

the object photographed serves to conceal the textuality of the photograph

itself⎯substituting passive receptivity for active (critical) reading. When confronted with

puzzle photographs of the “What is it?” variety (usually, familiar objects shot from

unfamiliar angles) we are made aware of having to select from sets of possible

alternatives, of having to supply information the image itself does not contain. Once we

have discovered what the depicted object is, however, the photograph is instantly

transformed for us⎯no longer a confusing conglomerate of light and dark tones, of

uncertain edges and ambivalent volumes, it now shows a “thing” which we invest with a

full identity, a being. With most photographs we see, this decoding and investiture takes

place simultaneously, unselfconsciously, “naturally”; but it does take place⎯the

wholeness, coherence, identity, which we attribute to the depicted scene is a projection, a

refusal of an impoverished reality in favor of an imaginary plenitude. (Burgin 146-147)

The problem with mechanical reproductions is not that they fool the eye or cause us to confuse

reality with its likeness, but that they generate the comforting illusion, as a function of their

realism, that symbolic representations spring readymade from the visual field. This is what

makes photographs so persuasive. Similarly, if a computer is able to decode an image, if it is able

to provide an appropriate answer every time it is asked “What is it?” this is only because the

visual field has been prepared in advance to respond like a code. The presence of an object is not


“recognized,” any more than a pattern is “decoded.” Rather, these functions⎯truth itself, in

essence⎯arrive later on the scene, as the result of an investment at the level of the symbolic.

Symbolist theories of human intelligence, taking their cues from technological advances in the

field of computer vision, thus proceed by treating the world as if it conformed to the rituals of

modern visual culture. What first confronts the eye as a confusing jumble of dark and light is

subsequently brought into good order by the logical play of representations. But this is not the

same thing as vision. It is only the happy coincidence of meaning and presence, a game of hide

and seek like the one Nietzsche describes in his challenge to the concept of truth: “When

someone hides something behind a bush and looks for it again in the same place and finds it

there as well, there is not much to praise in such seeking and finding. Yet this is how matters

stand regarding seeking and finding ‘truth’ within the realm of reason” (2000: 157).

We should not view such criticisms as mere philosophical wrangling. After all, it is by

the sanction of Allen Newell’s “unified theory of cognition”6 that SOAR, the symbolic cognitive

architecture that drives most of DARPA’s current designs for automata, has begun to incorporate

what DeLanda calls “executive capabilities.” Recent thinking on unmanned weapons points to

the tactical and psychological advantage to be gained by deploying a multitude⎯or

“swarm”⎯of smaller automata against an enemy. DARPA’s newest generation of unmanned

weapons will thus combine automatic targeting recognition with other algorithms meant to

mimic the way birds move in flocks and ants forage for food (Singer 2009: 232-233). As one

DARPA researcher puts it, these swarms might eventually reach the size of “zillions and zillions

of robots” (234). What is troubling about DARPA’s multitude of tiny robots is that each one will

6 It is notable that Newell argues strongly in favor of the view that symbol systems have the necessary and sufficient means for general intelligent action (“physical symbol system hypothesis”), but that “any theory of what Soar will learn must occur within a larger theory of the environments Soar inhabits, how tasks arise, and how they get defined—in short, a total theory of Soar as an agent in the world” (1994: 191).


really have to see. Unlike the PROWLER and other robots of its generation whose deductions

were subject to the discretion of a human operator, the sheer numbers of “Proliferated

Autonomous Weapons” (or PRAWNs) will make this kind of intervention impractical. “Rather

than a controller back in a mothership furiously trying to point and click at which target to

hit…the autonomous swarm would just figure it all out on its own” (234).

There is no need to reconstruct here any of the possible doomsday scenarios that such

automata bring to mind. What I would like to propose instead, in light of this notion of

investiture, is a hypothesis about the ideological tension between two moments that characterize

the social dimension of human perception, and that now plague its technological counterparts

(including many neurally-inspired processors). In the first moment, we find the profoundly

creative act that enables the mind, or perhaps a “collective mind,” to invest the visual field with

an order of its own. Burgin sees this as the projection of a pre-existing symbolic order onto a

chaotic collection of sensory data. We would do better, however, to consider it in broader terms

as a connection between two or more patterns, with the result that perceiver and perceived are

joined together in a determinate and productive relationship. We may even say, following the

example of such thinkers as Henri Bergson and Gilles Deleuze, that this simple intersection of

patterns constitutes the “actualization” of a world together with an intelligence in it7⎯so that the

two sides, perceiver and perceived, are precisely what this relationship produces. (This is a rather

confusing formula, and I will spend the second half of the paper trying to sort out its

implications). If we look closely at the examples above, we see a second moment as well, this 7 I am referring here to Bergson’s notion that “The more consciousness is intellectualized, the more matter is spatialized [Plus la conscience s’intellectualise, plus la matière se spatialise]” (1959). This notion is at the center of the Bergsonian method of intuition. Deleuze and Guattari follow the same formula when they write that “it is by slowing down that matter, as well as the scientific though able to penetrate it with proposition, is actualized” (1994). In his work on the connection between Bergson and Deleuze (Signature of the World), Eric Alliez uses a more explicit formulation: “The diversion [détournement] of Bergosnian (immediate) intuition is prepared both before and beyond…the realism-idealism opposition, when the act of knowing tends to coincide with the act that generates the real” (2004).


time characterized by a certain ideologically motivated sleight of hand. Here, the process of

actualization that creates both intelligence and world is carefully hidden from view. What was

actively produced in the first moment now appears to come readymade from an “outside” world.

Symbolism, as a philosophical doctrine, thus coincides with a visual culture (photographs,

television, cinema) that obscures the first moment by reinforcing the second. Like scientific

realism, it has the potential to actively reduce a heterogeneous complex of social codes to a

monolithic truth⎯a “royal science.”8 As Nietzsche puts it, there is not much to praise in such

seeking and finding.

PART II: Connectionism and ontological realism

Does connectionism preclude representation?

The objection might be raised that the treachery of images is a legitimate concern within

the limits of culture studies, perhaps even the other social sciences, but has nothing to do with

computer vision or science proper. The very fact that an automaton can interact with its

environment, the simple fact of applied science, should be taken as sufficient proof that task-

oriented symbolic knowledge really works. This is effectively the argument made by Newell and

Simon in their defense of the physical symbol hypothesis: “An expression [composed of

symbols] designates an object if, given the expression, the system can either affect the object

8 This concept is mapped out by Deleuze and Guattari in A Thousand Plateaus: “The ideal of reproduction, deduction, or induction is part of royal science…and treats differences of time and place as so many variables, the constant form of which is extracted precisely by the law…[royal science] implies the permanence of a fixed point of view that is external to what is produced” (372).


itself or behave in ways dependent on the object”1 (1976: 116). What right do we have to

question the truth-value of symbolic representations? On what grounds can we, like Burgin vis-à-

vis the photograph, describe them as no more than the effects of an imaginary plenitude? These

are reasonable objections, and we will have no reason to disagree with them.

The problem with symbolic knowledge is elsewhere. We spoke too soon when we said

that it is too full, that it recognizes too many “things” in the visual field. The problem is just the

opposite. In the symbolist conception, intelligence is reduced to a mechanism for verifying the

identity of being (X=X). It is, from this point of view, an epistemological machine, guided by its

truth-preserving structure and characterized by a passive receptivity rather than an active

engagement of patterns in the visual field. This is quite different from the case of the tank-

detection program, where the neural net failed precisely because it was active—too active. It did

not know that the color of the sky was an invalid attribute of the target object. It did not know

and it did not care: as an ASSOM-driven neural net, it was not trained to identify symbolic

representations, only to produce an alert signal in the presence of any pattern common to the

target samples used in the training period. The most we can say about such a neural net is that it

makes connections with the visual field. But it does not do this by investing visual data with pre-

existing symbolic value (even when operators use them as if they do). It would seem then, that

there are two ways of thinking about perception. On the one hand, it can involve the

reconstruction of “meaningless” visual data in a matrix of symbolic knowledge. In the field of

culture studies, Burgin considers how visual media naturalize this process, so that a photograph 1 Newell and Simon further specify that a physical symbol system consists of “a set of entities, called symbols, which are physical patterns that can occur as components of another type of entity called an expression (or symbol structure). Thus, a symbol structure is composed of a number of instances (or tokens) of symbols related in some physical way (such as one token being next to another). At any instant of time the system will contain a collection of these symbol structures. Besides these structures, the system also contains a collection of processes that operate on the expressions to produce other expressions: processes of creation, modification, reproduction and destruction. A physical symbol system is a machine that produces through time an evolving collection of symbol structures. Such a system exists in a world of objects wider than just these symbolic expressions themselves” (116).


or film conveys a reality that is already implicated in a matrix of symbols by the time it is

perceived. Thomas Kuhn proposes a similar theory on scientific breakthroughs, which, he

argues, do not augment an existing body of knowledge, but radically dissolve and reorganize it

around the discovery of new patterns formerly dismissed as statistical variance (concept of

“paradigm-induced gestalt switch”).2 Scientific observation, like all acts of perception, is

constructive, instrumental and historically specific—but is not for that matter less objective.

In each case, we are most likely to ask how symbolic representations give meaning and

presence to sense experience. But what happens when we trace our line of inquiry in the opposite

direction? What role does sense experience play in the production of representations? These are

not two sides of the same coin, but opposing interpretations of human intelligence that are

divided along the lines already discussed in this paper. On one side, symbolists like Newell and

Simon advance the claim that “A physical symbol system has the necessary and sufficient means

for general intelligent action” (1976: 116). Symbolism, for the same reasons as computationalism

in cognitive science and psychology, regards the mind as a kind of Turing machine which

interacts with the material world by performing purely formal operations on symbols. On the

other side, connectionists like Donald Norman and David Rumelhart argue that symbolic

representations and the rules which govern them do not play any necessary role in perception and

cognition, or in any other cognitive task for that matter. In the case of language acquisition, for

example, Norman and Rumelhart argue that “a system can appear to obey and follow general

rules of language even though it does not have those rules within it” (1981: 239). Their research

is particularly interesting since it provides evidence that even when we consciously use symbol

2 In The Structure of Scientific Revolutions, Kuhn writes that theoretical conflicts arising between existent and emergent scientific paradigms are “terminated by a relatively sudden and unstructured event like the [visual] gestalt switch” (122). It should be noted that Kuhn is in some ways critical of the comparison between scientific revolution and visual gestalt theory. He does not go so far, for example, as Norwood Hanson’s Patterns of Discovery: An Inquiry into the Conceptual Foundations of Science.


systems, these systems are not necessarily governed by rules (i.e. propositional logic). Neural

nets that are conditioned to perform basic linguistic tasks also exhibit some very human-like

error patterns during the learning process, such as the “U-shaped” acquisition of past tense

irregular verb forms11 (McClelland and Rumelhard 1988: 117).

DeLanda links the two sides of this debate to the famous controversy over the 29 Eskimo

words for snow.12 Do Eskimos perceive 29 kinds of snow because their linguistic categories “cut

up” reality in this way, or do the words derive from the sensual experience of 29 kinds of snow?

In other words, does diversity in a system of symbolic representations provide for diversity in

sense perception or is it the other way around? Hard line connectionists like McClelland and

Rumelhard claim that representations do not provide the conditions of possible experience. From

their point of view, there is no essential difference between linguistic aptitude and any other

pattern-association/completion faculty. Representations emerge as a secondary effect of

connections between one pattern and another; they are a symptom rather than a cause of complex

behavior—perhaps only a convenient metaphor. Others in the field question the usefulness, even 11 Children who correctly form past-tense irregular verbs in an initial stage will often cease to use them correctly while learning how to construct regular verb endings. In a third stage, the child learns that irregular verbs are a special case, and begins using them correctly once more. For example, a child who initially says “She went to school” might begin saying “She goed to school” during the initial stage of learning the regular verb form. Does this three-stage, or U-shaped, learning process occur because children learn to apply (and then over-generalize) a new rule? Or is learning governed only by exposure and usage, so that no explicit rule is ever internalized or applied? Proceeding from the second hypothesis, McClelland and Rumelhart constructed a neural network to mimic acquisition of the past tense. Results suggest that it followed the same U-shaped learning pattern as children who are learning a natural language (McClelland and Rumelhart 1988: 117). 12 “Everyone knows that Eskimos have 29 words for snow…The question is: Do Eskimos see 29 kinds of snow because they have 29 words for snow? That was the position taken by linguists in the 20th century⎯words cut out experience and in fact give form to experience; linguistic categories shape reality, at least phenomenological reality⎯and therefore their having 29 words for snow is the reason why they can see 29 kinds of snow. That’s one position. The other position would be the opposite: They have 29 words for snow because they can touch, feel, smell, build igloos with, and do all kinds of non-linguistic things with real snow, which comes in 29 kinds because there are at least 29 mixtures of the solid and the liquid….The two options remain quite different. In one, the 29 words are shaping perceptual reality literally. In the other, perception is not linguistic. Perception is multi-modal: it’s about sounds, smells, textures, images, all of which have their own modality and can be combined in different ways. It’s also about intervening causally in the world: to build igloos with that snow, to hunt over that snow, to find your way out of a snow storm…” (DeLanda 2007).


the validity, of continuing to use such a metaphor. Van Gelder, a leading proponent of dynamical

systems theory, compares cognitive faculties to Watt’s steam engine and centrifugal governor

(fig. 7): “when we see the relationship between engine [flywheel] speed and arm angle, we see

that the notion of representation is just the wrong sort of conceptual tool to apply…arm angle

and engine speed are at all times both determined by, and determining, each other’s behavior.”

This relationship is “much more subtle and complex…than the standard concept of

representation can handle” (van Gelder 1995: 353). Bechtel disagrees. New findings by

connectionists challenge our conventional notion of how representations work, but do not

preclude this notion altogether. In the case of Watt’s governor, Bechtel argues, a special

representation of speed is necessary in order to connect one system (referent) with another

(consumer): “Without the spindle arms and their linkage mechanism, the valve has no access to

information about the flywheel speed. They were inserted in order to encode that information in

a format that could be used by the valve-opening mechanism.” Representations thus allow the

state of affairs in one medium to be transmitted in a format that can be “understood” by another.


Deleuze and ontological realism

Put this way, however, there is no intrinsic difference between a representation and an

optical device or an eye, a taste bud or an antenna. One of the problems that arise whenever we

make a special case for representations is that we tend to get hung up on the question of truth. Is

our representation of the world an accurate one? How do we know? Can we know the world at

all? Such questions are of a purely epistemological value; they help us evaluate the status of

knowledge with respect to the unstable boundaries of what is. But they have limited value for

understanding the role of perception in constituting the real, that is, in explaining how vision

(and other senses) actively construct, dissolve and articulate patterns in a continuous coming to

be. If we put ourselves in the position of a self-organizing neural net as it engages patterns in its

environment, we come to the realization that there is no basis, much less practical reason, to

search elsewhere for “a world.” This activity is itself a world. Yet it comprises only one of many

worlds that periodically intersect, disperse, emerge and disappear again. We do not even have an

environment—an “outside”—only a milieu, assemblage or block of becoming to which our own

becoming is inextricably connected. The matrix of social codes identified by Burgin (and so

often vilified in culture studies) is just one more articulation in a network made up of many

worlds—not possible worlds or “monads,” but real intersections of abstract and material patterns.

As I have suggested elsewhere13, it is on these grounds that Deleuze interprets the predictive

success of scientific observation and the models that derive from it. From Deleuze’s point of

view, there is no need to question the truth or validity of such models, especially as this would

lead us once more to a merely epistemological evaluation of the problem. What Deleuze wants to

13 “Superposing Images: Deleuze and the Virtual after Bergson’s Critique of Science,” in Peter Gaffney ed. The Force of the Virtual (Minneapolis: University of Minnesota Press, 2010).


establish, starting with his monograph on Hume (Empiricism and Subjectivity) and borrowing

heavily from Bergson (Bergsonism), is an ontology of becoming that finds its root in all forms of

perception, conceived broadly as the creative force of the virtual in the process of actualizing

itself. A neural net does not see things in the outside world; rather, it introduces a very special

kind of eye, one that has the power to generate a set of representations that will articulate one

system with another—to generate a world made out of patterns that are themselves the

articulating feature of some other connection. Timothy Murphy has proposed that quantum

physical models provide a “realist ontological” framework for treating probably phenomena as

real events (1998: 213). In this paper, I have tried to show how connectionism provides the same

kind of framework for an ontological realist account of perception.

Conclusion

If a self-organizing neural net goes through the same stages as human intelligence when

confronted with the task of pattern-recognition and completion, then we must treat symbolic

representations as a strict redundancy. Surely representations exist, and there is no doubt that

they play a central role in the way we shape our world. But they are strange representations with

no beginning and no end, objective but not whole, real but not corporeal. They do not refer to

things; they are things. We will not be prevented, for that matter, from using representations in

the conventional manner (in this paper, for example). But it will be difficult to proceed on the

understanding that they approximate a world that precedes them, a world whose presence and

meaning confirms their validity, yet which remains just out of reach. This kind of scientific or

Cartesian realism14 no longer applies when the act of seeking is itself the constitutive feature of

14 With the term “Cartesian realism,” I am following the definition proposed by Joseph Margolis: “any realism, no matter how defended or qualified, that holds that the world has a determinate structure apart from all constraints of


whatever might be found. We cannot therefore speak of what a representation means, any more

than we can ask what a neural network learns. In each case, representation functions only as a

connection between two or more systems, one system organizing the other: a sunny sky system

organizing a neural net system, an incorporeal system organizing a material system, or vice

versa. A vast system of connections whose boundaries are continuously redrawn by the forces of

interest, desire and creativity: this is what we learn from the example of the tank-detection

device. There is nobody to blame for its failure; there is no failure. The human trainers could not

limit the task any more than the device could limit the means for accomplishing it. In any case,

the story is apocryphal.15

human inquiry and that our cognizing faculties are nevertheless able to discern those independent structures reliably. ‘Cartesianism’ serves as a term of art here, not confined to Descartes’ doctrine. It ranges over pre-Kantian philosophy, Kant’s own philosophy (quixotically), and over the views of such contemporary theorists as Putnam and Davidson” (2006: 194). 15 Though it is cited by several other authors (see note 1), I have found no published source which identifies the source of the story. Notably, it is not mentioned in William Bechtel and Adele Abrahamsen’s Connectionism and the Mind. Nick Bostrom and Milan M. Ćirković (Global Catastrophic Risk) write, “This story, although famous and oft-cited as fact, may be apocryphal…however, failures of the type described are a major real-world consideration when building and testing neural networks” (321).


WORKS CITED _______________________

Alliez, Eric. The Signature of the World: Or, What is Deleuze and Guattari's Philosophy? (New York: Continuum, 2004), 36.

Baudrillard, Jean. “The Evil Demon of Images,” in Clive Cazeaux ed. The Continental Aesthetics Reader (London: Taylor and Francis, 2000). Reprinted from The Evil Demon of Images. Trans. Paul Patton and Paul Foss (Sydney: Power Institute of Fine Arts, 1987), 13-34.

Bechtel, William and Adele Abrahamsen. Connectionism and the Mind, Second Edition (Malden, Massachusetts: Blackwell Publishers, Inc., 2002).

Bergson, Henri. L’Évolution créatrice (Paris: Les Presses universitaires de France, 1959), 132.

Bostrom, Nick and Milan M. Ćirković. Global Catastrophic Risk (Oxford: Oxford University Press, 2008), 321.

Burgin, Victor. Thinking Photography (London: Macmillan Education, 1982).

DeLanda, Manuel. War in the Age of Intelligent Machines (New York: Zone Books, 1991), 1-2.

--------“The Origins of Artificial Intelligence.” 2007 Penn Humanities Forum, University of Pennsylvania, Philadelphia. 7 November 2007.

Deleuze, Gilles and Félix Guattari, A Thousand Plateaus. Trans. Brian Massumi (Minneapolis: University of Minnesota Press, 1987).

--------What Is Philosophy? Trans. Hugh Tomlinson and Graham Burchell (New York: Columbia University Press, 1994), 118.

Dreyfus, Hubert L. “Artificial Intelligence,” American Academy of Political and Social Science, Vol. 412 (March 1974), 21-33.

Fraser, Neil. “Neural Network Follies,” http://neil.fraser.name/writing/tank)

Kehonen, Teuvo. Self-Organizing Maps (Berlin: Springer-Verlag, 1995, 2000).

Kehonen, Teuvo. “Self-Organized Maps of Sensory Events,” in Philosophical Transactions: Mathematical, Physical and Engineering Sciences, Vol. 361, No. 1807 (Jun. 15,

2003), 1177-1186.

Kuhn, Thomas. The Structure of Scientific Revolutions (Chicago: University Of Chicago Press, 1996).


McClelland, J. L. and David Rumelhard. Explorations in parallel distributed processing: a handbook of models (Cambridge, Massachusetts: Harvard University Press, 1988), 117.

Murphy, Timothy. “Quantum Ontology: A Virtual Mechanics of Becoming,” in Eleanor Kaufmann and Kevin Jon Heller eds. Deleuze and Guattari: New Mappings in Politics, Philosophy and Culture (Minneapolis: University of Minnesota Press, 1998), 211-229.

Newell, Allen and Herbert A. Simon. “Computer Science as Empirical Inquiry: Symbols and Search,” Communications of the ACM. vol. 19, No. 3 (March 1976), 113-126. Cited by Nils J. Nilsson, “The Physical Symbol System Hypothesis: Status and Prospects,” in Rolf Lungarella, Rolf Pfeifer, Josh Bongard eds., 50 Years of Artificial Intelligence, LNAI (Berlin: Springer Berlin / Verlag, 2007), 10.

Newell, Allen. Unified Theories of Cognition (Cambridge, Massachusetts: Harvard University Press, 1994).

Nietzsche, Friedrich. “On Truth and Lie in an Extra-Moral Sense,” in Clive Cazeaux ed. The Continental Aesthetics Reader (London: Taylor and Francis, 2000), 57.

Norman, Donald and David Rumelhart. “The LNR approach to human information processing.” in Cognition, Volume 10, Issues 1-3 (July-August 1981), 239.

Rockmore, Tom. “Incommensurability Modestly Recovered,” in Tom Rockmore and Joseph Margolis eds., History, Historicity and Science (Hampshire: Ashgate, 2006).

Searle, John R. “Minds, Brains, and Programs,” Behavioral and Brain Sciences, 3(3: 1980), 417-457.

Singer, P. W. Wired for War: The Robotics Revolution and Conflict in the Twenty-First Century (New York: The Penguin Press HC, 2009).

van Gelder, T. “What might cognition be, if not computation?” The Journal of Philosophy, 92 (1995), 345-381. Cited by Bechtel and Abrahamsen (1991, 2000: 275).


The problemInstruction in natural languagevariables substituted

connectionism and ontological...

Documents