Disciplines in Distress: Artificial Intelligence and

Connectionism

Ath. Kehagias

School of Engineering

Aristotle Univ.

Some Remarks

• Not a philosophical talk (since I am not a philosopher but a philo-philosopher).

• Not a thesis, just an interesting (?) story and some questions.

• Twenty-five slides, roughly one quote per slide.

• Feel free to interrupt at any point.

First Definitions

Artificial Intelligence:The science of making machines do things that would require intelligence if done by people.

Also known as Symbolicism, Symbolic Artif. Int. (SAI), Good Old-Fashioned Artif. Int. (GOFAI) etc.

(Because of the extensive use of symbol-manipulating approaches by the practitioners.)

Connectionism: A computational approach to modeling the brain which relies on the interconnection of many simple units to produce complex behavior

Also known as Neural Networks

(from the Dictionary of Philosophy of Mind, www.artsci.wustl.edu/~philos/MindDict)

01

An Example of AI (GPS)

02a

(define *school-ops*

(list

;; If your son is at home and your car works, it is

;; possible to drive him to school. (Then he'll be at

;; school and will no longer be at home.)

(make-op "drive son to school"

'(son-at-home car-works)

'(son-at-school)

'(son-at-home))

;; If your car needs a new battery, and the mechanic

;; knows the problem

;; and has been paid, it is possible him to install the

;; new battery. Then the car will work.

(make-op "have the mechanic install a new battery"

'(car-needs-battery mechanic-knows-problem

mechanic-has-money)

'(car-works)

'(car-needs-battery))

• Here, then, are a couple of problems that GPS can solve, using these operations:

> (GPS '(son-at-home car-works) '(son-at-school) *school-ops*)drive son to school

> (GPS '(son-at-home car-needs-battery have-phone-book have-money) '(son-at-school) *school-ops*)look up the telephone numbertelephone the mechanictell the mechanic what the problem ispay the mechanichave the mechanic install a new batterydrive son to school

02b

An Example of AI (GPS)

An Example of NN03

The training procedure for TD-Gammon is as follows: the network observes a sequence of board positions starting at the opening position and ending in a terminal position characterized by one side having removed all its checkers. The board positions are fed as input vectors x[1], x[2], . . . , x[f] to the neural network. Each time step in the sequence corresponds to a move made by one side. For each input pattern x[t] there is a neural network output vector Y[t] indicating the neural network's estimate of expected outcome for pattern x[t]. At each time step, the TD(lambda) algorithm is applied to change the network's weights. The formula for the weight change is as follows:

Second “Definitions”

There is a community of people who do (S)AI. They attempt to create intelligent entities (usually software). To this end they use computer programs which manipulate symbolic data structures (lists, trees, graphs etc.).

There is another community of people who do NN. They attempt to create intelligent entities and/or model human intelligence. To this end they use computer programs which manipulate numbers.

(from Thanasis’ Own Dictionary)

04

TimelineYear Connectionism Events Symbolic Artificial Intelligence

Events1943 McCulloch+Pitts: “A Logical Calculus of the Ideas Immanent in

Nervous Activity”1950 Turing: “Computing Machinery

and Intelligence”1954 Hebbian Learning1956 Dartmouth conference, 1st use of

the term “Artificial Intelligence”1957 Newell, Shaw+Simon: “General

problem solver”1958 Rosenblatt: The Original

PerceptronSamuel: Chekers playingprogramThe LISP language introduced

1960 Widrow: Adaline1962 Rosenblatt: The Classic

Perceptron, Principles ofNeurodynamics and the Theoryof Brain Mechanisms

Thomas: Analogy

1963 Dept. of Defence: beginning of ARPA projectsDendral (1st expert system)

1968 Minsky+Papert: Preceptrons1971 ETAOIN SHRDLU1972 The PROLOG language

introducedDreyfus: What ComputersCannot Do

1979

Neural network researchdrastically reduced

INTERNIST, MYCIN expertsystems

1982 Hopfield networks1986 PDP book1987 1st Int. Conference on Neural

Networks1990 Dept. of Defence: major funding

of neural network projects

Lots of stuff going on

1991 TD-GAMMON (Neural networkbackgammon playing program)

1993 Dreyfus: What Computers Still Cannot Do

05

Early AI Goals

“It is not my aim to surprise or shock you … But the simplest way I can summarize is to say that there are now in the world machines that think, that learn and that create. Moreover, their ability to do these things is going to increase rapidly, until --in a visible future-- the range of problems they can handle will be coextensive with the range to which the human mind has been applied.”

H. Simon and A. Newell, “Heuristic problem solving: the next advance in operations research”, Op. Res., vol.6, p.6, 1958.

06

Early Critique of AI

What Computers Can’t Do, H.L. Dreyfus, 1972.

– Four Assumptions

• Biological Assumption:

• Psychological Assumption

• Epistemological Assumption

• Ontological Assumption

– Two Important (and Missing) Factors:

• The Body (Embodied Intelligence)

• The Situation

07

Current AI Goals

“Douglas Lenat has a ten-year program of building a huge semantic memory (CYC). Then we will see … When people start to build programs at that magnitude and they still cannot do what they are supposed to, then we will start worrying.”

.

(H.A. Simon, “Technology is not the problem” In P.Baumgartner and S.Payr, Speaking Mind, Princeton UP, 1995.)

“AI no longer does Coginitive Modeling. It is a bunch of techniques in search of practical problems.”

(J. Feldman cited in H.L. Dreyfus, Artif. Intelligence, vol.80, p.171-191, 1996)

08

Current Critique of AI

What Computers Still Can’t Do, H.L. Dreyfus, 1993.

09

Early NN Goals

"Perceptrons are not intended to serve as detailed copies of any actual nervous system. They're simplified networks, designed to permit the study of lawful relationships between the organization of a nerve net, the organization of its environment, and the 'psychological' performances of which it is capable. Perceptrons might actually correspond to parts of more extended networks and biological systems; in this case, the results obtained will be directly applicable.”

(F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, Spartan Books, 1962)

10

Perceptrons (the book)

“The final episode in this era was a campaign led by Marvin Minsky and Seymour Papert to discredit neural network research and divert neural network research funding to the field of “artificial intelligence” … The campaign was waged by means of personal persuasion by Minsky and Papert and their allies, as well as by limited circulation of a technical manuscript (which was later de-venomized and, after further refinement and expansion, published in 1969 by Minsky and Papert as the book Perceptorns.”

(R. Hecht-Nielsen, Neurocomputation,1990)

11

NN Resurgence (AI Stagnation)

“PDP models...hold out the hope of offering computationally sufficient and psychologically accurate mechanistic accounts of the phenomena of human cognition which have eluded successful explication in conventional computational formalisms…”

(D.E. Rumelhart, J.L. McClelland, and the PDP Research Group,Parallel Distributed Processing: Explorations in the Microstructure of Cognition, MIT Press, 1987)

12

The BandWagon Effect

“Undoubtedly, the emergence of 'new' connectionism was accompanied by a certain amount of jumping on the proverbial connectionist bandwagon.”

Istvan S. N. Berkeley, “A Revisionist History of Connectionism”, 1997, http://www.ucs.louisiana.edu/~isb9112/dept/phil341/histconn.html

13

NN Stagnation 1

“I think and have thought for the last twenty years that the future consist of, just in a few days time, discovering efficient unsupervised learning algorithms that find a suitable representation. And I still believe that. I think this can be a huge technological payoff to making this work well. And I think the talk I gave this morning is a small amount of progress in that direction and on the technological front I think that is one of the major things that can happen in the next five years. For the last twenty years I’ve been saying "it’s gonna happen in the next five years" and I

keep believing that.”

G. Hinton, The Future and Prospects of Neural Networks: The Workshop in Edinburgh (Sep 8, 1999)

14

NN Stagnation 2

“ The maturing of neural networks presents an interesting study in hyperbole and substance. Those of us who jumped on the bandwagon early (in the heady days of "connectionism") foretold of a revolution that has not materialized as yet; looking back, I think we thought neural nets would have the same scale of impact as the World Wide Web has had.”

From the Editor, Control Systems Magazine, October 2000

15

NN Stagnation?

“Given the multiple relationships and interdependencies that now exist between financial markets, neural nets have a natural role to play. Their capacity to process and detect relationships and patterns in huge quantities of data goes far beyond that of a human trader. `Neural networks can find patterns in what would otherwise be disparate data that a human being would not visually be able to discern,’ says Mendelsohn. `From the standpoint of performing intermarket analysis, it is the right tool for the job’. ”

Vantage Point: Intermarket Analysis Software. At their WebSite (http://www.profittaker.com/futures_options_new.asp) Andrew Webb reports on the latest resurgence of interest in neural network technology (2000).

16

AI/NN: Similarities

• They both have attempted to perform some form of cognitive modeling.

• They both have claimed that they can produce intelligent behavior.

17

AI/NN: Differences

• AI: works at the symbol manipulation level.

• NN works at the parallel distributed computation (sub-symbolic) level.

18

AI/NN: Sociohistorical Comparison

• In both cases grandiose claims were made at the start.

• In both cases the claims were not realized.

• In both cases a sub-product was a toolbox of (very) useful agorithms

• In both cases we have a degenerating research program (?)

19

AI/NN: Interactions

A story from the 1988 Connectionist Models Summer School

“Hybrid” Systems (e.g. “Connectionist Symbol Processing”).

20

Beyond AI/NN: Computational Intelligence

“Flash forward to the late 1990s, and you'll find a hauntingly familiar atmosphere surrounding the evolutionary computation field (also referred to in some circles as genetic algorithms, after the technology that has been successfully embodied into software tools, or the loftier catch-all term artificial life). … Neural networks and evolutionary computing and fuzzy logic can all be lumped under the general phrase "computational intelligence," and for only the second time in this decade an entire conference was devoted to the research efforts of all three groups.”

Intelligent Systems Report, May 1998, Vol. 15, No. 5, www.lionhrtpub.com/ISR/isr-5-98/wcci98.html

21

Philosophical Issues

• Which of the two (AI vs. NN) is more succesful? Why?

22

Sociohistorical Issues

• The ebb and flow of each field’s popularity?

• Is there some kind of vacuum which must be filled by one theory of intelligence?

23

“Religious” Issues

• Why did the debate between Symbolicists and Connectionists become so emotional?

• Why is there so strong resistance to the idea of Artificial Intelligence? (Dreyfus, Searle, Fodor)

• Some Possible Explanations– Scientific Conflict

– Financial Conflict

– The Frankenstein Syndrome

24

Generalizations and Extensions

• Mathematics and Computer Science

• Anthropology and Sociology

• Philology and PostModern Studies

To what extend is

Academia / University changing

and how?

25