ALANTURING.NET

What is Artificial Intelligence?

Jack Copeland

©Copyright B.J. Copeland, May 2000

Artificial Intelligence (AI) is usually defined as the science of making computers do things that require intelligence when done by humans. AI has had some success in limited, or simplified, domains. However, the five decades since the inception of AI have brought only very slow progress, and early optimism concerning the attainment of human-level intelligence has given way to an appreciation of the profound difficulty of the problem.

Table of Contents

What is Intelligence?

1. Strong AI, Applied AI, and CS 2. Alan Turing and the Origins of AI 3. Early AI Programs 4. AI Programming Languages 5. Micro-World AI 6. Expert Systems 7. The CYC Project 8. Top-Down AI vs Bottom-Up AI 9. Connectionism 10. Nouvelle AI 11. Chess 12. Is Strong AI Possible? 13. The Chinese Room Objection 14. For More Information...

What is Intelligence? Quite simple human behaviour can be intelligent yet quite complex behaviour performed by insects is unintelligent. What is the difference? Consider the behaviour of the digger wasp, Sphex ichneumoneus. When the female wasp brings food to her burrow, she deposits it on the threshold, goes inside the burrow to check for intruders, and then if the coast is clear carries in the food. The unintelligent nature of the wasp's behaviour is revealed if the watching experimenter moves the food a few inches while the wasp is inside the burrow checking. On emerging, the wasp repeats the whole procedure: she carries the food to the threshold once again, goes in to look around, and emerges. She can be made to repeat this cycle of behaviour upwards of forty times in succession. Intelligence--conspicuously absent in the case of Sphex--is the ability to adapt one's behaviour to fit new circumstances.

Mainstream thinking in psychology regards human intelligence not as a single ability or cognitive process but rather as an array of separate components. Research in AI has focussed chiefly on the following components of intelligence: learning, reasoning, problem-solving, perception, and language-understanding.

Learning

Learning is distinguished into a number of different forms. The simplest is learning by trial-and-error. For example, a simple program for solving mate-in-one chess problems might try out moves at random until one is found that achieves mate. The program remembers the successful move and next time the computer is given the same problem it is able to produce the answer immediately. The simple memorising of individual items--solutions to problems, words of vocabulary, etc.--is known as rote learning.

Rote learning is relatively easy to implement on a computer. More challenging is the problem of implementing what is called generalisation. Learning that involves generalisation leaves the learner able to perform better in situations not previously encountered. A program that learns past tenses of regular English verbs by rote will not be able to produce the past tense of e.g. "jump" until presented at least once with "jumped", whereas a program that is able to generalise from examples can learn the "add-ed" rule, and so form the past tense of "jump" in the absence of any previous encounter with this verb. Sophisticated modern techniques enable programs to generalise complex rules from data.

Reasoning

To reason is to draw inferences appropriate to the situation in hand. Inferences are classified as either deductive or inductive. An example of the former is "Fred is either in the museum or the cafŽ; he isn't in the cafŽ; so he's in the museum", and of the latter "Previous accidents just like this one have been caused by instrument failure; so probably this one was caused by instrument failure". The difference between the two is that in the deductive case, the truth of the premisses guarantees the truth of the conclusion, whereas in the inductive case, the truth of the premiss lends support to the conclusion that

the accident was caused by instrument failure, but nevertheless further investigation might reveal that, despite the truth of the premiss, the conclusion is in fact false.

There has been considerable success in programming computers to draw inferences, especially deductive inferences. However, a program cannot be said to reason simply in virtue of being able to draw inferences. Reasoning involves drawing inferences that are relevant to the task or situation in hand. One of the hardest problems confronting AI is that of giving computers the ability to distinguish the relevant from the irrelevant.

Problem-solving

Problems have the general form: given such-and-such data, find x. A huge variety of types of problem is addressed in AI. Some examples are: finding winning moves in board games; identifying people from their photographs; and planning series of movements that enable a robot to carry out a given task.

Problem-solving methods divide into special-purpose and general-purpose. A special-purpose method is tailor-made for a particular problem, and often exploits very specific features of the situation in which the problem is embedded. A general-purpose method is applicable to a wide range of different problems. One general-purpose technique used in AI is means-end analysis, which involves the step-by-step reduction of the difference between the current state and the goal state. The program selects actions from a list of means--which in the case of, say, a simple robot, might consist of pickup, putdown, moveforward, moveback, moveleft, and moveright--until the current state is transformed into the goal state.

Perception

In perception the environment is scanned by means of various sense-organs, real or artificial, and processes internal to the perceiver analyse the scene into objects and their features and relationships. Analysis is complicated by the fact that one and the same object may present many different appearances on different occasions, depending on the angle from which it is viewed, whether or not parts of it are projecting shadows, and so forth.

At present, artificial perception is sufficiently well advanced to enable a self-controlled car-like device to drive at moderate speeds on the open road, and a mobile robot to roam through a suite of busy offices searching for and clearing away empty soda cans. One of the earliest systems to integrate perception and action was FREDDY, a stationary robot with a moving TV 'eye' and a pincer 'hand' (constructed at Edinburgh University during the period 1966-1973 under the direction of Donald Michie). FREDDY was able to recognise a variety of objects and could be instructed to assemble simple artefacts, such as a toy car, from a random heap of components.

Language-understanding

A language is a system of signs having meaning by convention. Traffic signs, for example, form a mini-language, it being a matter of convention that, for example, the hazard-ahead sign means hazard ahead. This meaning-by-convention that is distinctive of language is very different from what is called natural meaning, exemplified in statements like 'Those clouds mean rain' and 'The fall in pressure means the valve is malfunctioning'.

An important characteristic of full-fledged human languages, such as English, which distinguishes them from, e.g. bird calls and systems of traffic signs, is their productivity. A productive language is one that is rich enough to enable an unlimited number of different sentences to be formulated within it.

It is relatively easy to write computer programs that are able, in severely restricted contexts, to respond in English, seemingly fluently, to questions and statements, for example the Parry and Shrdlu programs described in the section Early AI Programs. However, neither Parry nor Shrdlu actually understands language. An appropriately programmed computer can use language without understanding it, in principle even to the point where the computer's linguistic behaviour is indistinguishable from that of a native human speaker of the language (see the section Is Strong AI Possible?). What, then, is involved in genuine understanding, if a computer that uses language indistinguishably from a native human speaker does not necessarily understand? There is no universally agreed answer to this difficult question. According to one theory, whether or not one understands depends not only upon one's behaviour but also upon one's history: in order to be said to understand one must have learned the language and have been trained to take one's place in the linguistic community by means of interaction with other language-users.

Strong AI, Applied AI, and CS Research in AI divides into three categories: "strong" AI, applied AI, and cognitive simulation or CS.

Strong AI aims to build machines whose overall intellectual ability is indistinguishable from that of a human being. Joseph Weizenbaum, of the MIT AI Laboratory, has described the ultimate goal of strong AI as being "nothing less than to build a machine on the model of man, a robot that is to have its childhood, to learn language as a child does, to gain its knowledge of the world by sensing the world through its own organs, and ultimately to contemplate the whole domain of human thought". The term "strong AI", now in wide use, was introduced for this category of AI research in 1980 by the philosopher John Searle, of the University of California at Berkeley. Some believe that work in strong AI will eventually lead to computers whose intelligence greatly exceeds that of human beings. Edward Fredkin, also of MIT AI Lab, has suggested that such machines "might keep us as pets". Strong AI has caught the attention of the media, but by no means all AI researchers view strong AI as worth pursuing. Excessive optimism in the 1950s and 1960s concerning strong AI has given way to an appreciation of the extreme difficulty of the problem, which is possibly the hardest that science has ever undertaken. To date, progress has been meagre. Some critics doubt whether research in the next few decades will produce even a system with the overall intellectual ability of an ant.

Applied AI, also known as advanced information-processing, aims to produce commercially viable "smart" systems--such as, for example, a security system that is able to recognise the faces of people who are permitted to enter a particular building. Applied AI has already enjoyed considerable success. Various applied systems are described in this article.

In cognitive simulation, computers are used to test theories about how the human mind works--for example, theories about how we recognise faces and other objects, or about how we solve abstract problems (such as the "missionaries and cannibals" problem described later). The theory that is to be tested is expressed in the form of a computer program and the program's performance at the task--e.g. face recognition--is compared to that of a human being. Computer simulations of networks of neurons have contributed both to psychology and to neurophysiology (some of this work is described in the section Connectionism). The program Parry, described below, was written in order to test a particular theory concerning the nature of paranoia. Researchers in cognitive psychology typically view CS as a powerful tool.

Alan Turing and the Origins of AI The earliest substantial work in the field was done by the British logician and computer pioneer Alan Mathison Turing.

In 1935, at Cambridge University, Turing conceived the modern computer. He described an abstract computing machine consisting of a limitless memory and a scanner that moves back and forth through the memory, symbol by symbol, reading what it finds and writing further symbols. The actions of the scanner are dictated by a program of instructions that is also stored in the memory in the form of symbols. This is Turing's "stored-program concept", and implicit in it is the possibility of the machine operating on, and so modifying or improving, its own program. Turing's computing machine of 1935 is now known simply as the universal Turing machine. All modern computers are in essence universal Turing machines.

During the Second World War Turing was a leading cryptanalyst at the Government Code and Cypher School, Bletchley Park (where the Allies were able to decode a large proportion of the Wehrmacht's radio communications). Turing could not turn to the project of building a stored-program electronic computing machine until the cessation of hostilities in Europe in 1945. Nevertheless, during the wartime years he gave considerable thought to the issue of machine intelligence. Colleagues at Bletchley Park recall numerous off-duty discussions with him on the topic, and at one point Turing circulated a typewritten report (now lost) setting out some of his ideas. One of these colleagues, Donald Michie (who later founded the Department of Machine Intelligence and Perception at the University of Edinburgh), remembers Turing talking often about the possibility of computing machines (1) learning from experience and (2) solving problems by means of searching through the space of possible solutions, guided by rule-of-thumb principles. The modern term for the latter idea is "heuristic search", a heuristic being any rule-of-thumb principle that cuts down the amount of searching required in order to find the solution to a problem. Programming using heuristics is a major part of modern AI, as is the area now known as machine learning.

At Bletchley Park Turing illustrated his ideas on machine intelligence by reference to chess. (Ever since, chess and other board games have been regarded as an important test-bed for ideas in AI, since these are a useful source of challenging and clearly defined problems against which proposed methods for problem-solving can be tested.) In principle, a chess-playing computer could play by searching exhaustively through all the available moves, but in practice this is impossible, since it would involve examining an astronomically large number of moves. Heuristics are necessary to guide and to narrow the search. Michie recalls Turing experimenting with two heuristics that later became common in AI, minimax and best-first. The minimax heuristic (described by the mathematician John von Neumann in 1928) involves assuming that one's opponent will move in such a way as to maximise their gains; one then makes one's own move in such a way as to minimise the losses caused by the opponent's expected move. The best-first heuristic involves ranking the moves available to one by means of a rule-of-thumb scoring system and examining the consequences of the highest-scoring move first.

In London in 1947 Turing gave what was, so far as is known, the earliest public lecture to mention computer intelligence, saying "What we want is a machine that can learn from experience", adding that the "possibility of letting the machine alter its own instructions provides the mechanism for this". In 1948 he wrote (but did not publish) a report entitled "Intelligent Machinery". This was the first manifesto of AI and in it Turing brilliantly introduced many of the concepts that were later to become central, in some cases after reinvention by others. One of these was the concept of "training" a network of artificial neurons to perform specific tasks.

In 1950 Turing introduced the test for computer intelligence that is now known simply as the Turing test. This involves three participants, the computer, a human interrogator, and a human "foil". The interrogator attempts to determine, by asking questions of the other two participants, which is the computer. All communication is via keyboard and screen. The interrogator may ask questions as penetrating and wide-ranging as he or she likes, and the computer is permitted to do everything possible to force a wrong identification. (So the computer might answer "No" in response to "Are you a computer?" and might follow a request to multiply one large number by another with a long pause and an incorrect answer.) The foil must help the interrogator to make a correct identification. A number of different people play the roles of interrogator and foil, and if sufficiently many interrogators are unable to distinguish the computer from the human being then (according to proponents of the test) it is to be concluded that the computer is an intelligent, thinking entity. In 1991, the New York businessman Hugh Loebner started the annual Loebner Prize competition, offering a $100,000 prize for the first computer program to pass the Turing test (with $2,000 awarded each year for the best effort). However, no AI program has so far come close to passing an undiluted Turing test.

In 1951 Turing gave a lecture on machine intelligence on British radio and in 1953 he published a classic early article on chess programming. Both during and after the war Turing experimented with machine routines for playing chess. (One was called the Turochamp.) In the absence of a computer to run his heuristic chess program, Turing simulated the operation of the program by hand, using paper and pencil. Play was poor! The first true AI programs had to await the arrival of stored-program electronic digital computers.

Early AI Programs The first working AI programs were written in the UK by Christopher Strachey, Dietrich Prinz, and Anthony Oettinger. Strachey was at the time a teacher at Harrow School and an amateur programmer; he later became Director of the Programming Research Group at Oxford University. Prinz worked for the engineering firm of Ferranti Ltd, which built the Ferranti Mark I computer in collaboration with Manchester University. Oettinger worked at the Mathematical Laboratory at Cambridge University, home of the EDSAC computer.

Strachey chose the board game of checkers (or draughts) as the domain for his experiment in machine intelligence. Strachey initially coded his checkers program in May 1951 for the pilot model of Turing's Automatic Computing Engine at the National Physical Laboratory. This version of the program did not run successfully; StracheyÕs efforts were defeated first by coding errors and subsequently by a hardware change that rendered his program obsolete. In addition, Strachey was dissatisfied with the method employed in the program for evaluating board positions. He wrote an improved version for the Ferranti Mark I at Manchester (with Turing's encouragement and utilising the latter's recently completed Programmers' Handbook for the Ferranti computer). By the summer of 1952 this program could, Strachey reported, "play a complete game of Draughts at a reasonable speed".

Prinz's chess program, also written for the Ferranti Mark I, first ran in November 1951. It was for solving simple problems of the mate-in-two variety. The program would examine every possible move until a solution was found. On average several thousand moves had to be examined in the course of solving a problem, and the program was considerably slower than a human player.

Turing started to program his Turochamp chess-player on the Ferranti Mark I but never completed the task. Unlike Prinz's program, the Turochamp could play a complete game and operated not by exhaustive search but under the guidance of rule-of-thumb principles devised by Turing.

Machine learning

Oettinger was considerably influenced by Turing's views on machine learning. His "Shopper" was the earliest program to incorporate learning (details of the program were published in 1952). The program ran on the EDSAC. Shopper's simulated world was a mall of eight shops. When sent out to purchase an item Shopper would if necessary search for it, visiting shops at random until the item was found. While searching, Shopper would memorise a few of the items stocked in each shop visited (just as a human shopper would). Next time Shopper was sent out for the same item, or for some other item that it had already located, it would go to the right shop straight away. As previously mentioned, this simple form of learning is called "rote learning" and is to be contrasted with learning involving "generalisation", which is exhibited by the program described next. Learning involving generalisation leaves the learner able to perform better in situations not previously encountered. (Strachey also investigated aspects of machine learning, taking the game of NIM as his focus, and in 1951 he reported a simple rote-learning scheme in a letter to Turing.)

The first AI program to run in the U.S. was also a checkers program, written in 1952 by Arthur Samuel of IBM for the IBM 701. Samuel took over the essentials of Strachey's program (which Strachey had publicised at a computing conference in Canada in 1952) and over a period of years considerably extended it. In 1955 he added features that enabled the program to learn from experience, and therefore improve its play. Samuel included mechanisms for both rote learning and generalisation. The program soon learned enough to outplay its creator. Successive enhancements that Samuel made to the learning apparatus eventually led to the program winning a game against a former Connecticut checkers champion in 1962 (who immediately turned the tables and beat the program in six games straight).

To speed up learning, Samuel would set up two copies of the program, Alpha and Beta, on the same computer, and leave them to play game after game with each other. The program used heuristics to rank moves and board positions ("looking ahead" as many as ten turns of play). The learning procedure consisted in the computer making small numerical changes to Alpha's ranking procedure, leaving Beta's unchanged, and then comparing Alpha's and Beta's performance over a few games. If Alpha played worse than Beta, these changes to the ranking procedure were discarded, but if Alpha played better than Beta then Beta's ranking procedure was replaced with Alpha's. As in biological evolution, the fitter survived, and over many such cycles of mutation and selection the program's skill would increase. (However, the quality of learning displayed by even a simple living being far surpasses that of Samuel's and Oettinger's programs.)

Evolutionary computing

The work by Samuel just described was among the earliest in a field now called evolutionary computing and is an example of the use of a genetic algorithm or GA. The term "genetic algorithm" was introduced in about 1975 by John Holland and his research group at the University of Michigan, Ann Arbor. Holland's work is principally responsible for the current intense interest in GAs. GAs employ methods analogous to the processes of natural evolution in order to produce successive generations of software entities that are increasingly fit for their intended purpose. The concept in fact goes back to Turing's manifesto of 1948, where he employed the term "genetical search". The use of GAs is burgeoning in AI and elsewhere. In one recent application, a GA-based system and a witness to a crime cooperate to generate on-screen faces that become closer and closer to the recollected face of the criminal.

Reasoning and problem-solving

The ability to reason logically is an important aspect of intelligence and has always been a major focus of AI research. In his 1948 manifesto, Turing emphasised that once a computer can prove logical theorems it will be able to search intelligently for solutions to problems. (An example of a simple logical theorem is "given that either X is true or Y is true, and given that X is in fact false, it follows that Y is true".) Prinz used the Ferranti Mark I, the first commercially available computer, to solve logical problems, and in 1949 and 1951 Ferranti built two small experimental special-purpose computers for theorem-proving and other logical work.

An important landmark in this area was a theorem-proving program written in 1955-1956 by Allen Newell and J. Clifford Shaw of the RAND Corporation at Santa Monica and Herbert Simon of the Carnegie Institute of Technology (now Carnegie-Mellon University). The program was designed to prove theorems from the famous logical work Principia Mathematica by Alfred North Whitehead and Bertrand Russell. In the case of one theorem, the proof devised by the program was more elegant than the proof given by Whitehead and Russell.

The Logic Theorist, as the program became known, was the major exhibit at a conference organised in 1956 at Dartmouth College, New Hampshire, by John McCarthy, who subsequently became one of the most influential figures in AI. The title of the conference was "The Dartmouth Summer Research Project on Artificial Intelligence". This was the first use of the term "Artificial Intelligence". Turing's original term "machine intelligence" has also persisted, especially in Britain.

Newell, Simon and Shaw went on to construct the General Problem Solver, or GPS. The first version of GPS ran in 1957 and work continued on the project for about a decade. GPS could solve an impressive variety of puzzles, for example the "missionaries and cannibals" problem: How are a party of three missionaries and three cannibals to cross a river in a small boat that will take no more than two at a time, without the missionaries on either bank becoming outnumbered by cannibals? GPS would search for a solution in a trial-and-error fashion, under the guidance of heuristics supplied by the programmers. One criticism of GPS, and other programs that lack learning, is that the program's "intelligence" is entirely second-hand, coming from the programmer (mainly via the heuristics, in the case of GPS).

Natural language communication

Two of the best-known early programs are Eliza and Parry. Details of both were first published in 1966. These programs gave an eerie semblance of conversing intelligently. Parry, written by Stanford University psychiatrist Kenneth Colby, simulated a human paranoiac. Parry's responses are capitalised in the following extract from a "conversation" between Parry and a psychiatric interviewer.

Why are you in the hospital? I SHOULDN'T BE HERE. Who brought you here? THE POLICE. What trouble did you have with the police? COPS DON'T DO THEIR JOB. What do they do instead? COPS ARREST THE WRONG PEOPLE. They arrest the right ones sometimes. THE LAW SHOULD GET THOSE ITALIAN CROOKS. Are you Italian? MY BACKGROUND IS BRITISH BUT I WAS BORN IN THIS COUNTRY. Do you know any Italian crooks? I TRY TO AVOID THE UNDERWORLD.

Psychiatrists who were asked to decide whether they were communicating with Parry or a human paranoiac were often unable to tell.

Eliza, written by Joseph Weizenbaum at MIT, simulated a human therapist. In the following extract, Eliza "speaks" second.

Men are all alike. IN WHAT WAY They're always bugging us about something or other. CAN YOU THINK OF A SPECIFIC EXAMPLE Well, my boyfriend made me come here. YOUR BOYFRIEND MADE YOU COME HERE He says I'm depressed much of the time. I AM SORRY TO HEAR THAT YOU ARE DEPRESSED It's true. I am unhappy. DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPY I need some help, that much seems certain. WHAT WOULD IT MEAN TO YOU IF YOU GOT SOME HELP Neither Parry nor Eliza can reasonably be described as intelligent. Parry's contributions to the conversation are "canned"--constructed in advance by the programmer and stored away in the computer's memory. As the philosopher Ned Block says, systems like Parry are no more intelligent than is a juke box. Eliza, too, relies on canned sentences and simple programming tricks (such as editing and returning the remark that the human participant has just made).

AI Programming Languages In the course of their work on the Logic Theorist and GPS, Newell, Simon and Shaw developed their Information Processing Language, or IPL, a computer language tailored for AI programming. At the heart of IPL was a highly flexible data-structure they called a "list". A list is simply an ordered sequence of items of data. Some or all of the items in a list may themselves be lists. This leads to richly branching structures.

In 1960 John McCarthy combined elements of IPL with elements of the lambda calculus--a powerful logical apparatus dating from 1936--to produce the language that he called LISP (from LISt Processor). In the U.S., LISP remains the principal language for AI work. (The lambda calculus itself was invented by Princeton logician Alonzo Church, while investigating the abstract Entscheidungsproblem, or decision problem, for predicate logic--the same problem that Turing was attacking when he invented the universal Turing machine.)

The logic programming language PROLOG (from PROgrammation en LOGique) was conceived by Alain Colmerauer at the University of Marseilles, where the language was first implemented in 1973. PROLOG was further developed by logician Robert Kowalski, a member of the AI group at Edinburgh University. This language makes use of a powerful theorem-proving technique known as "resolution", invented in 1963 at the Atomic Energy Commission's Argonne National Laboratory in Illinois by the British logician Alan Robinson. PROLOG can determine whether or not a given statement follows logically from other given statements. For example, given the statements "All logicians are rational" and "Robinson is a logician", a PROLOG program responds in the affirmative to the query "Robinson is rational?". PROLOG is widely used for AI work, especially in Europe and Japan.

Researchers at the Institute for New Generation Computer Technology in Tokyo have used PROLOG as the basis for sophisticated logic programming languages. These languages are in use on non-numerical parallel computers developed at the Institute. (The languages and the computers are known as "Fifth Generation" software and hardware.)

Other recent work includes the development of languages for reasoning about time-dependent data such as "the account was paid yesterday". These languages are based on tense logic, a type of logic that permits statements to be located in the flow of time. (Tense logic was invented in 1953 by the philosopher Arthur Prior at the University of Canterbury, New Zealand.)

Micro-World AI The real world is full of distracting and obscuring detail: generally science progresses by focussing on artificially simple models of reality (in physics, frictionless planes and perfectly rigid bodies, for example). In 1970 Marvin Minsky and Seymour Papert, of the MIT AI Laboratory, proposed that AI research should likewise focus on developing programs capable of intelligent behaviour in artificially simple situations known as micro-worlds. Much research has focussed on the so-called blocks world, which consists of coloured blocks of various shapes and sizes arrayed on a flat surface.

An early success of the micro-world approach was SHRDLU, written by Terry Winograd of MIT (details of the program were published in 1972). SHRDLU controlled a robot arm that operated above a flat surface strewn with play blocks (both the arm and the blocks were virtual). SHRDLU would respond to commands typed in natural English, such as "Will you please stack up both of the red blocks and either a green cube or a pyramid". The program would plan out a sequence of actions and in the virtual world the robot arm would arrange the blocks appropriately. SHRDLU could correctly answer questions about its world of blocks, for example "Can a pyramid be supported by a pyramid?" (SHRDLU attempts to stack up two pyramids and fails) and "Is there anything which is bigger than every pyramid but is not as wide as the thing that supports it?" (to which SHRDLU answered "Yes, the blue block"). SHRDLU could also answer questions about its own actions:

Had you touched any pyramid before you put one on the green block? YES, THE GREEN ONE. When did you pick it up? WHILE I WAS STACKING UP A RED CUBE, A LARGE GREEN CUBE, AND A LARGE RED BLOCK. Why? TO GET RID OF IT. Why did you do that? TO CLEAN OFF THE RED CUBE. Why did you clear off that cube? TO PUT IT ON A LARGE GREEN CUBE. Why did you do that? TO STACK UP A RED CUBE, A LARGE GREEN CUBE, AND A LARGE RED BLOCK. Why did you do that? BECAUSE YOU ASKED ME TO. Although SHRDLU was initially hailed as a major breakthrough, Winograd soon announced that the program was in fact a dead end. The techniques pioneered in the program proved unsuitable for application in wider, more interesting worlds. Moreover, the appearance that Shrdlu gives of understanding the blocks micro-world, and English statements concerning it, is in fact an illusion. Shrdlu has no idea what a red block is.

Another product of the micro-world approach was Shakey, a mobile robot developed at the Stanford Research Institute by Bertram Raphael, Nils Nilsson and their group, during the period 1968-1972. (Shakey can now be viewed at the Boston Computing Museum.) The robot occupied a specially built

micro-world consisting of walls, doorways, and a few simply-shaped wooden blocks. Each wall had a carefully painted baseboard to enable the robot to "see" where the wall met the floor (a simplification of reality that is typical of the micro-world approach). Shakey had about a dozen basic abilities, such as TURN, PUSH and CLIMB-RAMP. These could be combined in various ways by the robot's planning programs. Shakey's primary sensor was a black-and-white television camera. Other sensors included a "bump bar", and odometry that enabled the robot to calculate its position by "dead reckoning". A demonstration video showed Shakey obeying an instruction to move a certain block from one room to another by locating a ramp, pushing the ramp to the platform on which the block happened to be located, trundling up the ramp, toppling the block onto the floor, descending the ramp, and manoeuvring the block to the required room, this sequence of actions having been devised entirely by the robot's planning program without human intervention. Critics emphasise the highly simplified nature of Shakey's environment and point out that, despite these simplifications, Shakey operated excruciatingly slowly--the sequence of actions in the demonstration video in fact took days to complete. The reasons for Shakey's inability to operate on the same time-scale as a human being are examined later in this article.

FREDDY, a stationary robot with a TV "eye" mounted on a steerable platform, and a pincer "hand", was constructed at Edinburgh University under the direction of Donald Michie. FREDDY was able to recognise a small repertoire of objects, including a hammer, a cup and a ball, with about 95% accuracy; recognising a single object would take several minutes of computing time. The robot could be "taught" to assemble simple objects, such as a toy car, from a kit of parts. Envisaged applications included production-line assembly work and automatic parcel handling. FREDDY was conceived in 1966 but work was interrupted in 1973, owing to a change in the British Government's funding policy in the wake of a disparaging report on AI (and especially robotics) by the Cambridge mathematician Sir James Lighthill. Work on FREDDY resumed in 1982 with U.S. funding.

Roger Schank and his group at Yale applied a form of the micro-world approach to language processing. Their program SAM (1975) could answer questions about simple stories concerning stereotypical situations, such as dining in a restaurant and travelling on the subway. The program could infer information that was implicit in the story. For example, when asked "What did John order?", SAM replies "John ordered lasagne", even though the story states only that John went to a restaurant and ate lasagne. FRUMP, another program by Schank's group (1977), produced summaries in three languages of wire-service news reports. Impressive though SAM and FRUMP are, it is important to bear in mind that these programs are disembodied and have no real idea what lasagne and eating are. As critics point out, understanding a story requires more than an ability to produce strings of symbols in response to other strings of symbols.

The greatest success of the micro-world approach is a type of programs known as an expert system.

Expert Systems An expert system is a computer program dedicated to solving problems and giving advice within a specialised area of knowledge. A good system can match the performance of a human specialist. The field of expert systems is the most advanced part of AI, and expert systems are in wide commercial use. Expert systems are examples of micro-world programs: their "worlds"--for example, a model of a ship's hold and the containers that are to be stowed in it--are self-contained and relatively uncomplicated. Uses of expert systems include medical diagnosis, chemical analysis, credit authorisation, financial management, corporate planning, document routing in financial institutions, oil and mineral prospecting, genetic engineering, automobile design and manufacture, camera lens design, computer installation design, airline scheduling, cargo placement, and the provision of an automatic customer help service for home computer owners.

The basic components of an expert system are a "knowledge base" or KB and an "inference engine". The information in the KB is obtained by interviewing people who are expert in the area in question. The interviewer, or "knowledge engineer", organises the information elicited from the experts into a collection of rules, typically of "if-then" structure. Rules of this type are called "production rules". The inference engine enables the expert system to draw deductions from the rules in the KB. For example, if the KB contains production rules "if x then y" and "if y then z", the inference engine is able to deduce "if x then z". The expert system might then query its user "is x true in the situation that we are considering?" (e.g. "does the patient have a rash?") and if the answer is affirmative, the system will proceed to infer z.

In 1965 the AI researcher Edward Feigenbaum and the geneticist Joshua Lederberg, both of Stanford University, began work on Heuristic Dendral, the high-performance program that was the model for much of the ensuing work in the area of expert systems (the name subsequently became DENDRAL). The program's task was chemical analysis. The substance to be analysed might, for example, be a complicated compound of carbon, hydrogen and nitrogen. Starting from spectrographic data obtained from the substance, DENDRAL would hypothesise the substance's molecular structure. DENDRAL's performance rivalled that of human chemists expert at this task, and the program was used in industry and in universities.

Work on MYCIN, an expert system for treating blood infections, began at Stanford in 1972. MYCIN would attempt to identify the organism responsible for an infection from information concerning the patient's symptoms and test results. The program would request further information if necessary, asking questions such as "has the patient recently suffered burns?". Sometimes MYCIN would suggest additional laboratory tests. When the program had arrived at a diagnosis it would recommend a course of medication. If requested, MYCIN would explain the reasoning leading to the diagnosis and recommendation.

Examples of production rules from MYCIN's knowledge base are (1) If the site of the culture is blood, and the stain of the organism is gramneg, and the morphology of the organism is rod, and the patient

has been seriously burned, then there is evidence (.4) that the identity of the organism is pseudomonas. (The decimal number is a certainty factor, indicating the extent to which the evidence supports the conclusion.) (2) If the identity of the organism is pseudomonas then therapy should be selected from among the following drugs: Colistin (.98) Polymyxin (.96) Gentamicin (.96) Carbenicillin (.65) Sulfisoxazole (.64). (The decimal numbers represent the statistical probability of the drug arresting the growth of pseudomonas.) The program would make a final choice of drug from this list after quizzing the user concerning contra-indications such as allergies. Using around 500 such rules MYCIN achieved a high level of performance. The program operated at the same level of competence as human specialists in blood infections, and rather better than general practitioners.

Janice Aikins' medical expert system Centaur (1983) was designed to determine the presence and severity of lung disease in a patient by interpreting measurements from pulmonary function tests. The following is actual output from the expert system concerning a patient at Pacific Medical Center in San Francisco.

The findings about the diagnosis of obstructive airways disease are as follows: Elevated lung volumes indicate overinflation. The RV/TLC ratio is increased, suggesting a severe degree of air trapping. Low mid-expiratory flow is consistent with severe airway obstruction. Obstruction is indicated by curvature of the flow-volume loop which is of a severe degree. Conclusions: Smoking probably exacerbates the severity of the patient's airway obstruction. Discontinuation of smoking should help relieve the symptoms. Good response to bronchodilators is consistent with an asthmatic condition, and their continued use is indicated. Pulmonary function diagnosis: Severe obstructive airways disease, asthmatic type. Consultation finished.

An important feature of expert systems is that they are able to work cooperatively with their human users, enabling a degree of human-computer symbiosis. AI researcher Douglas Lenat says of his expert system Eurisko, which became a champion player in the star-wars game Traveller, that the "final crediting of the win should be about 60/40% Lenat/Eurisko, though the significant point here is that neither Lenat nor Eurisko could have won alone". Eurisko and Lenat cooperatively designed a fleet of warships which exploited the rules of the Traveller game in unconventional ways, and which was markedly superior to the fleets designed by human participants in the game.

Fuzzy logic

Some expert systems use fuzzy logic. In standard, non-fuzzy, logic there are only two "truth values", true and false. This is a somewhat unnatural restriction, since we normally think of statements as being nearly true, partly false, truer than certain other statements, and so on. According to standard logic, however, there are no such in-between values--no "degrees of truth"--and any statement is either completely true or completely false. In 1920 and 1930 the Polish philosopher Jan Lukasiewicz introduced a form of logic that employs not just two values but many. Lotfi Zadeh, of the University of California at Berkeley, subsequently proposed that the many values of Lukasiewicz's logic be regarded as degrees of truth, and he coined the expression "fuzzy logic" for the result. (Zadeh published the first of many papers on the subject in 1965.) Fuzzy logic is particularly useful when it is necessary to deal with vague

expressions, such as "bald", "heavy", "high", "low", "hot", "cold" and so on. Vague expressions are difficult to deal with in standard logic because statements involving them--"Fred is bald", say--may be neither completely true nor completely false. Non-baldness shades gradually into baldness, with no sharp dividing line at which the statement "Fred is bald" could change from being completely false to completely true. Often the rules that knowledge engineers elicit from human experts contain vague expressions, so it is useful if an expert system's inference engine employs fuzzy logic. An example of such a rule is: "If the pressure is high but not too high, then reduce the fuel flow a little". (Fuzzy logic is used elsewhere in AI, for example in robotics and in neuron-like computing. There are literally thousands of commercial applications of fuzzy logic, many developed in Japan, ranging from an automatic subway train controller to control systems for washing machines and cameras.)

Limitations of expert systems

Expert systems have no "common sense". They have no understanding of what they are for, nor of what the limits of their applicability are, nor of how their recommendations fit into a larger context. If MYCIN were told that a patient who has received a gunshot wound is bleeding to death, the program would attempt to diagnose a bacterial cause for the patient's symptoms. Expert systems can make absurd errors, such as prescribing an obviously incorrect dosage of a drug for a patient whose weight and age are accidentally swapped by the clerk. One project aimed at improving the technology further is described in the next section.

The knowledge base of an expert system is small and therefore manageable--a few thousand rules at most. Programmers are able to employ simple methods of searching and updating the KB which would not work if the KB were large. Furthermore, micro-world programming involves extensive use of what are called "domain-specific tricks"--dodges and shortcuts that work only because of the circumscribed nature of the program's "world". More general simplifications are also possible. One example concerns the representation of time. Some expert systems get by without acknowledging time at all. In their micro-worlds everything happens in an eternal present. If reference to time is unavoidable, the micro-world programmer includes only such aspects of temporal structure as are essential to the task--for example, that if a is before b and b is before c then a is before c. This rule enables the expert system to merge suitable pairs of before-statements and so extract their implication (e.g. that the patient's rash occurred before the application of penicillin). The system may have no other information at all concerning the relationship "before"--not even that it orders events in time rather than space.

The problem of how to design a computer program that performs at human levels of competence in the full complexity of the real world remains open.

The CYC Project CYC (the name comes from "encyclopaedia") is the largest experiment yet in symbolic AI. The project began at the Microelectronics and Computer Technology Corporation in Texas in 1984 under the direction of Douglas Lenat, with an initial budget of U.S.$50 million, and is now Cycorp Inc. The goal is to build a KB containing a significant percentage of the common sense knowledge of a human being. Lenat hopes that the CYC project will culminate in a KB that can serve as the foundation for future generations of expert systems. His expectation is that when expert systems are equipped with common sense they will achieve an even higher level of performance and be less prone to errors of the sort just mentioned.

By "common sense", AI researchers mean that large corpus of worldly knowledge that human beings use to get along in daily life. A moment's reflection reveals that even the simplest activities and transactions presuppose a mass of trivial-seeming knowledge: to get to a place one should (on the whole) move in its direction; one can pass by an object by moving first towards it and then away from it; one can pull with a string, but not push; pushing something usually affects its position; an object resting on a pushed object usually but not always moves with the pushed object; water flows downhill; city dwellers do not usually go outside undressed; causes generally precede their effects; time constantly passes and future events become past events ... and so on and so on. A computer that is to get along intelligently in the real world must somehow be given access to millions of such facts. Winograd, the creator of SHRDLU, has remarked "It has long been recognised that it is much easier to write a program to carry out abstruse formal operations than to capture the common sense of a dog".

The CYC project involves "hand-coding" many millions of assertions. By the end of the first six years, over one million assertions had been entered manually into the KB. Lenat estimates that it will require some 2 person-centuries of work to increase this figure to the 100 million assertions that he believes are necessary before CYC can begin learning usefully from written material for itself. At any one time as many as 30 people may be logged into CYC, all simultaneously entering data. These knowledge-enterers (or "cyclists") go through newspaper and magazine articles, encyclopaedia entries, advertisements, and so forth, asking themselves what the writer assumed the reader would already know: living things get diseases, the products of a commercial process are more valuable than the inputs, and so on. Lenat describes CYC as "the complement of an encyclopaedia": the primary goal of the project is to encode the knowledge that any person or machine must have before they can begin to understand an encyclopaedia. He has predicted that in the early years of the new millennium, CYC will become "a system with human-level breadth and depth of knowledge".

CYC uses its common-sense knowledge to draw inferences that would defeat simpler systems. For example, CYC can infer "Garcia is wet" from the statement "Garcia is finishing a marathon run", employing its knowledge that running a marathon entails high exertion, that people sweat at high levels of exertion, and that when something sweats it is wet.

Among the outstanding fundamental problems with CYC are (1) issues in search and problem-solving, for example how to automatically search the KB for information that is relevant to a given problem

(these issues are aspects of the frame problem, described in the section Nouvelle AI) and (2) issues in knowledge representation, for example how basic concepts such as those of substance and causation are to be analyzed and represented within the KB. Lenat emphasises the importance of large-scale knowledge-entry and is devoting only some 20 percent of the project's effort to development of mechanisms for searching, updating, reasoning, learning, and analogizing. Critics argue that this strategy puts the cart before the horse.

Top-Down AI vs Bottom-Up AI Turing's manifesto of 1948 distinguished two different approaches to AI, which may be termed "top down" and "bottom up". The work described so far in this article belongs to the top-down approach. In top-down AI, cognition is treated as a high-level phenomenon that is independent of the low-level details of the implementing mechanism--a brain in the case of a human being, and one or another design of electronic digital computer in the artificial case. Researchers in bottom-up AI, or connectionism, take an opposite approach and simulate networks of artificial neurons that are similar to the neurons in the human brain. They then investigate what aspects of cognition can be recreated in these artificial networks.

The difference between the two approaches may be illustrated by considering the task of building a system to discriminate between W, say, and other letters. A bottom-up approach could involve presenting letters one by one to a neural network that is configured somewhat like a retina, and reinforcing neurons that happen to respond more vigorously to the presence of W than to the presence of aany other letter. A top-down approach could involve writing a computer program that checks inputs of letters against a description of W that is couched in terms of the angles and relative lengths of intersecting line segments. Simply put, the currency of the bottom-up approach is neural activity and of the top-down approach descriptions of relevant features of the task.

The descriptions employed in the top-down approach are stored in the computer's memory as structures of symbols (e.g. lists). In the case of a chess or checkers program, for example, the descriptions involved are of board positions, moves, and so forth. The reliance of top-down AI on symbolically encoded descriptions has earned it the name "symbolic AI". In the 1970s Newell and Simon--vigorous advocates of symbolic AI--summed up the approach in what they called the Physical Symbol System Hypothesis, which says that the processing of structures of symbols by a digital computer is sufficient to produce artificial intelligence, and that, moreover, the processing of structures of symbols by the human brain is the basis of human intelligence. While it remains an open question whether the Physical Symbol System Hypothesis is true or false, recent successes in bottom-up AI have resulted in symbolic AI being to some extent eclipsed by the neural approach, and the Physical Symbol System Hypothesis has fallen out of fashion.

Connectionism Connectionism, or neuron-like computing, developed out of attempts to understand how the brain works at the neural level, and in particular how we learn and remember.

A natural neural network. The Golgi method of staining brain tissue renders the neurons and their interconnecting fibres visible in silhouette.

In one famous connectionist experiment (conducted at the University of California at San Diego and published in 1986), David Rumelhart and James McClelland trained a network of 920 artificial neurons to form the past tenses of English verbs. The network consisted of two layers of 460 neurons:

Each of the 460 neurons in the input layer is connected to each of the 460 neurons in the output layer

Root forms of verbs--such as "come", "look", and "sleep"--were presented (in an encoded form) to one layer of neurons, the input layer. A supervisory computer program observed the difference between the

actual response at the layer of output neurons and the desired response--"came", say--and then mechanically adjusted the connections throughout the network in such a way as to give the network a slight push in the direction of the correct response (this procedure is explained in more detail in what follows).

About 400 different verbs were presented one by one to the network and the connections were adjusted after each presentation. This whole procedure was repeated about 200 times using the same verbs. By this stage the network had learned its task satisfactorily and would correctly form the past tense of unfamiliar verbs as well as of the original verbs. For example, when presented for the first time with "guard" the network responded "guarded", with "weep" "wept", with "cling" "clung", and with "drip" "dripped" (notice the double "p"). This is a striking example of learning involving generalisation. (Sometimes, though, the peculiarities of English were too much for the network and it formed "squawked" from "squat", "shipped" from "shape", and "membled" from "mail".)

The simple neural network shown below illustrates the central ideas of connectionism.

A pattern-classifier

Four of the network's five neurons are for input and the fifth--to which each of the others is connected--is for output. Each of the neurons is either firing (1) or not firing (0). This network can learn to which of two groups, A and B, various simple patterns belong. An external agent is able to "clamp" the four input neurons into a desired pattern, for example 1100 (i.e. the two neurons to the left are firing and the other two are quiescent). Each such pattern has been pre-assigned to one of two groups, A and B. When a pattern is presented as input, the trained network will correctly classify it as belonging to group A or group B, producing 1 as output if the pattern belongs to A, and 0 if it belongs to B (i.e. the output neuron fires in the former case, does not fire in the latter).

Each connection leading to N, the output neuron, has a "weight". What is called the "total weighted input" into N is calculated by adding up the weights of all the connections leading to N from neurons that are firing. For example, suppose that only two of the input neurons, X and Y, are firing. Since the weight of the connection from X to N is 1.5 and the weight of the connection from Y to N is 2, it follows that the total weighted input to N is 3.5.

N has a "firing threshold" of 4. That is to say, if N's total weighted input exceeds or equals N's threshold, then N fires; and if the total weighted input is less than the threshold, then N does not fire. So, for example, N does not fire if the only input neurons to fire are X and Y, but N does fire if X, Y and Z all fire.

Training the network involves two steps. First, the external agent inputs a pattern and observes the behaviour of N. Second, the agent adjusts the connection-weights in accordance with the rules:

(1) If the actual output is 0 and the desired output is 1, increase by a small fixed amount the weight of each connection leading to N from neurons that are firing (thus making it more likely that N will fire next time the network is given the same pattern)

(2) If the actual output is 1 and the desired output is 0, decrease by that same small amount the weight of each connection leading to the output neuron from neurons that are firing (thus making it less likely that the output neuron will fire the next time the network is given that pattern as input).

The external agent--actually a computer program--goes through this two-step procedure with each of the patterns in the sample that the network is being trained to classify. The agent then repeats the whole process a considerable number of times. During these many repetitions, a pattern of connection weights is forged that enables the network to respond correctly to each of the patterns.

The striking thing is that the learning process is entirely mechanistic and requires no human intervention or adjustment. The connection weights are increased or decreased mechanically by a constant amount and the procedure remains the same no matter what task the network is learning.

Another name for connectionism is "parallel distributed processing" or PDP. This terminology emphasises two important features of neuron-like computing. (1) A large number of relatively simple processors--the neurons--operate in parallel. (2) Neural networks store information in a distributed or holistic fashion, with each individual connection participating in the storage of many different items of information. The know-how that enables the past-tense network to form "wept" from "weep", for example, is not stored in one specific location in the network but is spread through the entire pattern of connection weights that was forged during training. The human brain also appears to store information in a distributed fashion, and connectionist research is contributing to attempts to understand how the brain does so.

Recent work with neural networks includes:

(1) The recognising of faces and other objects from visual data. A neural network designed by John Hummel and Irving Biederman at the University of Minnesota can identify about ten objects from simple line drawings. The network is able to recognise the objects--which include a mug and a frying pan--even

when they are drawn from various different angles. Networks investigated by Tomaso Poggio of MIT are able to recognise (a) bent-wire shapes drawn from different angles (b) faces photographed from different angles and showing different expressions (c) objects from cartoon drawings with grey-scale shading indicating depth and orientation. (An early commercially available neuron-like face recognition system was WISARD, designed at the beginning of the 1980s by Igor Aleksander of Imperial College London. WISARD was used for security applications.)

(2) Language processing. Neural networks are able to convert handwriting and typewritten material to standardised text. The U.S. Internal Revenue Service has commissioned a neuron-like system that will automatically read tax returns and correspondence. Neural networks also convert speech to printed text and printed text to speech.

(3) Neural networks are being used increasingly for loan risk assessment, real estate valuation, bankruptcy prediction, share price prediction, and other business applications.

(4) Medical applications include detecting lung nodules and heart arrhythmia, and predicting patients' reactions to drugs.

(5) Telecommunications applications of neural networks include control of telephone switching networks and echo cancellation in modems and on satellite links.

History of connectionism

In 1933 the psychologist Edward Thorndike suggested that human learning consists in the strengthening of some (then unknown) property of neurons, and in 1949 psychologist Donald Hebb suggested that it is specifically a strengthening of the connections between neurons in the brain that accounts for learning.

In 1943, the neurophysiologist Warren McCulloch of the University of Illinois and the mathematician Walter Pitts of the University of Chicago published an influential theory according to which each neuron in the brain is a simple digital processor and the brain as a whole is a form of computing machine. As McCulloch put it subsequently, "What we thought we were doing (and I think we succeeded fairly well) was treating the brain as a Turing machine".

McCulloch and Pitts gave little discussion of learning and apparently did not envisage fabricating networks of artificial neuron-like elements. This step was first taken, in concept, in 1947-48, when Turing theorized that a network of initially randomly connected artificial neurons--a Turing Net--could be "trained" (his word) to perform a given task by means of a process that renders certain neural pathways effective and others ineffective. Turing foresaw the procedure--now in common use by connectionists--of simulating the neurons and their interconnections within an ordinary digital computer (just as engineers create virtual models of aircraft wings and skyscrapers).

However, Turing's own research on neural networks was carried out shortly before the first stored-program electronic computers became available. It was not until 1954 (the year of Turing's death) that Belmont Farley and Wesley Clark, working at MIT, succeeded in running the first computer simulations

of small neural networks. Farley and Clark were able to train networks containing at most 128 neurons to recognise simple patterns (using essentially the training procedure described above). In addition, they discovered that the random destruction of up to 10% of the neurons in a trained network does not affect the network's performance at its task--a feature that is reminiscent of the brain's ability to tolerate limited damage inflicted by surgery, an accident, or disease.

During the 1950s neuron-like computing was studied on both sides of the Atlantic. Important work was done in England by W.K. Taylor at University College, London, J.T. Allanson at Birmingham University, R.L. Beurle and A.M. Uttley at the Radar Research Establishment, Malvern; and in the U.S. by Frank Rosenblatt, at the Cornell Aeronautical Laboratory.

In 1957 Rosenblatt began investigating artificial neural networks that he called "perceptrons". Although perceptrons differed only in matters of detail from types of neural network investigated previously by Farley and Clark in the U.S. and byTaylor, Uttley, Beurle and Allanson in Britain, Rosenblatt made major contributions to the field, through his experimental investigations of the properties of perceptrons (using computer simulations), and through his detailed mathematical analyses. Rosenblatt was a charismatic communicator and soon in the U.S. there were many research groups studying perceptrons. Rosenblatt and his followers called their approach connectionist to emphasise the importance in learning of the creation and modification of connections between neurons and modern researchers in neuron-like computing have adopted this term.

Rosenblatt distinguished between simple perceptrons with two layers of neurons--the networks described earlier for forming past tenses and classifying patterns both fall into this category--and multi-layer perceptrons with three or more layers.

A three-layer perceptron. Between the input layer (bottom) and the output layer (top) lies a so-called 'hidden layer' of neurons.

One of Rosenblatt's important contributions was to generalise the type of training procedure that Farley and Clark had used, which applied only to two-layer networks, so that the procedure can be applied to multi-layer networks. Rosenblatt used the phrase "back-propagating error correction" to describe his method. The method, and the term "back-propagation", are now in everyday use in neuron-like computing (with improvements and extensions due to Bernard Widrow and M.E. Hoff, Paul Werbos, David Rumelhart, Geoffrey Hinton, Ronald Williams, and others).

During the 1950s and 1960s, the top-down and bottom-up approaches to AI both flourished, until in 1969 Marvin Minsky and Seymour Papert of MIT, who were both committed to symbolic AI, published a critique of Rosenblatt's work. They proved mathematically that there are a variety of tasks that simple two-layer perceptrons cannot accomplish. Some examples they gave are:

(1) No two-layer perceptron can correctly indicate at its output neuron (or neurons) whether there are an even or an odd number of neurons firing in its input layer.

(2) No two-layer perceptron can produce at its output layer the exclusive disjunction of two binary inputs X and Y (the so-called "XOR problem").

The exclusive disjunction of two binary inputs X and Y is defined by this table.

It is important to realise that the mathematical results obtained by Minsky and Papert about two-layer perceptrons, while interesting and technically sophisticated, showed nothing about the abilities of perceptrons in general, since multi-layer perceptrons are able to carry out tasks that no two-layer perceptron can accomplish. Indeed, the "XOR problem" illustrates this fact: a simple three-layer perceptron can form the exclusive disjunction of X and Y (as Minsky and Papert knew). Nevertheless, Minsky and Papert conjectured--without any real evidence--that the multi-layer approach is "sterile" (their word). Somehow their analysis of the limitations of two-layer perceptrons convinced the AI community--and the bodies that fund it--of the fruitlessness of pursuing work with neural networks, and the majority of researchers turned away from the approach (although a small number remained loyal).

This hiatus in research into neuron-like computing persisted for well over a decade before a renaissance occurred. Causes of the renaissance included (1) a widespread perception that symbolic AI was stagnating (2) the possibility of simulating larger and more complex neural networks, owing to the improvements that had occurred in the speed and memory of digital computers, and (3) results published in the early and mid 1980s by McClelland, Rumelhart and their research group (for example, the past-tenses experiment) which were widely viewed as a powerful demonstration of the potential of neural networks. There followed an explosion of interest in neuron-like computing, and symbolic AI moved into the back seat.

Nouvelle AI The approach to AI now known as "nouvelle AI" was pioneered at the MIT AI Laboratory by the Australian Rodney Brooks, during the latter half of the 1980s. Nouvelle AI distances itself from traditional characterisations of AI, which emphasize human-level performance. One aim of nouvelle AI is the relatively modest one of producing systems that display approximately the same level of intelligence as insects.

Practitioners of nouvelle AI reject micro-world AI, emphasising that true intelligence involves the ability to function in a real-world environment. A central idea of nouvelle AI is that the basic building blocks of intelligence are very simple behaviours, such as avoiding an object, and moving forward. More complex behaviours "emerge" from the interaction of these simple behaviours. For example, a micro-robot whose simple behaviours are (1) collision-avoidance and (2) motion toward a moving object will appear to chase the moving object while hanging back from it a little.

Brooks focussed in his initial work on building robots that behave somewhat like simplified insects (and in doing so he deliberately turned away from traditional characterisations of AI such as the one given at the beginning of this article). Examples of his insect-like mobile robots are Allen (after Allen Newell) and Herbert (after Herbert Simon). Allen has a ring of twelve ultrasonic sonars as its primary sensors and three independent behaviour-producing modules. The lowest-level module makes the robot avoid both stationary and moving objects. With only this module activated, Allen sits in the middle of a room until approached and then scurries away, avoiding obstacles as it goes. The second module makes the robot wander about at random when not avoiding objects, and the third pushes the robot to look for distant places with its sensors and to move towards them. (The second and third modules are in tension--just as our overall behaviour may sometimes be the product of conflicting drives, such as the drive to seek safety and the drive to avoid boredom.)

Herbert has thirty infrared sensors for avoiding local obstacles, a laser system that collects three-dimensional depth data over a distance of about twelve feet in front of the robot, and a hand equipped with a number of simple sensors. Herbert's real-world environment consists of the busy offices and work-spaces of the AI lab. The robot searches on desks and tables in the lab for empty soda cans, which it picks up and carries away. Herbert's seemingly coordinated and goal-directed behaviour emerges from the interactions of about fifteen simple behaviours. Each simple behaviour is produced by a separate module, and each of these modules functions without reference to the others. (Unfortunately, Herbert's mean time from power-on to hardware failure is no more than fifteen minutes, owing principally to the effects of vibration.)

Other robots produced by Brooks and his group include Genghis, a six-legged robot that walks over rough terrain and will obediently follow a human, and Squirt, which bides in dark corners until a noise beckons it out, when it will begin to follow the source of the noise, moving with what appears to be circumspection from dark spot to dark spot. Other experiments involve tiny "gnat" robots. Speaking of

potential applications, Brooks describes possible colonies of gnat robots designed to inhabit the surface of TV and computer screens and keep them clean.

Brooks admits that even his more complicated artificial insects come nowhere near the complexity of real insects. One question that must be faced by those working in situated AI is whether insect-level behaviour is a reasonable initial goal. John von Neumann, the computer pioneer and founder, along with Turing, of the research area now known as "artificial life", thought otherwise. In a letter to the cyberneticist Norbert Wiener in 1946, von Neumann argued that automata theorists who select the human nervous system as their model are unrealistically picking "the most complicated object under the sun", and that there is little advantage in selecting instead the ant, since any nervous system at all exhibits "exceptional complexity". Von Neumann believed that "the decisive break" is "more likely to come in another theater" and recommended attention to "organisms of the virus or bacteriophage type" which, he pointed out, are "self-reproductive and ... are able to orient themselves in an unorganised milieu, to move towards food, to appropriate it and to use it". This starting point would, as he put it, provide "a degree of complexity which is not necessarily beyond human endurance".

The frame problem

The products of nouvelle AI are quite different from those of symbolic AI, for example Shakey and FREDDY. These contained an internal model (or "representation") of their micro-worlds, consisting of symbolic descriptions. This structure of symbols had to be updated continuously as the robot moved or the world changed. The robots' planning programs would juggle with this huge structure of symbols until descriptions were derived of actions that would transform the current situation into the desired situation. All this computation required a large amount of processing time. This is why Shakey performed its tasks with extreme slowness, even though careful design of the robot's environment minimised the complexity of the internal model. In contrast, Brooks' robots contain no internal model of the world. Herbert, for example, continuously discards the information that is received from its sensors, sensory information persisting in the robot's memory for no more than two seconds.

AI researchers call the problem of updating, searching, and otherwise manipulating, a large structure of symbols in realistic amounts of time the frame problem. The frame problem is endemic to symbolic AI. Some critics of symbolic AI believe that the frame problem is largely insolvable and so maintain that the symbolic approach will not "scale up" to yield genuinely intelligent systems. It is possible that CYC, for example, will succumb to the frame problem long before the system achieves human levels of knowledge.

Nouvelle AI sidesteps the frame problem. Nouvelle systems do not contain a complicated symbolic model of their environment. Information is left "out in the world" until such time as the system needs it. A nouvelle system refers continuously to its sensors rather than to an internal model of the world: it "reads off" the external world whatever information it needs, at precisely the time it needs it. As Brooks puts it, the world is its own best model--always exactly up to date and complete in every detail.

Situated AI

Traditional AI has by and large attempted to build disembodied intelligences whose only way of interacting with the world has been via keyboard and screen or printer. Nouvelle AI attempts to build embodied intelligences situated in the real world. Brooks quotes approvingly from the brief sketches that Turing gave in 1948 and 1950 of the "situated" approach. Turing wrote of equipping a machine "with the best sense organs that money can buy" and teaching it "to understand and speak English" by a process that would "follow the normal teaching of a child". Turing contrasted this with the approach to AI that focuses on abstract activities, such as the playing of chess. He advocated that both approaches be pursued, but until now relatively little attention has been paid to the situated approach.

The situated approach is anticipated in the writings of the philosopher Bert Dreyfus, of the University of California at Berkeley. Dreyfus is probably the best-known critic of symbolic AI. He has been arguing against the Physical Symbol System Hypothesis since the early 1960s, urging the inadequacy of the view that everything relevant to intelligent behaviour can be captured by means of structures (e.g. lists) of symbolic descriptions. At the same time he has advocated an alternative view of intelligence, which stresses the need for an intelligent agent to be situated in the world, and he has emphasised the role of the body in intelligent behaviour and the importance of such basic activities as moving about in the world and dealing with obstacles. Once reviled by admirers of AI, Dreyfus is now regarded as a prophet of the situated approach.

Cog

Brooks' own recent work has taken the opposite direction to that proposed by von Neumann in the quotations given earlier. Brooks is pursuing AI's traditional goal of human-level intelligence, and with Lynn Andrea Stein, he has built a humanoid robot known as Cog. Cog has four microphone-type sound sensors and is provided with saccading foveated vision by cameras mounted on its "head". Cog's (legless) torso is capable of leaning and twisting. Strain gauges on the spine give Cog information about posture. Heat and current sensors on the robot's motors provide feedback concerning exertion. The arm and manipulating hand are equipped with strain gauges and heat and current sensors. Electrically-conducting rubber membranes on the hand and arm provide tactile information.

Brooks believes that Cog will learn to correlate noises with visual events and to extract human voices from background noise; and that in the long run Cog will, through its interactions with its environment and with human beings, learn for itself some of the wealth of common sense knowledge that Lenat and his team are patiently hand-coding into CYC.

Critics of nouvelle AI emphasis that so far the approach has failed to produce a system exhibiting anything like the complexity of behaviour found in real insects. Suggestions by some advocates of nouvelle AI that it is only a short step to systems which are conscious and which possess language seem entirely premature.

Chess Some of AI's most conspicuous successes have been in chess, its oldest area of research.

In 1945 Turing predicted that computers would one day play "very good chess", an opinion echoed in 1949 by Claude Shannon of Bell Telephone Laboratories, another early theoretician of computer chess.

By 1958 Simon and Newell were predicting that within ten years the world chess champion would be a computer, unless barred by the rules. Just under 40 years later, on May 11 1997, in midtown Manhattan, IBM's Deep Blue beat the reigning world champion, Gary Kasparov, in a six-game match.

Critics question the worth of research into computer chess. MIT linguist Noam Chomsky has said that a computer program's beating a grandmaster at chess is about as interesting as a bulldozer's "winning" an Olympic weight-lifting competition. Deep Blue is indeed a bulldozer of sorts--its 256 parallel processors enable it to examine 200 million possible moves per second and to look ahead as many as fourteen turns of play.

The huge improvement in machine chess since Turing's day owes much more to advances in hardware engineering than to advances in AI. Massive increases in cpu speed and memory have meant that each generation of chess machine has been able to examine increasingly more possible moves. Turing's expectation was that chess-programming would contribute to the study of how human beings think. In fact, little or nothing about human thought processes has been learned from the series of projects that culminated in Deep Blue.

Is Strong AI Possible? The ongoing success of applied Artificial Intelligence and of cognitive simulation seems assured. However, strong AI, which aims to duplicate human intellectual abilities, remains controversial. The reputation of this area of research has been damaged over the years by exaggerated claims of success that have appeared both in the popular media and in the professional journals. At the present time, even an embodied system displaying the overall intelligence of a cockroach is proving elusive, let alone a system rivalling a human being.

The difficulty of "scaling up" AI's so far relatively modest achievements cannot be overstated. Five decades of research in symbolic AI has failed to produce any firm evidence that a symbol-system can manifest human levels of general intelligence. Critics of nouvelle AI regard as mystical the view that high-level behaviours involving language-understanding, planning, and reasoning will somehow "emerge" from the interaction of basic behaviours like obstacle avoidance, gaze control and object manipulation. Connectionists have been unable to construct working models of the nervous systems of even the simplest living things. Caenorhabditis elegans, a much-studied worm, has approximately 300 neurons, whose pattern of interconnections is perfectly known. Yet connectionist models have failed to mimic the worm's simple nervous system. The "neurons" of connectionist theory are gross oversimplifications of the real thing.

However, this lack of substantial progress may simply be testimony to the difficulty of strong AI, not to its impossibility.

Let me turn to the very idea of strong artificial intelligence. Can a computer possibly be intelligent, think and understand? Noam Chomsky suggests that debating this question is pointless, for it is a question of decision, not fact: decision as to whether to adopt a certain extension of common usage. There is, Chomsky claims, no factual question as to whether any such decision is right or wrong--just as there is no question as to whether our decision to say that aeroplanes fly is right, or our decision not to say that ships swim is wrong. However, Chomsky is oversimplifying matters. Of course we could, if we wished, simply decide to describe bulldozers, for instance, as things that fly. But obviously it would be misleading to do so, since bulldozers are not appropriately similar to the other things that we describe as flying. The important questions are: could it ever be appropriate to say that computers are intelligent, think, and understand, and if so, what conditions must a computer satisfy in order to be so described?

Some authors offer the Turing test as a definition of intelligence: a computer is intelligent if and only if the test fails to distinguish it from a human being. However, Turing himself in fact pointed out that his test cannot provide a definition of intelligence. It is possible, he said, that a computer which ought to be described as intelligent might nevertheless fail the test because it is not capable of successfully imitating a human being. For example, why should an intelligent robot designed to oversee mining on the moon necessarily be able to pass itself off in conversation as a human being? If an intelligent entity can fail the test, then the test cannot function as a definition of intelligence.

It is even questionable whether a computer's passing the test would show that the computer is intelligent. In 1956 Claude Shannon and John McCarthy raised the objection to the test that it is possible in principle to design a program containing a complete set of "canned" responses to all the questions that an interrogator could possibly ask during the fixed time-span of the test. Like Parry, this machine would produce answers to the interviewer's questions by looking up appropriate responses in a giant table. This objection--which has in recent years been revived by Ned Block, Stephen White, and myself--seems to show that in principle a system with no intelligence at all could pass the Turing test.

In fact AI has no real definition of intelligence to offer, not even in the sub-human case. Rats are intelligent, but what exactly must a research team achieve in order for it to be the case that the team has created an artefact as intelligent as a rat?

In the absence of a reasonably precise criterion for when an artificial system counts as intelligent, there is no way of telling whether a research program that aims at producing intelligent artefacts has succeeded or failed. One result of AI's failure to produce a satisfactory criterion of when a system counts as intelligent is that whenever AI achieves one of its goals--for example, a program that can summarise newspaper articles, or beat the world chess champion--critics are able to say "That's not intelligence!" (even critics who have previously maintained that no computer could possibly do the thing in question).

Marvin Minsky's response to the problem of defining intelligence is to maintain that "intelligence" is simply our name for whichever problem-solving mental processes we do not yet understand. He likens intelligence to the concept "unexplored regions of Africa": it disappears as soon as we discover it. Earlier Turing made a similar point, saying "One might be tempted to define thinking as consisting of those mental processes that we don't understan"'. However, the important problem remains of giving a clear criterion of what would count as success in strong artificial intelligence research.

The Chinese Room Objection One influential objection to strong AI, the Chinese room objection, originates with the philosopher John Searle. Searle claims to be able to prove that no computer program--not even a computer program from the far-distant future--could possibly think or understand.

Searle's alleged proof is based on the fact that every operation that a computer is able to carry out can equally well be performed by a human being working with paper and pencil. As Turing put the point, the very function of an electronic computer is to carry out any process that could be carried out by a human being working with paper and pencil in a "disciplined but unintelligent manner". For example, one of a computer's basic operations is to compare the binary numbers in two storage locations and to write 1 in some further storage location if the numbers are the same. A human can perfectly well do this, using pieces of paper as the storage locations. To believe that strong AI is possible is to believe that intelligence can "emerge" from long chains of basic operations each of which is as simple as this one.

Given a list of the instructions making up a computer program, a human being can in principle obey each instruction using paper and pencil. This is known as "handworking" a program. Searle's Chinese room objection is as follows. Imagine that, at some stage in the future, AI researchers in, say, China announce a program that really does think and understand, or so they claim. Imagine further that in a Turing test (conducted in Chinese) the program cannot be distinguished from human beings. Searle maintains that, no matter how good the performance of the program, and no matter what algorithms and data-structures are employed in the program, it cannot in fact think and understand. This can be proved, he says, by considering an imaginary human being, who speaks no Chinese, handworking the program in a closed room. (Searle extends the argument to connectionist AI by considering not a room containing a single person but a gymnasium containing a large group of people, each one of whom simulates a single artificial neuron.) The interogator's questions, expressed in the form of Chinese ideograms, enter the room through an input slot. The human in the room--Clerk, let's say--follows the instructions in the program and carries out exactly the same series of computations that an electronic computer running the program would carry out. These computations eventually produce strings of binary symbols that the program instructs Clerk to correlate, via a table, with patterns of squiggles and squoggles (actually Chinese ideograms). Clerk finally pushes copies of the ideograms through an output slot. As far as the waiting interogator is concerned, the ideograms form an intelligent response to the question that was posed. But as far as Clerk is concerned, the output is just squiggles and squoggles--hard won, but completely meaningless. Clerk does not even know that the inputs and outputs are linguistic expressions. Yet Clerk has done everything that a computer running the program would do. It surely follows, says Searle, that since Clerk does not understand the input and the output after working through the program, then nor does an electronic computer.

Few accept Searle's objection, but there is little agreement as to exactly what is wrong with it. My own response to Searle, known as the Logical Reply to the Chinese room objection, is this. The fact that Clerk says "No" when asked whether he understands the Chinese input and output by no means shows that the wider system of which Clerk is a part does not understand Chinese. The wider system consists of

Clerk, the program, quantities of data (such as the table correlating binary code with ideograms), the input and output slots, the paper memory store, and so forth. Clerk is just a cog in a wider machine. Searle's claim is that the statement "The system as a whole does not understand" follows logically from the statement "Clerk does not understand". The logical reply holds that this claim is fallacious, for just the same reason that it would be fallacious to claim that the statement "The organisation of which Clerk is a part has no taxable assets in Japan" follows logically from the statement "Clerk has no taxable assets in Japan". If the logical reply is correct then Searle's objection to strong AI proves nothing.

For More Information About AI Read my book:

Artificial Intelligence: A Philosophical Introduction

Jack Copeland

Oxford UK and Cambridge, Mass.: Basil Blackwell, September 1993, reprinted 1994, 1995, 1997 (twice), 1998, 1999 (xii, 320). Translated into Hebrew (1995), Spanish (1996). Second edition forthcoming in 2001.

To view some of the book

Order this book through: Blackwell | Amazon.com | Barnes & Noble | Borders