Steven Pinker (Linguist / Psychologist)

““The main lesson of thirty-five years of AI The main lesson of thirty-five years of AI research is that the hard problems are easy research is that the hard problems are easy and the easy problems are hard. The mental and the easy problems are hard. The mental abilities of a four-year-old that we take for abilities of a four-year-old that we take for

granted – recognizing a face, lifting a pencil, granted – recognizing a face, lifting a pencil, walking across a room, answering a walking across a room, answering a

question – in fact solve some of the hardest question – in fact solve some of the hardest engineering problems ever conceived.... ”engineering problems ever conceived.... ”

Steven Pinker (Linguist / Psychologist)

““As the new generation of intelligent As the new generation of intelligent devices appears, it will be the stock devices appears, it will be the stock

analysts and petrochemical engineers and analysts and petrochemical engineers and parole board members who are in danger of parole board members who are in danger of being replaced by machines. The gardeners, being replaced by machines. The gardeners, receptionists, and cooks are secure in their receptionists, and cooks are secure in their

jobs for decades to come.”jobs for decades to come.”

Summing up 50 years’ progress in AI(From Part I of Course)

Pinker says we’re successful on “hard” problems, but not the “easy” We can say more:

More and more progress on the “hard” problems seems to be taking us no closer to solving the “easy” ones

We’re able to tackle specific specialist problems,i.e. Engineer a solution to a specialist problem

But the more we go into them, the further we get from the original goal of AI (“original goal” = AI as good as a human)

Like language moving more shallow than deep We move more to specific techniques,

but gain no insight into general intelligence

What about general purpose AI?

Matt Ginsberg, 1995reported in SIGART bulletin

Vol 6, No.2 April 1995

““AI is an Engineering AI is an Engineering discipline discipline

built on an unfinished built on an unfinished Science.”Science.”

The Science and Engineering of AI AI has an Engineering aspect and a Science aspect

Engineering: Build stuff that works, serves a practical function Physical:

– Bridge, Aeroplane

Information Processing: – Translation system, Autonomous Vehicle

Science: Discovery of knowledge; general truths and laws Physical:

– Mechanical forces, stresses, tension, material strength, aeronautics

Information Processing: We have some science …– Speed of certain routines (Computer Science)

– Limits and abilities of certain learning algorithms

… but we would really like a “Science of Intelligence”

The Science and Engineering of AI Good Engineering should rest on a solid scientific foundation

AI’s foundation looks a bit shaky…

Consider something like bridge building: Science exists, can make it strong enough to hold a certain load,

know how many pillars/cables etc. needed Similar for Aeroplanes. Science also exists, how many engines,

power, aerodynamic shape etc.

What about AI problems? For Machine Translation: only have science for some subtasks:

parsing, n-gram language model For Natural Language Understanding: not even sure how to describe

the problem!

Yet more ambitious: What we really want to build is something intelligent What about the Science of Intelligence? AI seems obsessed with better and better engineering

Drosophila Drosophila = Fruit Fly Drosophila Melanogaster heavily

used in research in genetics Small, easy to grow in laboratory Short generation time (two weeks) Only four pairs of chromosomes:

easy to study Genome sequenced in 2000

Some say Chess is Drosophila of AI Easy to study Studied a lot

““Chess is the Drosophila of artificial Chess is the Drosophila of artificial intelligence. However, computer chess intelligence. However, computer chess has developed much as genetics might has developed much as genetics might

have if the geneticists had have if the geneticists had concentrated their efforts starting in concentrated their efforts starting in 1910 on breeding racing Drosophila. 1910 on breeding racing Drosophila.

We would have some science, but We would have some science, but mainly we would have very fast fruit mainly we would have very fast fruit

flies.”flies.”

John McCarthy

AI seems obsessed with better and better engineeringWhere is the Science of Intelligence?…

What is Intelligence? There is no widely agreed-upon scientific definition of intelligence Try some dictionary definitions…

Understand world, reason about it Able to use knowledge to manipulate it ( to achieve any desired end) Profit from experience (i.e. not static, improving all the time, learning)

There seems to be an internal aspect Understand, reason

Difficult to come up with a precise definition for what this is What constitutes adequate “understanding”? Tied up with human “meaning” of things in world

There seems to be an external aspect Manipulate the world

Difficult to come up with a precise definition for what this is Manipulate what exactly? And manipulate it in what way? Tied up with external objects/forces/relationships in the world

We would like some clear abstract theory of “processing information” Not tied up with human meanings of internal processes Not tied up with external world objects

What is Artificial Intelligence? (See have the AI guys done any better for a definition) Definitions tied up with internal processes:

To automate ” …activities that we associate with human thinking, activities such as decision making, problem solving, learning...”(Bellman,1978)

“The exciting new effort to make computers think … machines with minds” (Haugeland, 1985)

”The study of mental faculties through the use of computational models.” (Charniak and McDermott, 1985)

”The study of computations that make it possible to perceive, reason, and act.” (Winston, 1992)

Definitions tied up with external world objects: ”The art of creating machines that perform functions that require intelligence

when performed by people.” (Kurzweil, 1990) ”The study of how to make computers do things at which, at the moment, people

are better.” (Rich and Knight, 1991) ”AI . . . Is concerned with intelligent behavior in artifacts.” (Nilsson, 1998)

AI definitions still tied up with poorly defined external or internal stuff AI definitions bring in a new aspect:

Explicit mention of humans

Not very helpful!

Towards a Scientific Definition of Intelligence

What would a precise definition of intelligence look like? Can expect it to be similar to the definition of

communication Also a human activity, very complicated with lots of human

meaning …But can be studied purely abstractly as a mathematical

problem Possibly a good example for AI because

Both are about processing information Unlike Physics/Chemistry/Biology

where theories are about physical objects/forces/processes

““The fundamental problem of communication is The fundamental problem of communication is that of reproducing at one point either exactly that of reproducing at one point either exactly

or approximately a message selected at or approximately a message selected at another point. another point.

Frequently the messages have meaning; that is Frequently the messages have meaning; that is they refer to or are correlated according to they refer to or are correlated according to

some system with certain physical or some system with certain physical or conceptual entities. These semantic aspects of conceptual entities. These semantic aspects of

communication are irrelevant to the communication are irrelevant to the engineering problem. engineering problem.

The significant aspect is that the actual The significant aspect is that the actual message is one selected from a set of possible message is one selected from a set of possible

messages. messages. The system must be designed to operate for The system must be designed to operate for

each possible selection, not just the one which each possible selection, not just the one which will actually be chosen since this is unknown at will actually be chosen since this is unknown at

the time of design.”the time of design.”Claude Shannon,“A mathematical theory of communication”,

1948

What if there is no theory? Maybe there is no “clean” theory of Intelligence

Maybe it’s just some stuff that happens in your head

Gravity, Electromagnetism, Light, Motions of planets, etc. all have “clean” theories

…but there’s no reason why intelligence must have a clean theory

Human intelligence evolved over millions of years Could well be just a messy load of neuron wiring that is intelligence

David Marr (1945-1980) described “Type 1” and “Type 2” theories…

Marr’s “Personal View” Two types of theory Type 1 “clean” theories

Clear what and how What: Clear description of what input needs to get transformed

to what output Different programs (how) could solve the same computational

problem (what)

Type 2 “messy” theories Problem is solved by the simultaneous action of a considerable

number of different processes, whose interaction is its own simplest description

There is no reason why all theories should be Type 1

(Marr acknowledges that it is not a pure dichotomy

a spectrum of possibilities exists in between 1&2)

Marr’s “Personal View” Progress in AI can consist in

1. Isolate an information processing problem

2. Formulate a computational theory for it (what)

3. Construct a program that implements it (how)

Example1. Find shape from shading in an image

2. Mathematical description of how input related to output

3. Working program

Part 2 tells you what and explains why This never needs to be reformulated Like a result in mathematics, or hard natural sciences

Part 3 tells you how (often many options)

Marr’s “Personal View” Progress in AI can consist in

1. Isolate an information processing problem

2. Formulate a computational theory for it tells you what and explains why it’s important

3. Construct a program that implements it tells you how

Marr criticises “Mimicry” Behaviour:

Mimic some aspect of human behaviour (chatterbot, IF-THEN rules)

Structure: Mimic some aspect of low level structures (neurons)

Problem is they are studying “how” (3) without any clear idea of “what and why” (2)

Marr’s “Personal View” Marr criticises “Mimicry”

Behaviour: Mimic some aspect of human behaviour

(chatterbot, IF-THEN rules)

Structure: Mimic some aspect of low level structures (neurons)

Problem is they are studying “how” (3) without any clear idea of “what and why” (2)

No need to copy flapping or feathers to fly Need to study “what” flight is Not “how” bird is built

Marr’s “Personal View” But remember, the breakdown only works for Type 1 theories

1. Isolate an information processing problem

2. Formulate a computational theory for it (what)

3. Construct a program that implements it (how)

For Type 2 what and how are tangled

Some dangers…

Going for Type 2 theories when Type 1 exist Can get something that works,

But sheds no light on the Type 1 theory if there is one

(?) Maybe this is what AI has been doing (Part I of this course)

Looking for Type 1 theory when the problem is messier Type 1 theory might approximate a real Type 2 process

Might be refusing to see the reality because there seems to be a nice elegant theory (which is wrong)

What if there is no Type 1 theory? Some science would help the Engineering effort of building systems

…but if science is hard to formulate, then…

Why not just keep building stuff that works, serves a practical function?

We have seen from Part I of course… There seem to be severe limits on what we can do by building specific

systems– Natural Language Understanding

– Recognising objects in vision

– Adapting old knowledge to new problems

– Having commonsense

Doesn’t look like we are getting closer to general solutions

Even if we can’t find a clean Type 1 theory for all of intelligence It might still be worthwhile to take a more scientific approach

Rather than Engineering all the time

Where to Find Inspiration? It looks like we should step back from specific problems

Diagnosing diseases, recognising vowels, playing chess, recognising faces…

We should also step back from specific techniques Search, logic, neural network, genetic algorithm…

We need to look at the big picture of what intelligence is

Where can we get some hints about this?

Cognitive Science…

But always bear in mind that we want to be clear about what we are doing and why. We don’t want to mimic behaviour or structure for its own sake

Cognitive ScienceDefinition:“the scientific study either of mind or of intelligence”

Essential Questions What is intelligence? How is it possible to model it computationally?

Takes ideas from Psychology Philosophy Linguistics Neuroscience Artificial Intelligence / Computer Science Maybe also minor contributions from:

Anthropology, Sociology, Emotion studies, Animal Cognition, Evolution