Knowledge

CSM10 Spring Semester 2007

Intelligent Information Systems

Professor Ian Wells

The journey so far ...

• intelligence ...

• perception (and cognitive processes) ...

• knowledge ....

2

Intelligence

• behaviour admired but not understood

• perception + cognition + motor control

• perception + knowledge + inference + action

3

Human

Computer

Cognitive processes

• perception

• influence of context

• categories and classes

• impose structure on real world

• problem solving

• need to identify optimal approach

4

1

2

3

4

Knowledge

• acquisition

• where does knowledge come from?

• representation

• why and how of representing on a computer

• reasoning

• problem to solution, limits & degradation

• dissemination

• understanding user and control

5

Representation

• semantic networks

• production rules

• frames

• neural networks

• hybrid expert systems

• cases, fuzzy systems, genetic algorithms ...

6

Book reviews

Recommendations for further study ... or just for enjoyment!

Book review

• good basic text

• different approach

• easy reading!

5

6

7

8

Book review

• another basic text

• closer to lectures

• subset of next book

• good alternative introduction

Book review

• more advanced text

• good for a deeper examination of the subject

Book review

• classic general text on psychology as a whole

• well recommended as a reference book

• relatively easy reading

Book review

• how to apply cognitive psychology to software development

• well recommended for all hard-core coders!

• easy to read and good to dip into from time to time

9

10

11

12

Semantic networks

Everyday knowledge• tree• flow diagram / process map / org chart / clinical algorithm

• matrix• bus / train / lecture timetable• periodic table of elements

• formula• Ohm’s Law (volts = amps x resistance)• power = volts x amps• chemistry: H-O-H C02 CH4

• rule book• cricket, health and safety, car repair manual

• others include?• map, web page (hypertext), book, photograph• video/picture, music, poem

14

Fruit hierarchy

FRUIT- sweet

APPLE- red, yellow, green- round- pips

PEAR- wider at bottom- stem- pips

MACINTOSH- red- round- some green

GOLDEN DELICIOUS

- yellow- round- some green

D'ANJOU- wider at bottom- stem & pips- green

BOSC- wider at bottom- stem & pips- brown

15

Semantic network

• nodes

• objects, concepts, situations

• links (arcs, edges)

• relationships

• specific (ISA), generic (AKO), specialised (HAS-A)

• labelled directed graph (digraph)

• propositional network (T/F)

16

13

14

15

16

San Francisco

Los AngelesHouston

Washington

Chicago

New York

17

sister-of

mother-of

husband-of

wife-of

father-of

friend-of

mother-offriend-of

related-to

Mary

RobertRichard

John

Susan

Carol

18

round

ellipse

balloon

blimp

Goodyear Blimp

aircraft

prop

special

Spirit of St Louis

jet

DC-3 747

Air Force 1

777 Concorde

has-shape

has-shape

is-a is-ais-a

ako akoakoako

ako ako

ako akoako

ako

Mach 2

had-speed

19

20

17

18

19

20

Clinical algorithm

21

• widely used in medicine

• so good keeps being reinvented!

• also known as ...

• criteria map (1970s)

• clinical algorithm (1980s & 1990s)

• care pathway (2000 +)

Advantages

• hierarchies

• causes and planning

• visual understanding

• inheritance

• inferring knowledge

• cognitive economy

• shallow knowledge (crisp reasoning)

22

Disadvantages

• scalability

• maintenance

• logically weak

• heuristically weak

• node identity

• too attractive for inexperienced knowledge engineer

23

Rule-based systems

21

22

23

24

Knowledge representation

• representational adequacy

• able to represent different types of knowledge

• inferential adequacy

• infer new knowledge from old

• inferential efficiency

• focus and direct inferencing mechanism

• acquisitional efficiency

• ease of maintenance and adding new knowledge

25

Procedural languages

• Imperative

• BASIC and Pascal

• FORTRAN and ADA

• C and C++

• Java

• web e.g. ASP and PHP

• application-specific languages

• Functional

• LISP and APL

26

• Declarative

• Object-oriented - Smalltalk

• Logic - Prolog

• Rule-based - R1/XCON, CLIPS, OPS5, ART

• Frame-based - KEE

• Non-declarative

• Rule-master

• Neural networks

Non-procedural languages

27

Post production rules

• Emil Post (Am J Math 65: 1943)

• more details: Giarratano & Riley pages 28 - 31

• any system of mathematics or logic

• can be re-written in ‘production rules’

• output string is transformation of input string

• no control strategy - order of rules not important

• linguistics: known as rewrite rules

• computing: Backus-Naur Form (BNF) notation28

25

26

27

28

Production rules

• antecedent --> consequent

• situation --> action

• person going-on holiday --> take passport

• IF person going-on holiday

• THEN take passport

• IF person going-on holiday AND destination is Nepal

• THEN see nurse about inoculations

29

Control strategies

• Markov algorithm (1954)

• apply group of production rules in order to input string

• repeat from start if production rule succeeds

• terminate if last production rule not applicable

• or production rule followed by terminator (.) succeeds

• Rete algorithm

• developed for OPS at CMU in 1979

• very fast pattern matching

• avoids searching all rules every cycle by storing changes

• also used in CLIPS30

Markov algorithm control

31

RulesSucceeds

Terminator rulesucceeds

No rules succeed

Start

Markov algorithm example

(rule 1) ßxy --> yßx

(rule 2) ß --> ^.(rule 3) ^ --> ß

Apply to the input string ABCNote:

• ^ is null string

• . is terminator symbol (missing in G&R on page 31)

• lower case symbols (e.g. x) represent any character

• Greek characters (e.g. ß) used for punctuation etc32

29

30

31

32

33

Rule Outcome Result

1 Fail ABC

2 Fail ABC

3 Succeed ßABC

1 Succeed BßAC

1 Succeed BCßA

1 Fail BCßA

2 Succeed BCA

2 Terminate

(rule 1) ßxy --> yßx

(rule 2) ß --> ^.(rule 3) ^ --> ß

Production system

Working memory

Input OutputInterpreter

Facts database Rule base

34

Rule-based expert system

External database

External programs

Inference engineDatabase

(facts)Knowledge base

(rules)

Working memory

Explanation system

User interface

User

Developer interface

Expert or knowledge engineer

35

Hybrid rule-based system (PROSE)

User inputConclusionsExplanations

Actions

Strategic layerStrategic rules

Foreign clauses & rules

StaticFacts

DynamicFacts

Deductive layerDeductive rules

Management of uncertainty

Unknowns Deductions

36

33

34

35

36

Prove H is true

(1) if A and B and C then D

(2) if D and F then G

(3) if A and J then G

(4) if B then C

(5) if F then B

(6) if L then J

(7) if G then H

Known facts: A, F

1 - choose strategy

2 - check facts & select rule

3 - fire rule

4 - check conclusion

5 - repeat cycle from 2

37

Forwards

(1) if A and B and C then D

(2) if D and F then G

(3) if A and J then G

(4) if B then C

(5) if F then B

(6) if L then J

(7) if G then H

If A & F are true prove H is true

38

Fire Known

A, F

5 B

4 C

1 D

2 G

7 H

Backwards

(1) if A and B and C then D

(2) if D and F then G

(3) if A and J then G

(4) if B then C

(5) if F then B

(6) if L then J

(7) if G then H

If A & F are true prove H is true

39

Fire Goal

H

7 G

2 D

1 B & C

5 C

4

Aspects of production system

40

what things are

how things work

rules about rules

why and how

adjust for expertise

search, external, terminate

declarative

procedural

meta-rule

explanation

interface

control

37

38

39

40

Perceptual model

Environment

Top down

Expected features

Deductive

Backward chaining

Bottom up

Feature analysis

Inductive

Forward chaining

Reasoning in the real world

Neisser’s cyclic model of perception41

Advantages

• easy to understand and communicate

• natural inference and explanation

• relatively easy to maintain and update

• uncertainly can be included in rule

• each rule is independent of all others

42

Limitations

• complex domains may have large rule bases

• too attractive for experienced knowledge engineer

• search and control can be complex in large systems

• less effective for causal knowledge and scenarios

43

Groups and subjects

41

42

43

44

Ideal group

• administrator

• domain expert

• one or two programmers

• one or two knowledge engineers

45

Subjects from 2000 to 2004

46

Bank loans

Career advice

Holiday destinations

Video production

Plant classification

Mobile phone calling plan selection

Advisory system for configuring PCs

Safety precautions for sub-aqua diving

Mobile phone purchase advice

Child care advice

Digital camera selection

Overseas travel advice

Car purchase advice

Strategy game assistant

Car fault advisor

University selection

Cricket umpiring

Lifestyle and diet

Video rentals

PC product pricing

Football team strategy

Intelligent travel planner for New Zealand

MSc student time advisor and organiser

Historical tour planner for Europe

Intelligent plant identifier for amateur use

House purchase advisor for Guildford area

Subjects for 2005 to 2007

47

Web site development

MSc course selection

Music selection

Cancer likelihood

Diabetes diagnosis/advice

Intelligent examination setter

Student lifestyle advisor

Investment capital advisor

London Underground guide

Accommodation selection

Laptop selection

MSc course selection

Restaurant selection

A -

B -

C -

D -

E -

Memory

It seems ... that we owe to memory almost all that we either have or are; that our ideas and

conceptions are its work, and that our everyday perception, thought and movement is derived

from this source.

45

46

47

48

Memory collects the countless phenomena of our

existence into a single whole; and, as our bodies

would be scattered into the dust of their component

atoms if they were not held together by the

attraction of matter, so our consciousness would be

broken up into as many fragments as we had lived

seconds but for the binding and unifying force of

memory.

Hering 1920

(Lecture to Vienna Academy of Sciences)

50

Memory processes

Encoding RetrievalStorage

Sensory(SM)

Long-term(LTM)

Short-term(STM)

Stimulus input Response

51

Sensory memory

• less than one second duration

• iconic - images

• echoic - sounds

• buffer to provide time to hear whole sentence or to visualize whole image

• Sperling’s experiments (1960)

52

Short-term memory

• limited capacity and duration (5 to 30 seconds)

• Miller’s ‘magical number 7 +/- 2’ (1956)

• immediate memory span experiments

• Peterson & Peterson’s Trigrams (1959)

• ‘chunking’ improves performance

• examples of STM and the limitations it imposes

• telephone dialling programming diagnosis

• eidetic images - do you have this ability?

49

50

51

52

53

Eidetic memory experiment

54

Computing analogy

Encoding

Retrieval

Sensorybuffer

Long-termstorage

Short-termprocessing

Stimulus input Response

Working memory

55

Long-term memory

• procedural - how

• to swim, ride bicycle or horse, fix car

• declarative - what

• mathematics, French, a person

• episodic - when

• personal experience and events (e.g. holiday)

• semantic - why

• general world experience

• flashbulb events

• !wow! - vivid recollection of very specific event

56

Free recall experiment

Murdock 1962: lists of 30 words

1 Position in list 30

0

Rec

all a

ccur

acy

1

Primacy Recency

53

54

55

56

Next week ...

• inside an expert system - MYCIN

• can it outperform doctors in complex cases?

• reasoning and uncertainty in the real world

• project groups and discussion

• introduction to the Penny shell

• note for programmers: bring your laptop computers and make sure you have registered your copy of 4D

57 57