csm10 intelligent information systems - university of … · ¥the doctor pr escribes antibiotics...
TRANSCRIPT
Week 4
CSM10Intelligent
Information Systems
Reasoning in the real world
Case study: MYCIN
Coursework project and report
CSM10 Spring Semester 2007
Intelligent Information Systems
Professor Ian Wells
The journey so far ...
• introduction to the module
• what is intelligence?
• cognitive processes
• how we perceive, remember, solve puzzles
• how a computer can solve problems
• representing knowledge using semantic networks and production rules
• reasoning in the real world
• case study (MYCIN), uncertainty, chess, problems
3
Intelligence
Perception & cognitive processes
Knowledge representation
Real-world reasoning
1
2
3
4
Inside an expert system:
MYCIN
A case study of one of the first major expert
systems and an analysis of its performance when
compared with humans at solving complex problems
Once upon a time ...
• it is a dark night in 1972
• you are on holiday in the USA
• you have to be admitted to hospital
• the doctor prescribes antibiotics
• what should you do .... ?
• be very worried ... !!
6
Treatment of infection
• _____ % acceptable
• _____ % questionable
• _____ % clearly irrational
• (Roberts & Visconti: Am J Hosp Pharm 1972)
7
History of MYCIN
• diagnosis of bacteraemias
• suggestions for suitable therapy
• PhD thesis - Edward Shortliffe (Stanford 1974)
• project lasting 10 years + 5 more PhDs
• numerous offspring e.g. EMYCIN, PUFF, CLOT
• only Oncocin was ever used routinely
• foundation for much of today’s KBS technology
8
5
6
7
8
MYCIN
• if ... then production rules
• backward chaining inference
• management of uncertainty
• explanation & justification
• complex but bounded domain
• concrete and assessable output
9
MYCIN rule - English
if: the gramstain of the organism is negative
and: the aerobicity of the organism is anaerobic
and: the morphology of the organism is rod
then: the genus of the organism is bacteroides
with a certainty factor of 0.6
10
MYCIN rule - LISP
($AND(SAME CNTXT GRAM GRAMNEG)
(SAME CNTXT MORPH ROD)
(SAME CNTXT AIR AEROBIC))
(CONCLUDE CNTXT CLASS BACTEROIDES TALLY 0.6)
11
MYCIN rule - Prolog
rule(150, Context, genus, bacteroides,0.6) :-
same(Context, gramstain, gramneg),
same(Context, aerobicity, anaerobic),
same(Context, morphology, rod).
12
9
10
11
12
Certainty factors
• every deduced or requested value has a Cf
• tally of a rule is lowest Cf of its components
• Cf for rule x tally = Cf of new value deduced
• Cf combined = Cf1 + Cf2 x (1 - Cf1)
• if Cf = 1 all alternatives are deleted
13
MYCIN structure
Consultation Program
Dynamic Knowledge
about Patient
Context Tree
Values of Clinical
Parameters
Acquisition details
Static Knowledge
about Disease and Therapy
Properties of Contexts and
Clinical Parameters
Production Rules
Tables and Lists
Explanation Program
Clinical Users
Knowledge Acquisition Program
Infectious Diseases Experts
14
MYCIN objects
Context
PersonCulture
OrganismDrug
Operation
ClinicalParameter
NameSite
MorphologyDoseDate
Value
JonesBloodCoccus
100mg bd20 Jan 2004
Properties
Value formatText
Askable
Certainty Factor
15
MYCIN context tree
Patient-1
Culture-1
Organism-1
Therapy-1
Culture-2
Organism-2
Drug-1
Organism-1
Drug-2
Operation-1
Drug-3
16
13
14
15
16
Objects
Control
Relationships
MYCIN analysis
17
Meta-rules
Backward chaining
Rule categories
Production rules
Certainty factors
Context tree
Clinical parameters
Contexts
MYCIN on trial
• blinded evaluation by Yu et al (1979)
• ten selected disease case histories
• 8 doctors + original + MYCIN
• 10 x 10 conclusions sent to 8 experts
• evaluators not aware MYCIN included
18
MYCIN’s performance
60%
62.5%
60%
57.5%
55%
42.5%
45%
30%
57.5%
65%19
Dept. of Infectious Diseases fellow
University faculty member 1
University faculty member 2
University faculty member 3
University faculty member 4
University faculty member 5
Hospital house officer
Hospital medical student
Actual therapy prescribed
MYCIN
MYCIN’s family tree
20
DENDRAL
PUFFGRAVIDA
CLOTONCOCIN
GUIDON
NEOMYCIN
MYCIN
EMYCIN
17
18
19
20
ONCOCIN
21
• Stanford Oncology Clinic
• routine operation from 1981 to 1987
• 30 to 60 protocols of 40 to 60 pages each
• multiple drugs - up to ten per treatment
• 80% of plans accepted by experts
• used as an advisor by less specialised doctors
• enter their plans for comments; integrated into system
• eventually replaced by newer systems (Protégé, EON)
MYCIN revisited ...
rule 9 states that: the context has a genus neisseria
provided that: the gramstain is gramneg
and: the morphology is coccus
or ...
rule(9, Context, genus, neisseria) :-
same(Context, gramstain, gramneg),
same(Context, morphology, coccus).
trace(Context, Parameter, Value) :-
know(Context, Parameter, Value);
deduce(Context, Parameter, Value);
ask(Context, Parameter, Value).
23
PROSE1 - A RECONSTRUCTION OF MYCIN IN PROLOG============================================
Enter <start.> when ] ? prompt appears
] ?start.
m - menu - details of available routines h - help - instructions for use of Prose system s - select knowledge base f - full consultation of expert system b - abbreviated consultation only k - list the knowledge acquired l - list all rules in memory p - return to Primos - ek - edit knowledge base ep - edit Prose file tn - turn on trace mode tf - turn off trace mode pr - return to Prolog command level
24
21
22
23
24
> h
PROSE.1 - a reconstruction of the essential features of MYCIN
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
‘full’ gives a complete ‘MYCIN-like’ consultation establishing patient,
infection and culture before proceeding to identify the organism
‘brief’ moves directly to the identification of the organism
Confidence Factors -------------------------once the essential patient data has been entered, an optional ‘confidencefactor’ in the range -1 (definitely not) to 1 (definitely true) can beadded to the evidence supplied (a Cf of 1 is assumed if omitted)
eg. > gramneg .75
N.B. once ‘start.’ has been entered, no . is required at the end of each line
25
> s
> choose from : bugs - identification of organisms using MYCIN rules
bugs
> knowledge base loaded ...
source: subset of MYCIN rules from Bramer/Shortliffe
goal: to determine the genus of an organism
> f
------------------------- patient1 -------------------------
[1] Surname of patient : Smith
[2] Sex : Male
[3] Age : 37
26
------------------------- infection1 -------------------------
[4] Enter the date at which the infection first appeared : 23/8/89
The most recent culture associated with infection1
will be referred to as :
------------------------- culture1 -------------------------
[5] Enter the date on which the culture was obtained : 25/8/89
The first significant organism from culture1
will be referred to as :
------------------------- organism1 -------------------------
[6] Enter the gramstain of organism1 ... gramneg
[7] Enter the morphology of organism1 ... rod
27
[8] Enter the aerobicity of organism1 ... anaerobic
Rule 35 succeeds with a confidence factor of 0.6
[9] Enter the site of organism1 ... ?
Please enter one of the following : unknown armpit blood csf urine
blood
[10] Enter the portal of organism1 ... gi .6
Rule 200 succeeds with a confidence factor of 0.54
28
25
26
27
28
PROSE.1 concludes that the genus of organism1 is bacteroides
with a confidence of 81.5 percent
> k
> knowledge acquired during the consultation session:
the gramstain of organism1 is gramneg with a Cf of 1 the morphology of organism1 is rod with a Cf of 1 the aerobicity of organism1 is anaerobic with a Cf of 1 the site of organism1 is blood with a Cf of 1 the portal of organism1 is gi with a Cf of 0.6 the genus of organism1 is bacteroides with a Cf of 0.816
> p> returning to Primos ...
OK,
29
PROSE expert system shell
(Ahmad & Wells 1985)
User input
ConclusionsExplanations
Actions
Strategic layerStrategic rules
Foreign clauses & rules
StaticFacts
DynamicFacts
Deductive layerDeductive rules
Management of uncertainty
Unknowns Deductions
30
PROSE applications
• Clinical Biochemistry and medical knowledge
• original design aim and achievement
• front end to WASP
• Wallingford Storm Sewer Project
• allowed non-expert engineers to use the system
• intelligent tolerancing module for Medusa
• Prime CAD/CAM program
• replaced numerous rule books and tables
• BAC Journeyman project
• something to do with things that fly and go bang
31
Book reviews
Recommendations for further study ... or just for enjoyment!
29
30
31
32
Book review
• alternative basic text
• more technical approach than Negnevitsky
• updated version of early classic!
Book review
• how to apply cognitive psychology to software development
• well recommended for all hard-core coders!
• easy to read and good to dip into from time to time
Expert system shellsIntroduction to Penny
Expert system shells
• an expert system with the knowledge removed
• performance and design can be improved in the process
• first example was EMYCIN
• CLIPS is an expert system language - not a shell
• Penny is an expert system shell written in 4D
• based on production rules and Markov algorithm
36
33
34
35
36
Expert system shell for 4D
Download from www.4dcoop.com37
38
39
Why change from CLIPS?
• look at www.ghg.net/clips/CLIPS.html
• CLIPS is free, well documented & powerful but ...
• still a text-based system
• not easy to use in a real-world setting
• Windows versions not very stable
• MSc students wanted something more modern!
40
37
38
39
40
Advantanges of Penny over CLIPS
• written specifically for this course!
• 4D is a modern and open environment
• plenty of scope for GUI and visual effects
• internal working easier to understand and modify
• teaches how to represent knowledge in a database
• 4D skills more likely to prove useful in the future
• Penny shell can be used ‘as is’ for code-shy groups
41
Disadvantages
• Penny is still evolving
• shortcomings will be taken into account
• 4D has a non-trivial learning curve
• has worked well for those who persevere
• documentation on Penny needs improving
• Penny is a ‘grace and favour’ undertaking
42
Development of Penny
• watch the web site for new versions
• please feed back bugs and RFCs via link
• be patient as Penny matures - but enjoy!
• learn to think creatively in the context provided
• keep in contact with Lo Farnan or myself
43
Design of Penny
• based on classic production rules
• if condition (left hand side)
• then action (right hand side)
• combined with semantic network
• if condition then go to node
• inference uses Markov algorithm (G & Riley)
44
41
42
43
44
Inference• the rule cycle always starts at rule one
• rules are always tested in strict rule number order
• if a rule fires (condition evaluates to true) then its action(s) are executed and rule cycle starts again at rule one
• if a rule action is ‘exit’ then the session terminates gracefully
• if no rules fire then the session terminates with a warning (this should normally be avoided)
• a rule will normally only fire once and will be skipped in subsequent cycles
45
Rules
• each rule contains a single condition and one or more actions (but see below)
• multiple conditions handled by salience (rule execution order)
• ‘headless’ rules are allowed in order to construct semantic networks
• rule sets are provided to determine which rules and facts are to be used in a given session
46
Rule properties• name - used to identify rule especially in ‘goto’ actions
• number - order in which rules are tested during a cycle
• set ID - rule set to which rule belongs
• description - free text description of rule
• creation date - date on which rule was created
• created by - name of person who first created the rule
• revision notes - free text history of changes to the rule
• fact name - name of fact to be tested in rule condition
• operator - used to test fact (named above) against the stored value
• value - stored value against which fact is tested
• value type - type of above
• (string, value-list, text, integer, real, date, time, boolean)47
Rule conditions• equal to - test named fact against stored value
• greater than - test named fact against stored value
• greater than or equal to - test named fact against stored value
• less than - test named fact against stored value
• less than or equal to - test named fact against stored value
• true - test named fact against stored value
• false - test named fact against stored value
• headless - no test performed, rule can only be accessed from a goto action
48
45
46
47
48
Rule actions• each rule must have at least one action
• all actions are executed in the order specified
• assert - set the named fact to the stored value
• askfor - ask for a value and set the named fact to the value supplied
• goto - go to the named rule (which must be headless)
• exit - terminate the session (at least one rule should have this action)
• execute - run the named 4D method (hook for special functions or actions)
• display - display the stored comment
• write - write the stored comment to a text file
• open - open the named document
• play - play the named sound
• run - show the named video clip49
Control options
• buttons in dialog window to control session
• help - display free text comment to provide assistance in answering a question
• why - display free text comment explaining why a question is being asked
• how - display trace of session showing how it has arrived at this rule
• back - roll back the session to the previous rule fired and state of dynamic facts
50
Facts
• facts are addressed by name in the same way as rules
• facts also belong to rule set to determine which will be used in a given session
• static facts are stored in the database and are loaded at the start of a session
• dynamic facts are always cleared at the start of each session
• semi-dynamic facts are ‘remembered’ between successive sessions unless manually cleared
51
Fact properties• name - used to identify fact in rule conditions and actions
• set ID - rule set to which the fact belongs
• description - free text description of the fact
• prompt - help text to assist with answering question associated with fact
• creation date - date on which the fact was created
• created by - name of person who first created the fact
• revision notes - free text history of changes to the fact definition
• value - stored value which is assigned to the fact when session is loaded
• value type - type of the fact
• (must match that set by the rule/rules which test the fact)52
49
50
51
52
Groups and subjects
Week 4 update
Ideal group
• administrator
• domain expert
• one or two programmers
• one or two knowledge engineers
54
Subjects from 2000 to 2004
55
Bank loans
Career advice
Holiday destinations
Video production
Plant classification
Mobile phone calling plan selection
Advisory system for configuring PCs
Safety precautions for sub-aqua diving
Mobile phone purchase advice
Child care advice
Digital camera selection
Overseas travel advice
Car purchase advice
Strategy game assistant
Car fault advisor
University selection
Cricket umpiring
Lifestyle and diet
Video rentals
PC product pricing
Football team strategy
Intelligent travel planner for New Zealand
MSc student time advisor and organiser
Historical tour planner for Europe
Intelligent plant identifier for amateur use
House purchase advisor for Guildford area
Subjects for 2005 to 2007
56
Web site development
MSc course selection
Music selection
Cancer likelihood
Diabetes diagnosis/advice
Intelligent examination setter
Student lifestyle advisor
Investment capital advisor
London Underground guide
Accommodation selection
Laptop selection
MSc course selection
Restaurant selection
A - gastronomic advice
B -
C -
D -
E -
53
54
55
56
Reasoning in the real world
Moving on from puzzles ...
Habituation
“Einstellung - habituation - creates a
mechanised state of mind; a blind attitude
towards problems; one does not look at the
problem on its own merits but is led by a
mechanical application of a used method”
58
From puzzles to the real world
• problem space concepts
• lack of definition
• experts and novices
• knowledge vs intelligence vs wisdom
• algorithms vs heuristics
• encoding and representation
59
Algorithms and heuristics
• an algorithm is a method or procedure that will always solve the problem
• but it may take a VERY long time
• a heuristic is a ‘rule of thumb’ that may well solve the problem quickly but may also fail to provide a solution
• more often than not it will succeed and save considerable time and effort
60
57
58
59
60
Chess as a problem space
• complex but bounded
• well defined rules and problem space
• large pool of experts and novices
• numerous computer simulations
• what makes an expert?
61
Kasparov vs Big Blue in 1997
Kasparov lost by 2.5 games to 3.562
Kramnik vs Deep Fritz in 2002
Kramnik drew 4 games all after a tight finish63
Kasparov vs X3D Fritz in 2003
Kasparov drew 2 games all64
61
62
63
64
Studies with chess• de Groot (1960s) and Simon (1970s)
• consider 30 moves and up to six turns
• board positions from actual games
• masters 91% but novices 41%
• no difference with random positions
• reconstruction and chunking
• chess masters store 100,000 patterns
• and have studied for 10 years
• heuristics (Holding & Reynolds 1982)65
General Problem Solver
• Allen Newell and Herb Simon
• General Problem Solver (1963)
• Human Problem Solving (1972)
• problem-space theory
• means-ends analysis
• progressive deepening
66
Problem-space theory
• problem has initial state and goal state
• number of intermediate states and alternative paths
• mental operators used to move from one state to the next taking account of legal constraints
• knowledge and heuristics to search space
• limitations of cognitive system include capacity of working memory and the speed of retrieval from long term storage
67
Means-ends analysis
• note the difference between the current state
and the goal state
• create an achievable intermediate state (sub-
goal) that will reduce this difference
• select and apply a suitable operator to move to
the intermediate state
68
65
66
67
68
Problem solving
Start
GoalSub-goals
69
Tower of Hanoi problem
70
Physics problem
A block of mass M is dropped from a height h onto a spring of force constant K.
What is, neglecting friction, the maximum distance the spring will be compressed?
71
Physics solutions
• novices vs experts
• surface features - novices
• deep structure - experts
• analysis phase - longer for experts
• forward chaining - experts
• backward chaining - novices
72
69
70
71
72
Deductive reasoning
• impossible for the conclusion to be false if the premises are true
• modus ponens
• if A then B ... given A then infer B
• modus tollens
• if A then B ... if A is false infer B is false
• affirmation of consequent and denial of antecedent
73
Deductive reasoning examples
If it is raining I take an umbrella
It is raining
Therefore I take an umbrella
If it is raining I take an umbrella
If I take an umbrella I lose it
It is raining
Therefore I lose my umbrella
74
Wason’s four card problem (1971)
If a card has a vowel on one side
Then it has an even number on the other side
Which card(s) need to be turned over to prove the rule?
E K 2 7
75
Wason’s four card problem -2
If a student is drinking beer
Then the student must be over 18 years old
Which card(s) need to be turned over to prove the rule?
Beer Coke Age22
Age16
76
73
74
75
76
Inductive reasoning
• improbable that conclusion is false if premises are true
• general conclusion from premises referring to specific instances
• base rate rule
• probability of class membership increases with number of members
• conjunction rule
• probability of a proposition cannot be less if it is combined with another
77
Inductive reasoning example 1
• Linda is 31 years old and single; she is outgoing and
outspoken; she has a degree from Exeter University; she is very concerned with issues of discrimination
• which is more likely?
A - Linda works for a firm of management consultants
B - Linda is an ardent feminist and works for a firm of
management consultants
78
Inductive reasoning example 2
• Linda is 31 years old and single; she is outgoing and outspoken; she has a degree from Exeter University; she is very concerned with issues of discrimination
• does the probability of C increase given B?
A - Linda works for a firm of management consultants
B - Linda is a member of a club where 90% of the members are management consultants
C - Linda is a management consultant
Hilgard pp 328 to 33279
Reasoning strategies
• from facts to conclusions
• forward chaining
• bottom up
• from hypotheses to proof
• backward chaining
• top down
80
77
78
79
80
Perceptual model
Environment
Top downExpected features
DeductiveBackward chaining
Bottom upFeature analysisInductiveForward chaining
Reasoning in the real world
Neisser’s cyclic model of perception81
• work forwards to find what solutions follow from facts
• planning, monitoring, control
• present to future
• antecedent to consequent
• data driven, bottom up reasoning
• better for breadth-first search
• antecedents determine search
• not easy to provide explanations
Forward chaining
82
Backward chaining
• work backwards to find facts that support hypothesis
• diagnosis (medical e.g. MYCIN or general)
• present to past
• consequent to antecedent
• goal-driven top-down reasoning
• depth-first search facilitated
• consequents determine search
• easy to provide explanations
83
Next week ...
• filling the gaps ...
• frames, cases and other new directions
• uncertainty ...
• the real world is like this
• projects - support and feedback
84
81
82
83
84