Expert SystemsExpert Systems

Chapters 24 and 26Chapters 24 and 26

Expert SystemsExpert Systems

Expert systems are systems that embody the Expert systems are systems that embody the knowledge of an expertknowledge of an expert• ““know” how to deal with problems in that fieldknow” how to deal with problems in that field

Used for:Used for:• replacing retired employeesreplacing retired employees• training new employeestraining new employees• freeing experts to deal with hard casesfreeing experts to deal with hard cases

Some Expert SystemsSome Expert Systems

MYCIN – medical knowledgeMYCIN – medical knowledge XCON – computer configurationXCON – computer configuration COOKER – Campbell’s soup tureen expertCOOKER – Campbell’s soup tureen expert AARON – an artistAARON – an artist American Express credit approvalAmerican Express credit approval Blue Cross claims processingBlue Cross claims processing

Rule-Based SystemsRule-Based Systems

Systems work by following rulesSystems work by following rules• rules about the domainrules about the domain• rules about answering questionsrules about answering questions

Backward-chainingBackward-chaining• question question answer (like Prolog) answer (like Prolog)

Forward chainingForward chaining• facts & rules facts & rules more facts more facts

RulesRules

ForwardForwardifif

A’s species is S andA’s species is S and

A is a parent of BA is a parent of B

thenthen

B is of species SB is of species S

BackwardBackwardB’s species is S ifB’s species is S if

A is a parent of B andA is a parent of B and

A’s species is SA’s species is S

“Fires” when there is something of known species with a child

“Applied” when we want to know something’s species

Rule VariablesRule Variables

Variables in rules are like Prolog variablesVariables in rules are like Prolog variables• must have one value all the way thru the rulemust have one value all the way thru the rule• are assigned implicitlyare assigned implicitly

Rules easy to do in PrologRules easy to do in Prolog• even forward chainingeven forward chaining

Not so easy in LISPNot so easy in LISP• need to deal with variables explicitlyneed to deal with variables explicitly

Pattern Matching for LISPPattern Matching for LISP

Rules are patterns, like Prolog patternsRules are patterns, like Prolog patterns Can match data elements with rulesCan match data elements with rules

• unbound variables match anythingunbound variables match anything• symbols must match same symbolsymbols must match same symbol

Need a way to write patterns in LISPNeed a way to write patterns in LISP Need a function to match patterns to dataNeed a function to match patterns to data

Sample Pattern LanguageSample Pattern Language

Data will appear as listsData will appear as lists• (colour apple red)(colour apple red) c.f.c.f. colour(apple, red) colour(apple, red)• (colour orange orange)(colour orange orange)

Patterns will have atoms and variablesPatterns will have atoms and variables• variable is a list: (? variable is a list: (? namename))• same name = same variablesame name = same variable• (colour (? fruit) red)(colour (? fruit) red)• (colour (? x) (? x))(colour (? x) (? x))

MatchingMatching

To see if a pattern matches a piece of dataTo see if a pattern matches a piece of data• line them up – just like Prologline them up – just like Prolog((colourcolour (? fruit)(? fruit) redred) = () = (colourcolour appleapple redred))((colourcolour (? fruit)(? fruit) red) red) ( (colourcolour orangeorange orange) orange)((colourcolour (? x)(? x) (? x)) (? x)) ( (colourcolour appleapple red) red)((colourcolour (? x)(? x) (? x)(? x)) = () = (colourcolour orangeorange orangeorange))

Matching FunctionMatching Function

Assume first argument is a patternAssume first argument is a pattern• second argument is just data – no variablessecond argument is just data – no variables

Return a list of “bindings”Return a list of “bindings”• what value does each variable getwhat value does each variable get

> > (match ‘(colour (? fruit) red) ‘(colour apple red))(match ‘(colour (? fruit) red) ‘(colour apple red))

((FRUIT APPLE))((FRUIT APPLE))

> > (match ‘(colour (? x) (? x)) ‘(colour orange orange))(match ‘(colour (? x) (? x)) ‘(colour orange orange))

((X ORANGE))((X ORANGE))

Matches to GroundMatches to Ground

Patterns may have no variablesPatterns may have no variables• (match ‘(colour apple red) ‘(colour apple red))(match ‘(colour apple red) ‘(colour apple red))• should succeed iff equalshould succeed iff equal

Ground patterns produce no bindingsGround patterns produce no bindings• the list of bindings is empty = NILthe list of bindings is empty = NIL

Need a special symbol for failureNeed a special symbol for failure• FAILFAIL

ExerciseExercise

What value is returned by each of the What value is returned by each of the following?following?• (match ‘(parent mark (? p)) ‘(parent mark bob))(match ‘(parent mark (? p)) ‘(parent mark bob))• (match ‘(colour (? f) (? c)) ‘(colour apple red))(match ‘(colour (? f) (? c)) ‘(colour apple red))• (match ‘(weight sarah (? w)) ‘(height sarah 2))(match ‘(weight sarah (? w)) ‘(height sarah 2))• (match ‘((? r) alex mark) ‘(parent alex mark))(match ‘((? r) alex mark) ‘(parent alex mark))

Matching in AbstractMatching in Abstract

Match is a simple functionMatch is a simple function(defun match (p d &optional bindings)(defun match (p d &optional bindings)

(cond(cond

((((atomsatoms p d) p d) ((match-atomsmatch-atoms p d bindings)) p d bindings))

((((variablevariable p) p) ((match-variablematch-variable p d bindings)) p d bindings))

((((listslists p d) p d) ((match-piecesmatch-pieces p d bindings)) p d bindings))

(T(T 'FAIL)))'FAIL)))Note the use of function abstractionFunction easier to read & debug

Matching TestsMatching Tests

Check parts to see which way to matchCheck parts to see which way to match(defun atoms (p d)(defun atoms (p d)

(and (atom p) (atom d)))(and (atom p) (atom d)))

(defun variable (p)(defun variable (p)

(and (listp p) (eq (first p) ‘?)))(and (listp p) (eq (first p) ‘?)))

(defun lists (p d)(defun lists (p d)

(and (listp p) (listp d)))(and (listp p) (listp d)))

Matching Sub-TasksMatching Sub-Tasks

Atoms match only if they are eqlAtoms match only if they are eql Variables match anythingVariables match anything

• but once bound can only match the same againbut once bound can only match the same again Lists match if each element matchesLists match if each element matches

• need to apply bindings from early in the list to need to apply bindings from early in the list to patterns later in the listpatterns later in the list

• all bindings must appear in the resultall bindings must appear in the result

Matching AtomsMatching Atoms

Atoms match if they are eqlAtoms match if they are eql• if they match, return the bindings listif they match, return the bindings list• else return failelse return fail

(defun match-atoms (p d bindings)(defun match-atoms (p d bindings)

(if (eql p d) bindings ‘fail))(if (eql p d) bindings ‘fail))

Matching VariablesMatching Variables

Need to check for previous bindingsNeed to check for previous bindings• if there, it needs to match the data givenif there, it needs to match the data given• if not, add a new bindingif not, add a new binding

(defun match-variable (p d bindings)(defun match-variable (p d bindings)

(let ((p-binding ((let ((p-binding (find-bindingfind-binding p bindings))) p bindings)))

(if p-binding(if p-binding

(match ((match (extract-valueextract-value p-binding) d bindings) p-binding) d bindings)

((add-bindingadd-binding p d bindings)))) p d bindings))))

Variable Binding AuxiliariesVariable Binding Auxiliaries

Find-binding finds the binding for a variableFind-binding finds the binding for a variable• given (? colour) and (…(colour red)…)given (? colour) and (…(colour red)…)• returns (colour red)returns (colour red)

Extract-value gets the value out of a bindingExtract-value gets the value out of a binding• given (colour red), returns REDgiven (colour red), returns RED

Add-binding adds a binding to the bindingsAdd-binding adds a binding to the bindings• given (? colour), red and ((fruit apple))given (? colour), red and ((fruit apple))• returns ((colour red) (fruit apple))returns ((colour red) (fruit apple))

Why Two Functions?Why Two Functions?

Why not just use one function to get the Why not just use one function to get the variable’s value?variable’s value?• (defun binding-value (v a) (second (assoc v a)))(defun binding-value (v a) (second (assoc v a)))

(defun match-variable(defun match-variable-buggy-buggy (p d bindings) (p d bindings)

(let ((p-value ((let ((p-value (binding-valuebinding-value p bindings))) p bindings)))

(if p-value(if p-value

(match p-value d bindings)(match p-value d bindings)

((add-bindingadd-binding p d bindings)))) p d bindings))))

Matching ListsMatching Lists

Lists match if each pair from the lists matchLists match if each pair from the lists match• done when lists are both null (return bindings)done when lists are both null (return bindings)• if only one list null, failif only one list null, fail• otherwise match the first elementsotherwise match the first elements• if it didn’t fail, carry on using the new bindingsif it didn’t fail, carry on using the new bindings

Bindings generated in first match need to be Bindings generated in first match need to be checked first & then used for later matcheschecked first & then used for later matches• use let to save result so it only gets done onceuse let to save result so it only gets done once

List Matching Pseudo-CodeList Matching Pseudo-Code

If both lists are NULL, return bindingsIf both lists are NULL, return bindings Else if one list null, return FAILElse if one list null, return FAIL ElseElse

• get the result of matching the first elements of get the result of matching the first elements of each list using the current bindingseach list using the current bindings

• if it’s FAIL, return FAILif it’s FAIL, return FAIL• else match the rests of each list using bindings else match the rests of each list using bindings

you just gotyou just got

Matching Lists CodeMatching Lists Code

(defun match-pieces (p d bindings)(defun match-pieces (p d bindings)(cond(cond

((((nullsnulls p d) p d) bindings)bindings)((or (null p) (null d))((or (null p) (null d)) ‘fail)‘fail)(T(T

(let ((r (match (first p) (first d) bindings)))(let ((r (match (first p) (first d) bindings)))(if (eq 'fail r)(if (eq 'fail r)

'fail'fail(match-pieces (rest p) (rest d) r)))))(match-pieces (rest p) (rest d) r)))))

Rule LanguageRule Language

SemanticsSemanticsifif

A’s species is S andA’s species is S and

A is a parent of BA is a parent of B

thenthen

B is of species SB is of species S

In LISPIn LISP(parentage(parentage

((? a) species is (? s))((? a) species is (? s))

((? a) a parent of (? b))((? a) a parent of (? b))

((? b) species is (? s)))((? b) species is (? s)))

“parentage” is the nameso we can talk about it

conclusion appears last in list of patterns

ExerciseExercise

Write a rule in our rule language:Write a rule in our rule language:if a man is unmarried, then he is a bachelorif a man is unmarried, then he is a bachelor• (man mark yes)(man mark yes)• (man bill yes)(man bill yes)• (married mark yes)(married mark yes)• (married bill no)(married bill no)• (bachelor bill yes)(bachelor bill yes)

Knowledge BaseKnowledge Base

We need to keep track of rules and factsWe need to keep track of rules and facts We will keep separate listsWe will keep separate lists

• rules are used to generate new factsrules are used to generate new facts Global variablesGlobal variables

• *rules* for rules, *facts* for facts*rules* for rules, *facts* for facts Functions for adding rules/factsFunctions for adding rules/facts

• remember-ruleremember-rule and and remember-factremember-fact(defun remember-rule (r) (push r *rules*))(defun remember-rule (r) (push r *rules*))

Inference in ActionInference in Action

(hairy-mammals(hairy-mammals

((? a) has hair)((? a) has hair)

((? a) is a mammal))((? a) is a mammal))

((? a) has hair) matches (robbie has hair)((? a) has hair) matches (robbie has hair)• variable a gets bound to robbievariable a gets bound to robbie

Add the fact (robbie is a mammal)Add the fact (robbie is a mammal)

(robbie has hair)(robbie has hair)

factrule

Inference EngineInference Engine

Repeatedly go thru all the rules, looking for Repeatedly go thru all the rules, looking for any that have all antecedents satisfiedany that have all antecedents satisfied• antecedent is satisfied if it matches a factantecedent is satisfied if it matches a fact

If antecedents satisfied, add conclusionIf antecedents satisfied, add conclusion• need to use the bindings for variablesneed to use the bindings for variables• instantiate-variables functioninstantiate-variables function

Stop only when no new facts got addedStop only when no new facts got added

Inference Engine Pseudo-CodeInference Engine Pseudo-Code

1.1. Let new-facts be the set of new Let new-facts be the set of new conclusions we can generate from *rules* conclusions we can generate from *rules* and *facts*and *facts*

2.2. If new-facts is NULL, stopIf new-facts is NULL, stop

3.3. ElseElsea)a) add each element of new-facts to *facts*add each element of new-facts to *facts*b)b) return to step onereturn to step one

Inference EngineInference Engine

(defun inference-engine ()(defun inference-engine ()(do(do ((new-facts((new-facts

((make-new-factsmake-new-facts))(make-new-facts)))(make-new-facts)))

((null new-facts) NIL)((null new-facts) NIL) ((add-new-factsadd-new-facts new-facts))) new-facts)))

(defun add-new-facts (L)(defun add-new-facts (L)(dolist (f L nil) (remember-fact f)))(dolist (f L nil) (remember-fact f)))

Make New Facts Pseudo-CodeMake New Facts Pseudo-Code

Initialize results to the empty listInitialize results to the empty list For each rule in *rules*For each rule in *rules*

• if the elements of the body can be matched to if the elements of the body can be matched to the database under a single binding, add the the database under a single binding, add the conclusion (under that binding) to resultsconclusion (under that binding) to results

• [[use a function that returns the conclusion use a function that returns the conclusion (suitably bound) or NIL(suitably bound) or NIL]]

Return resultsReturn results

Make-New-FactsMake-New-Facts

(defun make-new-facts ()(defun make-new-facts ()

(let ((results ()))(let ((results ()))

(dolist (rule *rules* results)(dolist (rule *rules* results)

(let ((conclusion ((let ((conclusion (apply-factsapply-facts rule))) rule)))

(when conclusion(when conclusion(unless ((unless (is-knownis-known conclusion) conclusion)

(push conclusion results)(push conclusion results)

)) )) )) )) )) ))

Applying FactsApplying Facts

Rule consists of:Rule consists of:• name (ignored)name (ignored)• body (all but first & last elements)body (all but first & last elements)• conclusion (last element)conclusion (last element)

If we can find a single binding that makes If we can find a single binding that makes each element of the body a fact…each element of the body a fact…• return the (bound) conclusionreturn the (bound) conclusion• else return NILelse return NIL

Apply-FactsApply-Facts

(defun apply-facts (rule)(defun apply-facts (rule)(let ((bindings ((let ((bindings (match-to-factsmatch-to-facts ( (bodybody rule)))) rule))))

(when bindings(when bindings((instantiate-variablesinstantiate-variables

((conclusionconclusion rule) rule)bindingsbindings

)) )) )) ))

(defun body (r) (rest (butlast r)))(defun body (r) (rest (butlast r)))

And So On…And So On…

More functions to writeMore functions to write• match-to-factsmatch-to-facts• instantiate-variablesinstantiate-variables• conclusionconclusion• is-knownis-known

We know what each of them doesWe know what each of them does• because we decided alreadybecause we decided already

ExerciseExercise

Write instantiate-variablesWrite instantiate-variables• takes a pattern & bindingstakes a pattern & bindings• returns bound version of patternreturns bound version of pattern> > (instantiate-variables ‘(ratso is a (? r)) ‘((r rat)))(instantiate-variables ‘(ratso is a (? r)) ‘((r rat)))(RATSO IS A RAT)(RATSO IS A RAT)> > (instantiate-variables ‘((? a) (? b)) ‘((a 1) (c 3)))(instantiate-variables ‘((? a) (? b)) ‘((a 1) (c 3)))(1 (? B))(1 (? B))• use use variablevariable, , find-bindingfind-binding, and , and extract-valueextract-value

Hints on the SolutionHints on the Solution

Recursion or mapcarRecursion or mapcar If the element is a variable with a value, If the element is a variable with a value,

then we need to map it to the value of the then we need to map it to the value of the variablevariable

Else we need to leave it the sameElse we need to leave it the same

Inference StepsInference Steps

When a round of rule-checking adds new When a round of rule-checking adds new facts, another round followsfacts, another round follows• just in case any old rules got satisfiedjust in case any old rules got satisfied• once again, all rules checked against all factsonce again, all rules checked against all facts• if more facts added again, process repeatsif more facts added again, process repeats

Not very efficientNot very efficient• satisfactory for small domainssatisfactory for small domains

More Efficient InferenceMore Efficient Inference

For each rule, keep track of the facts it’s For each rule, keep track of the facts it’s interested ininterested in• whenever a fact like that is added, check the whenever a fact like that is added, check the

rule to see if it firesrule to see if it fires Keep track of partially satisfied rulesKeep track of partially satisfied rules

• with their separate bindingswith their separate bindings• avoids checking all the antecedents againavoids checking all the antecedents again

ExtensionsExtensions

Our program cannot answer questions about Our program cannot answer questions about its reasoningits reasoning• can’t explain how it came up with a factcan’t explain how it came up with a fact• (tho it does print out a brief explanation at the (tho it does print out a brief explanation at the

time it makes the inference)time it makes the inference) Keeping track of inferences can helpKeeping track of inferences can help

• need to separate what was inferred from what need to separate what was inferred from what was toldwas told

ExtensionsExtensions

User interface is very poorUser interface is very poor• hard to add facts, ruleshard to add facts, rules• output is not very niceoutput is not very nice

Many LISPs provide user interface toolsMany LISPs provide user interface tools• GUIs can be generatedGUIs can be generated

ExtensionsExtensions

Some backward chaining can be helpfulSome backward chaining can be helpful• especially for recursive rulesespecially for recursive rules• would otherwise go into an infinite loopwould otherwise go into an infinite loop

Self-learning could be usefulSelf-learning could be useful• programs that learn rules on their ownprograms that learn rules on their own• SOAR project at U.MichiganSOAR project at U.Michigan

FinisFinis