expert systeivis: working systems and the research...
TRANSCRIPT
---I_ - ---
- 7__.
-. ------I------ _---- .-._ --
Dcccmher 1985 Report No. S’I’AN-CS-86- 1094Also rwdered KSh!U-37
EXPERT SYSTEIVIS:Working Systems and the Research Literature
Hrucc C 13uchanan
Department of Computer Science
St;mfortl UnivwsityStrrnford. CY\ 94305
Knowledge Systems LaboratoryReport No. KSL-85-37
December 1985
EXPERT SYSTEMS:
Working Systems and the Research Literature
Bruce G. Buchanan
Knowledge Systems LaboratoryDepartment of Computer Science
Stanford UniversityStanford, CA 94305
Abstract:
Many expert systems have moved out of development laboratories into field test and routineuse. About sixty such systems are listed. Academic research laboratories are contributingmanpower to fuel the commercial development of AI. But the quantity of AI research maydecline as a result unless the applied systems are experimented with and analyzed.
i
Table of Contentsl.INTRODUCTION 12.EXPERTSYSTEMSINROUTINEUSEORFIELDTESTING 33.THEINTERDEPENDENCYOFAiAPPLICATIONS&AIRESEARCH 104.AIASANEXPERIMENTALSCIENCE 11%ARETHESESYSTEMS"REALLY"EXPERT? 146.CONCLUSIONS 167ACKNOWLEDGMENTS 18&AN EXTENDED t~[11I.IOGRAPHY OF F,XPF.RTSYS'I'F..LIS 20
1
EXPERT SYSTEMS:WORKING SYSTEMS AND THE RESEARCH LITERATURE
Bruce G. Buchanan
1. INTRODUCTION.
.
According to the popular press, expert systems can do everything. But responsible developers know
differently. Although the possibililies for AI programs arc limitless. and the aclual working applications arc
more and more numerous, we have much work to do in order to learn from the applications of Al Jrcndv in
place, so that more comprehensive applications can be delivered. Unfortunately, very few of the applications
will tell us much about AI.
There are now four areas of application of Al that have commercial significance: robotics (i.e., both vision
and manipulation), natural language understanding, automatic programming, and expert systems. This paper
concentrates on expert systems. although all four areas share characteristics of all symbolic reasoning systems.
with similar overall value of applications.
There is no single definition of an expert system. and thus no precisely defined set of programs or set of
literature references that represent work on expert systems. Nevertheless, I have attempted to put together
such lists in an effort to further research and technology transfer.
The major dimensions along which I have defined expert systems [48] are the following:
1. AI METHODOLOGY -- Expert systems are AI programs. That is. they are programs that reasonwith symbolic information and use heuristic (non-algorithmic) inference procedures.
2. HIGH PERFORMANCE -- Expert-level performance is what the designers are attempting toachieve, but this, too. is not always well defined. In narrow problem areas, it is possible toconstruct systems that reason as well as the specialists in those areas. ln some areas, it is beneficialto construct systems that solve only a fraction of the problems that an expert can solve -- but solvethem correctly -- if, for instance, those systems can free an expert’s time for the more difficult
L problems.
3. FLEXlBILITY -- AI programs, generally, are more flexibly designed than algorithmic programs,partly because they have to be in order to allow modification as problems become better defined.In addition to the flexibility needed at design time. it is desirable for expert systems to exhibitflexibility at run time. In particular, the more tolerant they XC of unanticipated input, newcontexts of application, and different kinds of users. the more “expert” they would stem :o be.
2
4. UNDERSTANDABILITY -- Just as an expert can explain his/her reasoning’ , an expert systemshould be able to explain its line of reasoning and the contents of its knowledge base. This, too. isimportant both at development time, for debugging, and at run time. for accepting thereasonableness of the system’s conclusions.
One of the key elements of an expert system that makes possible this dcgrce of flexibility and
understandability is the separation of the knowledge base from the inference engine. This has become the
fundamental organizing principle of all successfU1 work on expert systems. McCarthy’ noted years ago that a
straightforward, modular, declarative representation of knowlcdgc was a prcrcquisitc for a system that could
be told new facts and relations. Because AI systems are often used to help dctine ill-structured problems. they
arc constructed incrementally. Thus representing the knowledge base in a form outside of the main body of
code will make it easier to modifv and explain.
A significant development in research on expert systems was the introduction of framework systems that
provide an inference engine and syntax for knowledge but contain no problem-specific knowledge
themselves. EMYCIN was designed and written in the mid-1970’s by van LMelle [341] as a test of our claim at
Stanford that MYCIN’s inference engine was completely independent of the knowledge base. He developed
generalizations of the tools in MYCIN. and developed new tools. that made EMYCIN a useful environment
for building and interpreting knowledge bases for new problem areas. It is not a framework for building
every kind of expert system. On the contrary, its utility is limited to a class of problems like MYCIN’s:
selecting plausible answers to a problem from a fixed set of alternatives by gathering and weighing evidence
for the alternatives. By fixing the representation of knowledge and the modes of inference. however,
framework systems allow builders of expert systems to start from a substantial base of programs and to
concentrate on formulating the contents on the knowledge bases without having to design new data structures
and programs that manipulate them.
By now, several other framework systems have been built and used in both research and commercial
settings [ 1481. They offer considerable power for experimentation in AI because these systems can be held
constant over several problem areas, or most of the system can be held constant while one part varies. The
commercially available framework systems (see [141]) are built on the same principles. frequently merging
1 Plato. In the Theaererus. dlstlguished a person’s knowrng somethmg fmn merei! be~!eTf It by the abthty to explain the underlgmgreasons for a belief. Similarly. It seems odd to say that a person has el<Penlse In a reasoning task of he/she cannot explain the line ofrcasonmg. S. Savory points out (298) that L’trgll also refers to understandablht) In his phrase “~‘ellX qul potult rerum cognoscere causas.”which m@t be translated as “Happy 1s he who has been able to learn the causes of thmgs ”
‘“Programs wtth Common Sense.” Proceed1qs @‘the S,mporlum on Ihe .~~dmwar~on o/‘Thoughr Processes. IYS8. pp. 77-M. ~1x1repnnted m Semanrlc lnformaflon Processmg, \j. Mansky, Ed., .CllT Press. 19t&
ideas from several paradigms in a single hybrid system. The existence of these commercial tools -- and. more
importantly, of expert systems using these tools -- marks the transition of AI from a purely academic
discipline to a commercially important set of products.
2. EXPERT SYSTEMS IN ROUTINE USE OR FIELD TESTING
Depending on which speakers you believe it has been suggested that only ~tze expert system, at most, is
“really” working (namely Rl) or that there arc I~&eij.s of opcrntlonal systems. Rccause the instituitions
developing systems are slow to publish, and thcrc is so much rnarkcting “h~pc” surrounding :\I, it is difficult
to separate fact from fiction. And there is some fiction. Ho~+cvcr. in many meetings over the last year expert
systems were discussed that have been moved out of ‘1 l;lbor,ltory dcvclopmcnt environment into ticld test.
and some out of field test into routine use. The list below is the result of my having collcctcd the names of
several such systems, and having tried to follow up with addition;tl ~rification of their status.
The list is based on information supplied largely by rcliablc sources among the developers. A few systems
have been included based on strong and unambiguous claims in reputable journals. I have omitted other
systems reported in the literature without a clear indication of status, unless I could talk with someone in
authority who knew the status. A survey on AI in Engineering [315] resulted in a list of expert systems, most
of which are still under development. Several journals, including Expert S~srems, present reports on systems.
but these. too. are largely early prototypes. Classified systems done for the U.S. government have been
omitted also, because it is so difficult to determine their status.
This list is almost certainly incomplete. It is regrettable that published accounts of working systems do not
always exist. I have attempted to supply as many references as possible or. when not available, the name ofthe person who furnished information about status. Additional classified and proprietary systems were
mentioned by representatives at several companies. There are news stories of working systems in Japan which
1 was unable to follow up on. For these reasons, and because many systems are under development, the list is
only a sample of what exists in late 1985, and presumably is much smaller than one we will be able to compile
next year.
My criteria for including a system were that (a) it is based on AI principles. as described above, and (b) it
runs. as well as I could determine. in a tield-test or run-time environment outside the development laboratory.
There may well be some systems on the list whose status J misintct-pretcd: unfortunately, time did not permit
first-hand cxammation of these systems. Therefore this list is meant as evidence that there arc several expert
systems out of the laboratory and in the hands of users.
4
AGRICULTIJRE
SITE EXPERT SYSTEM AND DESCRIPTION SOURCE
ICl WHEAT COUNSELLOR Advise on control of disease WIin winter wheat crops
Virginia Polytechnic Inst. POMME Advise farmers on management of~tpplc [289 Iorchards. including pest managcmcnt, drought control.pcsticidc sclcction. treatment of hinter injuncs
CH l--&l IS’I’RY
SITE EXPERT SYSTEM (1ND I)ESCRlYI’lO~ SOL RCE____-___-_____--__-________________ _____--_-_____--__________--______----__--_----------------- -------_-__-_--_-__-------------------------------
British Gas Advise on appropriate hcrbicidc for specific application Tim l%oyd. ICI - (Sb dncy )
Lawrence Liver-moreNatl. Labs
TQMSTUNE Tune triple quadnlpolc mass spectrometer Carla Wang. I.13 I. [94]
Molecular Design Ltd DENDRAL (parts) Search chemical structure libraries James Nourse. Mol. Designfor substructures [42. 2071
Shell Institute Screen new chemicals for herbicidal properties(Kent England) based on structure- activity relationships
Donald Michie, Turing Institute(Glasgow)
SUNY-Stonybrook SYNCHEM Plan chemical synthesis steps Herb Gelernter, SUNY[122]
5
.
COMPUTERS AND ELECTRONICS
SITE EXPERT SYSTEM AND DESCRIPTION SOURCE__--__-_------_----------------------------------------------------------------------------------------------------------------------------------DFC
DEC
DEC
DEC
DFC
DEC
Fairchild
GTE
Hewlett-Packard
Hughes Electra-Optical &Data Systems
IBM
IBM L
IBM
I.C.L.
XCON,XSEL,XSITE Configure VAX orders, Check ordersfor accuracy, Plan site !ayout
AI-SPEAR Diagnose failures in tlipc drives and suggestprcventivc actions
CAILIS’fO Help manage resources for chip dcsigncrs
CDx Analyze VMS dump tilts after system crashes
DAS-LOGIC Assist circuit designers bithlogic design
NTC Troubleshoot problems related to Ethernet &DECntt networks
PIES Diagnose problems on circuit fabrication line
COMPASS Analyze maintcnancc records for telephoneswitching system and suggest maintenance actions
PHOTOLITHOGRAPHY ADVISOR Troubleshootphotolithography steps in circuit fabrication
HI CLASS Sequence steps in pc board assembly
CSS Aid in planning relocationreinstallation & rearrangement of IBM mainframes
PINE Guide people writing reportson analysis of software problems
YES/W’S lllonitor M VS operating system
Contigure Series 39 computers
[214.215]125 11
Neil Pundit. DEC( Hudson) [30]
Mark Fox. Carnegie Mellon
Neil Pundit. DEC(Hudson)
John McDcmxxt. CarncgicGroup
Neil Pundit. 1)I-C(Hudson) [264J
Marty Tencnbaum.Schlumberger (Palo Alto)
Chuck Rich, MITWI
[ 185,661
WI
A. Way ne Elwood, IBM(San Jose)
A. Wayne Elwood. IBM(San Jose)
Peter Hirsch. IBM (Palo (\lto)[ 135. 142. 2-u)
Tim Boyd. ICL (Sydney)
6
COMPI JTERS AND ELECTRONICS. cont.
I SITE EXPERT SYSTEM AND DESCRIPTION SOURCE
ITT (Germany) Diagnose faults on printed circuit boards Donald Michie, Turing Institute(Glasgow)
Lockheed
Lockheed
NCR
Nixdorf
BDS Troubleshoot baseband distribution subsystemof communications hardware
DIG VOLTAGE TESTER Aid troubleshootingdigital voltage sources in testing lab
OCEAN Check orders for computersystems, contigure orders
FAULTFINDER Diagnose failures in disk drives
Wait Perkins. I .ockhecd(Palo Alto)
Tom Laffey, Lot k heed(Palo Alto)
Barry Plotkin.Tcknowledge
WI
N ixdorf
S.W. Bell
Travelers Insurance
CONAD Check order entry and configure computer systems [297]
ACE Troubleshoot telephone lines [343.229, 3581
DIAG8100 Diagnose failures in DP equipment Luther WeeksTravelers [336]
.CONSUMER SERVICES
SITE EXPERT SYSTEM AND DESCRIPTION SOURCE__-___-_-_-___-----_____________________-------------------------------------- __-_---__-_--------------------------------------------------------Infomart. Dallas INFOMAU ADVISOR Advise shoppers on computer John Alden, TI (Dallas)
purchases
EDUCATION
SITE - EXPERT SYSTEM AND DESCRIPIION SOURCE------------__-_--______________________------------------------------ _____-__-__-__--_----- -----------------------------------------------------DEC TVX Tutor users of VMS operating system Neil Pundit DEC (Hudson) [3I]
1 .ockheed (Sunnyvale) DECGUIDE Tutor designers in design checking Walt Perkins. I,ockheed(Palo Alto)
c;/
XEROX, PARC BUGGY Debug students* subtraction errors Kurt Van Lehn. CMUi [field tested. now dormant] [36 491!I
FINANCIAL,
s I’I’E EXPERT SYSTEM AND DESCRIYIION SOURCE______-_-__-_-___-______________________---------------------------------------------------------------------------------------------------------A IG [American] Advise & support commercial insurance underwriters Peter Hart. SyntclligtnceInternational Group] (e.g. on risks)
First Financial Planning APEX System .\id prot’cscional financial pl;inncrsSystems [‘I‘ravclcrs Ins.] manage clients’ ;ICCOLI ncs
St.Paul Insurance Co. Assess a variety of commcrci;tl insurance risks Pctcr Hart. Syntclligcncc
GEOLOGY
S I’I’E EXPERT SYSTEM ,\NI> DESCRIPTION SOURCE_----_-_-_-_-___-_-------------------------------------------------------------------------------------------------------------------------------Elf-Aquitaine
NASA
NL Indus.
Schlumberger
SITE
EPA +
SECOFOR Advise on drill-bit sticking problems in oil wells Barry Plotkin. Teknowledgc[training tool] WI
GEOX Identify earth surface minerals from Wun Chiou. Lockheedremotely sensed hyperspectral image data (Palo Alto) [58]
MUDMAN Diagnose problems in composition of John McDermott. CMU [ 169)drilling mud during oil well drilling
DIPMETER ADVISOR Analyze oil well logging dataWI
INFORMATION MMWAGEME?JT
EXPERT SYSTEM AND DESCRIPTION SOURCE.__---_----____-____----------------------- ------------------------------------------------------------------EDDAS Advise on disclosure ofcontidential business WI
information
8
MANUFACTURING & ENGINEERING
SITE EXPERT SYSTEM AND DESCRIPTION SOURCE_-________-____-________________________------__-____________________________________------------------------------ _-_---__-____--__-_-__________British Steel Corp.(Scunthorpe rod mill)
Campbcli Soups
Delco Products
Delco Products
DEC
DEC
G E
Hitachi
Kawasaki Steel(Mizushina Works)
Kawasaki S reel
Westinghouse
Westinghouse.
Westinghouse
Xerox. ReprographicsBusiness Group
ICLX Aid technicians diagnose faultsin rod milling process
Troubleshoot problems in soup cookers. anticipate failures
ENGINE COOLING ADVISOR Diagnl>secauses of noise in automobile engine cooling system
MOTOR LIRUSH DESIGNER Construct designof brushes & springs for small elcc. motors
ISA Schedule orders for manufacturing and dclivcry
DISPATCHER Schedule dispatching of parts for robots
CATS Diagnose problems in diesel-electric locomotive
Control railroad train braking for accuracy and comfort
Detect cracks in billets & direct grinding
STOWAGE PLANNER Develop cargo storage plans forwarehous.
VT Configure orders for new elevatorsystems
Nuclear fuel enhancement
ISIS Schcdulc manufacturing steps in job shop
PRIDE Create and analyze new designs for copiers
WI
John ,\ldcn. ‘I’1 (Dallas)[ .308]
Stcvc Dourson, Delco [86]
Steve Dour-son. Delco [277]
Neil Pundit. DEC(Hudson) [25-4]
John McDermott.Carnegie Group
Piero Bonissone,GE (3261
Edward Feigenbaum, Stanford
Akira Miyajima, Kawasaki(Chiba, Japan)
Akira Miyajima. Kawasaki(Chiba. Japan)
John McDermott,Sandra Marcus. ClMU
Donald Michie. Turing Institute(Glasgow) [ 131
Mark Fox. CMU [ 1081
Sanjay Mittal. Xerox(PARC) [235]
9
MEDICINE
S ITE EXPERT SYSTEM AND DESCRIPTION SOURCE----_-------------------------------------------------------------------------------------------------------------------------------------------Hclcna I .abs Serum protein analysis (354
Pacific Medical Ctntcr PUFF lntcrprct pulmonary hmction tests
St. Vincent’s Hospital Intcrprct th>.roid hormone assays WI(Sydney)
Stanford Oncology Clinic ONCOCIN ,Ll,lnagcmenwith cancer
t of therapy for patients E.H. Shortliffc[152. 1801
XIII-ITi\RY3
SITE EXPERT SYSTEM AND DESCRIPTION SOURCE_.---_--____-_---__------------------------------------------------------------------------------------------------------------------------------U.S. Army AALPS Plan optimal loading of equipment & cargo Chuck Rich, MIT
on aircraft WI
. SOFTWARE
SITE EXPERT SYSTEM AND DESCRIPTION SQURCE-_________-_-___-_--____________________---------------------------------------------------------------------------------------------------------Shell Petroleum Intelligent front-end for complex software Donald Mic hie,
Turing Institute (Glasgow)
3Clsssltied s! stems arc not mcludcd here.
10
3. THE INTERDEPENDENCY OF Al APPLICATIONS & Al RESEARCH
The commercial goals of companies applying AI to their problems are decreasing cost and increasing
quality of goods and services. However. the longer-range scientific goal of AI research is to understand howto build intelligent systems better and faster in the titure. In a well-established discipline like physics or
chemistry. it is possible to separate the applications from the research. A chemical engineering company
applies known methods: a research laboratory looks for new ones. In AI and other young disciplines, on the
other hand. there are mitigating circumstances that make progress dependent on closer collaboration between
applications and research.
First, in AI thcrc is still a small .~cyyl~l of cxpcrienced researchers in AI. Thirty years ago there wcrc adozen or so persons working in :!I and defining the field. Ten years ago thcrc wcrc a couple of hundred.
mostly clustered in three university research labs: CMU. MIT, and Stanford. In the first year after the AAAl
was formed in 1979 -- six years ago -- the membership in AM1 has about 400. That is. the supply of persons
with 5-10 years of experience in AI is very small. The annual rate of new PhD’s in Computer Science has
changed little over the last few years -- about 250 per year. Of these, perhaps 60-80 (roughly l/4 to l/3) are
specialized to AI. So the annual increase in the supply of PhD-level researchers is also small.“
Second, as everyone in the AAAI knows, the demand for trained workers far outstrips the supply. There is
intense competition for people who understand the principles of AI and can apply them in practice. Thatdemand comes from three sources: universities, industrial and non-profit research labs. and the new
applications industry..
Universities are starting new Computer Science deparunents with unfilled slots for AI faculty. And
established CS departments are expanding their course offerings in 111. This is healthy, of course. because
some of these faculty will start nm research projects and will have a fraction of their time for research. It is
necessary to increase the number of teachers if we expect an increase in the number of students (although
some university administrators seem to believe otherwise). The disappointing part of the university picture,
however. is that the faculty slots are unfilled. What is more. because of salaries. work loads, and the absencee
of a larger AI community around many schools, these schools will continue to have difficulty hiring AI faculty
in the next several years.
Non-profit research laboratories have a distinguished history of producin, 30 *ood 11i research. Places likeR[j;\jD. l,incoln Labs. HBN and SRI-International hake fostered much of the work that is now being
1 If the growth of the ,\AAI IS any lndicatlon. however, the situauon should ImProle considcrabl! wlthm ti~c !ears. In 1984 the totalmembershIp was 7200. mcludmg 1400 student memberships. In 1985 it was 9.935 members. mcludmg 743 student members.
11
dcvcloped commercially. Industrial research labs, like XEROX-PARC and (more recently) Schlumberger,
are also places where the research spirit fosters excellent AI research. In the last few years, more and more
positions have opened up in these laboratories. About 25 large (Fortune-500) companies posted recruiting
notices at the 1984 AAAI meeting, for example. The only disappointing aspect of this is that the research
positions have largely been filled by AI researchers from universities. As a result. there is nearly a steady state
in the number of persons doing Al rcscarch, and a net loss in the number who are teaching.
By far, though, the largest demand for Al horkcrs comes from the ncu industry growing up to develop and
market Al software. This is as true in robotics and natural language applications ;rs in cspcrt systems.
Overall, this is a healthy development for AI because it is creating many more jobs than the uni\crsitics ;\nd
rcscarch labs could offer. And the availability of employment certainly encourages bright. young pcoplc to
enter the field in the first place.
But there is a possibility of killing the goose that lays the golden egg. as many have pointed out. The N
rcscarch community has effectively lost a number of good people as they are attracted to the commercial side
of AI. This may slow down the research necessary to sustain the industry in its second 5-IO-year period.
Staffing for AI research is now in a period of instability.
This, then, is a fundamental problem in carrying out AI research. The applications community, however,
can help alleviate it. The action item I am proposing to remedy this situation is to experment wirh
applicabons and publish the resuffs. That is, use the applications already being built to make incremental1 progress in the research.
4. Al AS AN EXPERIMENTAL SCIENCEThe paradigm of successful, experimental research has six steps. Too many of us try to take shortcuts since
all six are difficult and time-consuming. But if we’re going to advance the state of the art by experimenting
with applications programs, then we have to consider each application as applied research. That means
following all six steps -- with many iterations and loops among them.
The six are:
1. Define rhe Task. Experimental research at its best is hypothesis testing. Given a well-framed hypothesis,
dctemine the extent to which it is credible. This implies that the experimenter starts with a question and
knows how to recognize an answer.
In AI, unfortunately, we see too many papers whose strongest claim is that the author set out to
12
“investigate” some issue. These papers do not contain results because nothing was analyzed. no comparisons
made, no measurements were taken. In short, this is useless research if future workers cannot determine what
problem the investigator was trying to solve.
2. implement and refine and prototype. The most credible demonstration that ideas in Al have power is
with a running program. We have all seen numerous “proofs” of the right ways to cmpowcr computers with
intelligence. But they lack the con\ incingness of a program that runs.
Unfortunately. an impl~mentLttion of an idea provides only a dcmonstr;ltion ch,lt ti new method is .~clji~*iotlr
for solving a problem. WC cannot show, through the implementation aione, that the method is rlecessagr for
the solution. Subscqucnt cxpcrimentation and analysis are necessary to make the dcmonstmtion of
sufficiency more interesting.
3. Experiment wirh the system. There is not enough well-planned, controlled cxpcrimcn&on in Al. Yet
experimentation is important to establish the robustness of a new method. its scope of applicability, its
weaknesses as well as its strengths. For example, an empirical sensitivity analysis rcvcalcd the extent to which
MYCIN’s rules and reasoning methods did not depend on precise values of certainty factors [47]. Other
studies have tested AI programs against new problems to determine where and why they fail. Others have
systematically varied methods used in a program to compare relative performance.
4. Analyze the issues -- what are the design and implementation features that contribute to success? Which
. are redundant? Where are improvements needed..3 With a complex system like an Al program, data collection
is easier than data analysis. It took a Kepler, if you recall. to interpret the data painstakingly collected by his
professor. Tycho Brahe (a warning to the next generation of students). With an artificially constructed
system, the designer is often in a much better position to analyze the results of experiments because the pointsof brittleness in the design are more easily known to the designer. lt is undesirable and unscientific for us to
accept the words of the designer uncritically: independent verification is important. As a practical matter,
however, few large programs are ever examined or tested by others because of their complexity. This has to
change.
5. Genera/i;e. Some speculation about the generality of methods is dcsirablc. especially if it is backed up
by cvidcncc. This becomes, in cffcct, a testable h)pothcsis that c)rhcrs cdn follow up on. In c~‘eq scicncc,progress results from advancing hypotheses and testing them. :I1 jhould bc no different.
l%ly in the implementation of MYCls. for example. be ciaimcd that the inference method was
sufficiently general to use with other medical or non-medical knoklcdgc bases. Van Mcllc tcstcd his
13
hypothesis and, -after considerable work, made it true.
6. Publish -- what is the good idea? How can others use it?. The quality of publications in AI does not often
reflect the high quality of the researchers. Communicating the results of an analysis is an essential part of
science, yet far too many of our conference and journal papers seem to be descriptions of programs with no
analysis and no results. McCarthy has described these publications as being of the form “LOOK MA, NO
HANDS!” because they announce only that a program works. In order to contribute to AI rcscarch. apublication must bc clear and must identify the reasons why an application is presumed to work well. or the
reasons why it dots not work bcrtcr.
Out of the theoretical l+ork in Al h&is grown a body of knocslcdge about such issues as rhc formal propcrtics
of search algorithms and extensions nccdcd to our logical apparatus to deal with truth maintcnancc and
non-monotonic reasoning. In our own terminology. this rcscarch falls under a model-driven or top-down
rcscrirch strategy. Some of the less charitable call it “development of solutions in search of problems,” which
is always a charge leveled at “pure” research.
The experimentalists in AI are looking at the same issues and have the same ultimate goals as the
thcorcticians: to understand the nature of intclligcnce well enough to build intelligent machines. The
expcrimentalists’ style is data-driven, or bottom-up, based on observed instances of problems and their
solutions as coded in running programs. Whereas the theoreticians start with an issue that needs to be
resolved -- like common sense reasoning -- the experimentalists start with a task whose solution requires some. intelligence. There is naturally some hope that these will meet somewhere in the middle.
Applications have focused attention on some specific issues and have advanced methods to deal with them.
For example, applications have contributed to research on reasoning strategies. explanation. knowledge
acquisition, inexact reasoning, meta-level knowledge, causal reasoning, models of interaction, and validation.
The primary reason for this is that an application is generally unforgiving of shortcuts and simplifying
assumptions. Designing software for persons outside the research laboratory imposes a discipline on AI that it
had not had to face in its early, formative years. And. in the proms. it forces attention to some of the issues
outside the traditional spheres of A[ research: methods for symbolic inference and techniques for
rtprcscnting symbolic knowledge.
*rhc cxpcriments in A[ arc often not well dcsigncd. hobc\er. and the results of the experiments are not easy
to state crisply. But we are still able to learn from the appkmm we build. [It the least. it should not require
much effort to record ideas that failed and decisions to rcimplcment parts of an emerging system. Sensitivity
analyzes can illuminate the sources of power in a system. Comparative studies can help us understand he
14
relative strengths of different architectural choices. An example of this sort of analysis is the one of Rl [217]
or of the Schlumberger Dipmeter Advisor [312]. In every case there is some extra effort required to performexperiments, analyze results, and publish. This cost, if shared among all developers, however, will help insure
against the risk of depleting the pool of AI research talent.
5. ARE THESE SYSTEMS “REALLY” EXPERT?It has been popular to argue against the very idea of artificial intelligence by cl,riming that c\ cn though A I
programs appear at times to bchakc intclligcntly they arc not “really” intclligcnt. ,\ variant of this argument
has surfaced with respect to expert systems: that they only uppear to behave cxpcrtly but arc not “really”
expert. Use of the term “really” is slippery. however. and hides the shifting criteria that arc cmploycd cvcry
time the behavior of the programs improves. There is no reason to take this criticism seriously.
Howcvcr. three substantial criticisms of cxpcrt systems warrant brief discussion hcrc. .l’hcy arc that cxpcrt
systems arc not going to pay off in the long run because they lack three kinds of knowlcdgc:
1. successively deeper layers of knowledge of their task areas to use when the shallow, compiledknowledge fails to reach a satisfactory answer,
2. common sense to avoid errors due to reading the expressions of knowledge too literally or due toincomplete coverage of possibilities within the explicitly stated knowledge base, and
3. knowledge about how to learn from experience.
Expert systems of the current generation do lack these three kinds of knowledge. but that is not to say thatfuture systems will. Nor does that lead directly to the conclusion that today’s systems arc not able to perform
at the level ofexperts and contribute positively in well-defined contexts. Let US look at them separately.
1. Deeper Layers of Knowledge
Rule-based systems encourage encoding judgmental expertise. in the form of empirical associations, in the
knowledge base to help expert systems reason about plausible solutions to problems. This is true also of
frame-based systems, and it can be true of the associations in logic-based systems. ln ,MYCIN. many of the
rules are empirical associations that lack a sound theoretical justificauon, often because medical scientists have
not yet discovered the theory. Other rules are definitional. and thus encode a part of the existing theory of
medicine. Still other rules are theoretically based associations betbcen Cases and effects, which skip the
underlying, “deeper” layers of knowledge that explain and jusufy the assoctations [61]. In this sense, a ru[e
may be “compiled” knowledge in that it accurately allows a system to reason from A to B but has skipped
over the intermediate steps that persons sometimes go through, or appeal to. to justify B in the context of A.
15
One serious manifestation of this problem appears in the explanations that expert systems currently give of
their line of reasoning. While a person can explain a phenomenon at successively deeper layers of detail,
current systems show the individual elements of the knowledge base used to draw a conclusion without
showing the “decompiled” forms that would justify those elements. Another related manifestation appears in
the context of tutoring. A student trying to learn the contents of an expert system’s knowledge base needs
deeper layers of structure to help tie the elements of the knowledge base together [47].
MYCIN’s mlcs. as \sith most current rulebased sl’stems. were urittcn and rctincd ~ih cl \pccttic task in
mind -- diagnosis and therapy in M\‘Cl;\i’s USC. The knowlcdgc base is not gcncr41~ US&II for- cjthcr tasksbut is cngineercd tightly for a single purpose. This is another sense in which ;I set of r-ulcs constitutescompiled knowledge and is a strong argument for a more declarative rcprcscntation of knoulcdgc than ;I set
of rules provides.
MYCIN would be admittedly more knowledgeable if it had more knowledge. in particular. kno\vicdgc of
the physiological and biochemical processes that justify many of its rules. Its cxplanarions clnd tutorialdialogues could be smarter, and the deeper layers could make it easier to build and maintain the knowlcdgc
base. But it would not necessarily perform its tasks of diagnosis and therapy bcttcr in the kind of constrained
context in which systems are now being designed.
2. Common Sense
. McCarthy [212] has argued that MYCIN, and other expert systems. are bound to behave poorly at times
because they lack common sense. He describes an interchange in which MYCIN looks stupid to him because
it fails to object to the possibility of amniocentesis for a male patient. Obviously. this could be remedied witi
the same kind of rule that prev‘ents MYCIN from accepting the possibility of pregnancy for males. But
McCarthy is pointing to a general fault that without common sense, there will always be failures of this simple
kind. (People sometimes are misled when they fail to use their common sense -- so just having it is not
enough.).
McCarthy’s point is well taken: if expert systems know more. particularly if they have more relevant
common sense, they will probably perform better. Current systems often exhibit the kind of brittleness that
McCarthy points out because they make strong assumptions about the context In hhich they will be used, thetypes of users, the vocabulary, the “reasonablcncss” of other lines of rc=oning. dnd SO forth. They also tend
to have rather sharp fall-off in performance at the boundaries of their knowledge. In common parlance, hey
“fall off knowledge cliffs” when we would expect an expert’s performance to degrade gracetilly at tic
boundaries of his or her knowledge.
16
But on the positive side. the context for which an expert system is designed to be used can limit the amount
of common sense that is necessary in practice. For MYCIN, physicians using the system were assumed to
have common sense enough not to tell MYCIN that a male patient has had amniocentesis -- or thousands of
other things that would make MYCIN appear to be stupid. The users were assumed to want help enough to
supply sensible information to MYCIN. Establishing a common framework between the system and its users
is the responsibility of the design team, mostly of the expert. Without a shared vocabulary and shared
assumptions, a system’s recommendations may easily be misunderstood.
3. Lcarninl: Fro111 CIspcricncc
Expert systems currently do not improve their own behavior based on expcricncc: does that mean they are
not “really” expert? This is a dctinitional question on which one may take a dogmatic stand’ . However, if we
use a performance-based detinition of cxpcrtise and not a dispositional one. then we may be less dogmatic- .and say that a person. or a program, is an expert by virtue of excellent performance, rcgardlcss of how he, she
or it gained his/her/its knowledge in the first place (and kept it current). For example. some of us. anyway,
would prefer to have a medical problem diagnosed by a physician with 20 years’ experience who knows most
of what is relevant for our problem -- even if he or she has stopped learning -- instead of having the problem
diagnosed by a recent medical school graduate who knows only little of what is relevant but who is learning
rapidly. When the knowledge curves of the two cross, if we could measure them, then we might change
physicians.
6. CONCLUSIONSEvery limitation of an expert system presents opportunities for research. including the three areas of
criticism listed above. One of the major benefits of focusing sharply on an application is that the limitations
are difficult to ignore and proposed improvements have to pass the operational test of improving the
performance of the expen system. Current AI research in many different areas can mean increased
capabilities for expert systems. For example, research on qualitative reasoning could enhance the reasoning
power of systems over their present capabilities. Also, current work on meta-level reasoning can giveL
programs a better sense of knowing their own limitations -- a form of common sense knowledge.
As a result of research in the last decade, simple. rule-based systems are now strarghtforward to build. They
can be important for helping people solve problems for which expertise is in short supply, or is not well
distributed. or is not available around the clock. More complex problems will require more complex
5 As Schank did In the &Se+Leherer I’V intervIew and In 129%
17
knowledge structures and reasonmg methods, and may require knowledge of a qualitatively different kind.
The next decade should prove to be a time of trying and testing many new ideas for extending the capabilities
of expert systems.
There is a shortage of research people, however, whose charter is to understand the hows and whys of
successful (and unsuccessful) applications. That is why it is necessary for people who are building newapplications to aid in the analysis nnd to publish the results.
Wc arc in many \id)S m d position similar to tlut of bLlSinCSS ddu prowssing 30 yxrs .go. In the
proceedings of a confcrcncu held at Harvard in 1955. there was a report on the first su~cssful application of
computers to a payroll system [303]. It was begun at General Electric in October. 1953 :ind by the time of the
confcrcncc two years later was paying approximately 5.500 hourly cmployccs working under manycombinations of special-case conditions. The parallels with applications of expert systems arc ;triking. I closewith an extcndcd quotation from that report:
“Developing such a program for computer processing involves a tremendous amount ofmeticulous work -- far more than we realized in the beginning. What have been the results’?
1. We proved that the job could be done.
2. We quickly found out that a number of revisions could and should be made to obtaingreater efficiency and lower costs.
3. Cost savings, based on initial performances, would only approximate half of what ouroriginal studies predicted. Displacement of clerical personnel. however, appears to bereasonably close to original estimates.
“If we had this to do 0-r. I think we would again start with the same project. Despite the factthat it is probably one of the most complicated projects. payroll does permit displacement of thegreatest number of clerical personnel and thus helps to defray expensive starting costs. E-‘urther, ithas provided an excellent basis for accumulating a lot of good cxpcricnce on how to usecomputers. . . .
.What Have We Learned in This One Year
of Practical Computer Experience?
“1. ‘I‘he initial ovcrenthusiasm, which inevitably accomp;lnics d project of this scope. can anddots make the job harder. v[‘~o many pcoplc had the imprwh ht Ihis was the answer to allproblems. Perhaps I[ is, but we haven’t been smart enough to deici(JP all of them....
“2. Some of our original thinking has been partly confirmed in mat the greatest benefits to bederived from a computer will probably consist of information impossible to obtain previously....
18
“3. Our experience has shown that the computer is more adaptable to some projects thanothers.... -
“4. Programmers should be recruited within your own company.... It is easier to teach men therequired computer and program techniques than to acquaint them properly with the complexprocedures and routines of modern-day industry....
“5. I doubt if it is possible to overemphasize the desirability of pro\ iding for convcnicntcorrections or dclction of errors in data....
“6. ‘l’hc maximum justifiable amount of flexibility for cxtcnding or integrating ‘ipplicationsmust bc included in the initial programming.
“Albert Einstein once said, ‘Concepts can only acquire content when they arc connected.however. indirectly. with sensible experience. But no logical investigation can rcvcal thisconnection. it can only be experienced.’ Similarly. we feel that our down-to-earth operatingcxpcricnce has given form to our original concepts. Our cxpericncc has verified many of ouroriginal concepts of computer application....
“In conclusion, it is my humble opinion that computers are here to stay. WC have got toincrease our efforts toward understanding them and knowing how best to use [them]. Further, wehave to do more experimenting with new fields that ultimately should utilize the equipment to agreater degree and thus return greater dividends.”
Expert systems are also here to stay. They have their weaknesses, but careful problem selection and design,.
explicit definition of context, and additional research will alleviate them. Even with their limitations, they can
be applied successfully. Finally, the successes will be all the stronger when the limitations are explicitly noted
as opportunities for experimentation and greater understanding.
7. ACKNOWLEDGMENTSThis work was funded in part by the following contracts and grants: b\fWA N00039-83-C-0136,
NIH/SUMEX RR-00785, National Aeronautics and Space ~~dministratlon-,\mes NCC-2-274. BoeingComputer Services W266875. National Science Foundation IST-8312148. and a gift from Lockhcecd.
Parts of this paper were presented at the IEEE Conference on ijpplications of Expert Systems. Denver.
November 1983: the IJCAI tutorial on Expert Systems. Los f2ngeles. AU~USL 1985: and a confcrcnce on
expert systems at the lBIM Scientific Center, Santa Clara, CA. November 1985. Many persons have provided
19
helpful suggestions for improvement, including additions to the list of working systems: Ed Feigenbaum,
Randy Davis, and Don Waterman, in particular. Special thanks to Grace Smith and Anthea Waleson for
preparing the bibliography and manuscript.
20
8. AN EXTENDED BIBLIOGRAPHY OF EXPERT SYSTEMS
The following list includes books, articles, and dissertations of special relevance to the construction of
expert systems. The entire body of AI literature is also relevant, of course. For the most part, articles on
prototype systems and otherwise unpublished technical reports are not included.
111
PI
PI
161
VI
PI
WI
Abadir, M.S. and Breuer, M.A.A knowledge-based system for designing testable VLSI chips.IEEE Des. & Test. Cornput. 2(4):56-68, Aug. 1985.
Addis. T.R.Towards an ‘Expert’ Diagnostic System.Kl. Techtzical Jtd. :79- 105. May, 1980.
d’:\gapcycff. A.A Short Survey of Experl Systetns in UK Busitless.‘l’cchnical Report, The Alvey Directorate. Department of Industry. Millbank Tower. Millbank,
Il.
London, 1984.
Aicllo. N., C. Bock. H. P. Nii, and W. C. White.A GE’ Referetlce Matual.Technical Report, , Stanford Comp. Sci. Memo HPP 81-24. 198
Aiello, N.A Comparative Study of Control Strategies for Expert Systems: AGE implementation of Three
Variations of PUFF.In Proc. AAAI-83, pages l-4. Washington, D.C., August. 1983.
Aikins J. S.Representation of control kgowledge in expert systems.In Proc. AAAI-82, pages 121-123. 1980.
Aikins. J.S.Prototypes and Production Rules: A Knowledge Representation for Computer Consullalions.PhD thesis, Stanford University, August. 1980.
Aikins. J.S.. Kunz. J.C.. Shortliffe. E. H.. and Fallat R.J.PUFF: An expert system for interpretation of pulmonary function data.Comput Biomed Res 16: 199-208.1983.
Aikins, J.S.Prototypical Knowledge for Expert Systems.Artificial Inrelligence 20(2): 163-210, February, 1983.
AIRcport85.The Artificial Intelligence Report.Vol. 2. No. 1, January 1985.
21
WI
Wl
Alexander, J.H., Freiling, M.J., Messick, S.L. and Rehfuss, S. .Efficient expert system development through domain-specific tools.In Fifih Inletnational Workshop on Experl Syslems and their Applicarions. 1985.
Alty, J.L.Use of expert systems.Cotnpuler Aided Engineering Journal 2( 1):2-9, February, 1985.
Applied Artificial Intelligence Reporter.Wcjtingtiwse: $10 million cab ings with expert system software.ICS Rcsc;Irch Institute. C:ni\ . of Miami, 1985, L-01. 2. No. 7.
[l?
WI
WI
116
117
.
118
I
WI
WI
121 1
Aptc. C.V. and Weiss, SM.An ‘tpproach to expert control of interactive software systems.IEEL Trans. on Pattern Analysis arid lllachine ln1elligence 7( 5):586-59 1, Scptcmber. 1985.
Balzer. R., Erman. L.. London, P. and Williams. C.HEARS\?‘-III: A Domain-Independent Framework for Expert Systems.In Proc. MAI-80. Stanford, CA, 1980.
Banares-Alcantara. R., Westerberg, A.W., and Rychencr. M.D.Development of an expert system for physical property predictions.Compur. & Chem. Eng. 9(2):127-142,1985.
Banares- Alcantara, R.DECADE: Design experr for calalysl developmenr.PhD thesis, Carnegie-Mellon University, Dept. of Chemical Engineering, 1985.In preparation.
Barnett. J.A.. and Erman, L.D.Making Control Decisions in an Expert Syslem is a Problem-Solving Task.Technical Report, USC/Information Sciences Institute. April. 1982.
Barr, Avron. Paul R. Cohen and Edward A. Feigenbaum.The Handbook of Arrificial Inrelligence. Volumes I. II. and III.William Kaufmann. Inc., Los Altos, CA. 1981 and 1982.
Barzilay. A.An expert system for tuloring probabilily theon!.PhD thesis, Univ. of Pittsburgh, November. 1983.
Basdcn. A. and Kelly. B.A.An Application of Expert Systems Techniques in %teriais Engincermg.In Proc. Colloquiutn on ‘Applrcation of Knowledge hsed (or E-vrf) .S),stetns ‘. January, 1982.
WI
w1
?
PI
WI
1261
WI
129 I
22
Basden, A.On the application of expert systems.Int. J. Man-Machine Studies 19( 5):461-478, November, 1983.
Bell, M.Z.Why expert systems fail.J. Op. Res. Sot. 36(7):613-619, July, 1985.
Bcndig. H. and Witting. T.Comparison of the statistical and the expert system ,lpproach for the intcrprctation of ship noise.In .4tluplive .Ilerhods III C’rldensulrer .4cousiic’s: Proc. o$.\‘,.l TO .,lJvrlc-ctl .Strr~!,~ IHSIIIUI~~. p;qcs 67 l-7.
NA’: 0. Rcidcl, July-Aug. 1985.Luncburg, Germany.
Bennett. J.S. and Engelmore, R. S.SACON: /I Knowledge-based Consultant for Structural Analysis.In f’roc. Kill-79, pages 47-49. IJCAl, Tokyo, Japan, August, 1979.Stanford Comp. Sci. Report No. HPP 75-23.
Bennett. J.S.. and Goldman. D.CLOT: A knowledge- based consultant for bleeding disorders.Technical Report. Computer Science Department Stanford University, April. 1980.Memo HPP-80-7.
Bennett, J.S.lnfotech State of the Art Report: Machine Intelligence (9). Volume 3: On the Structure of the
Acquisition Process for Rule-based Systems.Pergamon Infotech Limited, Maidenhead. Berkshire, England. 1981. pages 189-204, .
Bennett. J.S. and Hollander, Clifford .R.DART: An Expert System for Computer Fault Diagnosis.In Proc. IJCA1-81, pages 843-845. 1981.Vancouver, B.C.. Canada.
Bennett, J.S.ROGET: A knowledge based system for acquiring the conceptual structure of an expert system.Jnl. Automated Reasoning l( 1). 1985.Also Stanford Comp. Sci. Memo HPP-83-24.
Billmers. M. and Swartout. M.AI-SPEAR: Computer system failure analysis tool.In Proc. European CUR$* otl Ar(ificiu/ Illreiiigence. Italy. Scptcmbcr. 1984.
23
[31] Billmers, M. and Carifio, M.TVX: An expert system advisor for VMS.In Proc. Conf: on Human Inletface. 1985.to be published.
[32] Bobrow, D. G., and Stefik, M.The LOOPS Manual.Report KB-VLSI-8 l- 13, XEROX PARC. 198 1.
[33] Boden. M.A.Expert systems and the common man.In Bevan, N. and Murray, D. (editors). :\lan/muchine integration: State of (he art report, pages 23-30.
Pergamon Infotech, Maidenhead, Berks.. England. 1985.
[34] Bonnet, Alain, and Claude Dahan.Oil-Well Data Interpretation Using Expert System and Pattern Recognition Technique.In Proc. IJCAf-83. pages 185-189. IJCAI, Karlsruhe, West Germany, August. 1983.
[35] Bramer, M.A.Expert Systems: The Vision and the Reality.In M.A. Bramer (editor), Research and Development in Expert Systems, . Cambridge University Press.
Cambridge, 1984.
[36] Brown, J. S. and Burton, R. B.Diagnostic Models for Procedural Bugs in Basic Mathematical Skills.Cognitive Science 2(2):155-192, April-June, 1978.
.(371 Brown, D. C. and B. Chandrasekaran.
Expert systems for a class of mechanica design activity.In Proc. IFIP WGS.2 Working ConJ: on Knowledge Engrneering in Computer Aided Design. 1984.
[38] Brown, D.C.Expert system for design problem-solving using design refinement Hith plan selection and redesign.PhD thesis, Ohio State University, August, 1984.
[39] Brown, H., Tong, C. and Foyster, G.L Palladio: An exploratory environment for circuit design.
Computer Magazine 16( 12):41-56, December. 1983.
[4O] Brownridge, G.. Fitter, l/1., and Sime, IM.The Doctor’s Use of a Computer in the Consulting Room: an Analysis.lnt. J. :\lun-.Clachine .‘irudies 21( 1):65-90, July, 1984.
WI
143
WI
WI
WI
WI
WI
WI
24
Brownston, L., Farrell, R., Kant. E. and Martin. N.Programming Expert Systems in OPSS.Addison- Wesley, Reading, MA, 1985.
Buchanan, B.G. and Feigenbaum. E.A.Dendral and Meta-Dendral: Their Applications Dimension.Artificial Intelligence ll( 1):5-24. 1978.
Buchanan, B.G.Issues of representation in convq ing the scope and limitations of intelligent assistant programs.In J.E. Hayes, D. Michie, L.I. Mikulich (editors), Machine intelligence 9. pages 407-426. John Wiicy,
New York. 1979.
Buchanan, B.G.New research on expert systems.In J.E. Hayes, Donald Michie, and Y-H Pao (editors), ,Ilachine intelligence IO. pages 269-299. Ellis
Hot-wood Limited, Chichester, England, 1982.
Buchanan, B.G. and Richard 0. Duda.Principles of Rule-Based Expert Systems.In M. Yovits (editor), Advances in Computers, pages 163-216. New York: Academic Press, 1983.
Buchanan B.G. et al.Constructing an Expert System.In F. Hayes-Roth, D. Waterman. and D. Lenat (editor), Building Expert Systems. . Addison-Wesley,
New York, 1983.Chapter 5.
Buchanan. B.G., and E.H. Shortliffe.Rule- Based E,upert Systems: The M YCIN Experiments of the Stanford Heuristic Programming Project.Addison- Wesley, Reading, MA, 1984.
Buchanan, B.G.Expert Systems.J. of Automated Reasoning 1(1):28-35, 1985.
Burton, R. R.Diagnosing bugs in a simple procedural skill.[n D. Sleeman and J. S. Brown (editors), lntelligenf Tutoring .J:lvstem.s. pages 157-183. [Icadcmic Press,
New York, 1982.
25
[50] Cantone, R.R., Lander, W.B., Man-one, M.P. and Gaynor. M.W.IN-ATE/2: Interpretating High-Level Fault Modes.In The First Conference on Artificial Intelligence Applicatrons, pages 470-475. IEEE. IEEE Computer
Society Press, December, 1984.
[51] Carhart, R.E.CONGEN: An Expert System Aiding the Structural Chemist.In D. Michie (editor). E-vyerr .J:I.sIc~I)Is [II r/le .Ilm-o Ele/rorlics .Qe. pages 65-82. Edinburgh Uniicrsity
Press. 1979.
[52] Ccndrowska. J. and MA. Bramcr.,1 rational reconstruction of the bt YCIN consultation system.IN. J. Alaw!llachirle Studies 20( 3):229-3 17. %Iarch. 1981.
[53] Chandrasekaran. B.On Evaluating I\ I Systems for Medical l>iagnosis.rl/ Msg. 4( 2). summer. 1983.
[54] Chandrasekaran. B. and Mitral. S.Deep versus compiled knowledge approaches to diagnostic problem-solving.In&. J. Man-Machine Studies 19( 5):425-436. 1982.
[55] Chandrasekaran, B. and Mittal. S.Conceptual representation of medical knowledge for diagnosis by computer: .1lDX and related
systems.In Yovits, M.D. (editor), Advances in Computers, pages 217-293. Academic Press. New York, 1983.
[56] Chandrasekaran. B.Expert systems: Matching techniques to tasks.In W. Reitman (editor). AI Applications for Busmess. pages 116-132. Ablex Publishing Corp., 1984.
[57] Chandrasekaran. B.. Mittal. S., and Smith, J.W.RADEX -- Towards a Computer-Based Radiology Consultant.In Gelsema and Kanal (editor). Pattern Recognrrion in hcrice. pages 363-374. North Holland, , 1980.
[58b Chiou, W.C.NASA image-based geological expert system development project for hkperspectral image analysis.Appl. Opt. 24( 14):2085-91. July, 1985.
[59] Cicsiclski. V. ,.A merhodology for the corlslmcliorl of natural language frofll ejldsft)r ~drcal consuildtion sJ*sterrrs.PhD thesis, Rutgers University. 1980.Also Technical Report CBM-‘TR- 112.
26
[60] Clancey, W.J.Transfer of Rulebased Expertise through a Tutorial Dialogue.PhD thesis, Computer Science Department, Stanford University, September. 1979.techreport #STAN-CS-769.
[61] Clancey, W.J.
.
The Epistemology of a Rule- Based Expert System: A Framework for Explanation.Artificial lrrtelligerlce 20( 3):215-25 1. May. 1983.(Also in Buchanan. B.G. dnd Shortliffc. t:. H. (cds.). Rule B,lscd Expert Slstcms. Reading, \l,\.
Addison-Wesley. 1984.).
[62] Clanccy, W.J. and Lctsingcr. R.NEOMYCIN: Reconfiguring a rule-based expert system for application to teaching.In Clanccy. W.J. and Shortliffc. E. H. (editors). Reudirlgs 111 ,\ledical Artificial lrltellige,lce: The fi‘irst
Decade. pages 361-381. (Addison-Wesley. Reading, 1984.(Originally published in Proc. IJC.-I I-8 1: Vancouver. 1981.).
[63] Clancey. W.J.Knowledge acquisition for classification expert systems.In Proc. ACM Anrlual Colrference, pages 11-14. October, 1984.
[64] Clancey. W.J.The operators of diagnostic strategy.In Proc. Sixth Annual Conference of the Cognitive Science Society. Boulder, CO, June. 1984.
[6S] Clark. K.L, and McCabe, F.G.
. PROLOG: a language for implementing expert systems.In J.Hayes, D.Michie and Y-H Pao (editors), Machine Intelligence 10, pages 455-470. Ellis Horwood
Ltd., Chichester, England, 1982.Appendix by P. Hammond.
[66] Cline. T.. Fong, W., Rosenberg, S. and Walker, M.Photolithography Advisor.SIGA RT Newsletter (92):42-43, April. 1985.
[67] Coombs, M.J. and J. Alty.LExpert systems: an alternative paradigm.lnt. J. Man-LVachine Studies 20( 1):21-43. January, 1984.
[68] Cooper. N.B. and Kidd. A.L.A man-machine interface for an expert system.Br. Telecom Tech&. J. 3( 1): 112- 15, Jan. 1985.
27
[69] Courteille, J.-M.. Fabre, M. and Hollander, C.R.An advanced solution: The drilling advisor SECOFOR.In Proc. 58rh Annual Technical ConJ: and Exhibition. Society of Petroleum Engineers of AIME, San
Francisco, CA, October, 1983.
[70] Cross, S.Qualitative Reasoning in an Expert System.PhD thesis, University of Illinois, 1983.
[71] Cross, S.E.Computer understanding of air traffic control displ,ibs.IEEE Trans. Systems, Man & C),bcrnetics SMC-15( 1): 133- 135. Jlinu~ry - February , 1985.
(721 Cucna. J.The Use of Simulation Models and Human Advice to Build an Expert System for the Defense and
Control of River Floods.In l’roc. I./CA I-83, pages 236-249. IJC\l, Karlsruhe. West Germany. I\ugust. 1983.
[73] Davis, R., Buchanan. B., and Shortliffe. E.Production Rules as a Representation for a Knowledge-Based Consultation Program.Artificial Intelligence 8: 15-45, 1977.
(74) Davis, R.Interactive Transfer of Expertise: The Acquisition of New Inference Rules.Artificial Intelligence 1 l( 1):5-24. Aug, 1978.
[7S] Davis, R.AKnowledge acquisition in rule-based systems: Knowledge about representations as a basis for system
construction and maintenance.In D. A. Waterman and F. Hayes-Roth (editors). Pattern-Directed Inference Systems, pages 99-134.
Academic Press, New York, 1978.
[76] Davis. R.Interactive Transfer of Expertise: Acquisition of New Inference Rules.A I 12(2): 121- 157, August. 1979.
[77] Davis. R.Expert systems: Where are we’? And where do we go from here?The r! I Mugazine 3( 2): l-22. spring, 1982.
[78] Davis, R.Diagnostic Reasoning Based on Structure and Function: hth of Interaction and the I.ocaIity
Principle.Artificial Intelligence 24( l-3):347-410. 1984.
28
WI
WI
WI
WI
WI
WI
Deering, M.F.Architectures for AI.&re : 193-206, April, 1985.
DeGregorio, E.A., Silva. A.H. and Brazil, D.Applied artificial intelligence in the submarine combat control environment.In IEEE Proc. Interrlutional Corlf: on Cyberrletics arid Society. IEEE, 1982.
Dickey, F.J. and Touscaint. ;\.I..ECESIS: An Application of F.upcrt Scstcms to Manned Space Stations.In The First C’or!th-ewe wt Jr.tyic*Lll I~ltelligerrce Applicutiorls. pngcs -183-489. I FEE. I FEE Compllrcr
Society Press. Dccem bcr, 198-L
Dincbas. M.A Knowledge-Based Expert System for Automatic Analysis and Synthesis in CAD.In Proc. IFIP C’orlgress. pages 705-710. 1980.
Dixon, J.R. and Simmons, M.K.Computers that design: expert systems for mechanical cnginccrs.Computers in Mechanical Engineering : lo- 18. 1983.
Dolata, D.P.QED: automated inference in planning organic synthesis.PhD thesis, Univ. of California, Santa Cruz, October, 1984.
Douglass, R.J.A qualitative assessment of parallelism in expert systems.IEEE Sofrware 2(3):70-81. May, 1985.
Dourson. S. and Joyce. J.Building of expert systems to diagnose noise in automotive engine cooling systems.In Proc. of the Conf: on Surface Vehicle Noise and Vibration, pages 19-26. Sot. of Automotive
Engineers, Michigan, May, 1985.
Doyle, Jon.Methodological Simplicity in Expert System Construction: The Cast of Judgments and Reasoned
Assumptions.AI Msg. 4(2), summer, 1983.
Duda. R-O., Hart, P.E.. Nilsson. N. J., and Sutherland. G. L.Semantic Network Rcprcsentations in Rule-Based Infcrencc Systems.In Donald I1. Waterman and Frederick Hayes-Roth kdltors). l’lrttern-Directed Iilference Jj-sterns,
pages 203-221. Academic Press. New York. 1978.
29
1891
WI
p11
192
193
WI
1951
WI
WI
.
WI
WI
Duda, R.O.. 1. Gaschnig, and P. Hart.Model Design in the Prospector Consultant System for Mineral Expioratlon.In D. Michie (editor), Expert Systems in the Microelectronic /lge. pages 153- 167. Edinburgh
University Press, Edinburgh, 1979.
Duda, R.O., and J.C. Gaschnig.Knowledge-Based Expert Systems Come of Age.Byte 6(9):238-281. Septcmbcr. 198 1.
Duda. R.O. and Shortliffc. E. H.Expert systems rcscarch.Scierxe 220:26 1-268. April. 1953.
Dqm. C.L. and Mittal. S.Knowledge acquisition from multiple experts.A I Mug. 6(2):32-37. Summer. 1985.
Electronic Design.Languages for expert systems seek to combine simplicity and ct‘ticicncy.Electronic Design 33(7): 108 ff.. March. 1985.
Energy & Technology Review.Artificial Intelligence in Chemical Analysis.February, 1985.
Ennis, S. P.Expert Systems: A User’s Perspective of Some Current Tools.In Proc. AA AI-82, pages 319-321. August. 1982.
Eur. Digital Management Rep.The answer lies in the shell (expert system development. ICI’S Wheat Counscllor).June 1985, pp. 2-6.
Feigcnbaum, E. A.The Art of Artificial Intelligence: I. Themes and Case Studies of Knowledge Engineering.In Pruc. IJCAI-77, pages 1014-1029. Cambridge, :Mass.. August. 1977.
Feigcnbaum. E. A., and Pamela McCorduck.The Iq’ifrh Generation: Artificial Irltelligerxe and Japarr ‘s (‘~~a1lew to the Verdi.Addison- Wesley, Reading. MA. 1983.
Fcinstein. J.L. and Sicms. F.EDAAS: An expert system at the US Environmental Protection .\gency for Ltk oiding disclosure of
contidcntial business information.Expert S>lsterns 2( 2):72-85. April. 1985.
(100
[ 1021
uo31
WI
WI
WI
.
WI
.WI
3 0
Fenves. S.J., Bielak. J., Rehak. D., Ruchener. M.. Gram. D. and Maher. M.L.An expert system for diagnosis and repair of an automated transportation system.Technical Report, Carnegie-Mellon Univ., Robotics Institute, January, 1985.
Fickas, S.Design issues in a rule-based system.IEEE Trarlsactions OH S~~stems. ,llan and CJ*bernetics ShlC-15(2):208-215. March/April, 1985.
Fikcs, R. and Kchlcr. T.The role of frame based rcprcwltation in rcasontns.CXX ZS( 9):904-920. Sclprcm tw. PBS.
Finin. -I’.. McAdams. J. and Klcinocky. P.FORES’l’--An Expert Shstcm for hcomatic l’cst Fquipmcnt.In The ki’rst Corlftirence on Artificial lrrlelligerlce .4ppiicatiorrs. pages 350-356. IEEE. IEEE Computer
Society Press. Dcccm bcr. L98-l.
Fink, PK., L.usth, J.C. and Duran. J.W.A general expert system design for diagnostic problem solving.IEEE Trans. ori Pattern Analysis and ,Ilachirre Intelligence 7(5):553-560. Scptcmbcr. 1985.
First. M.B., Soffer. L.J. and Miller, R.A.QUICK (Quick Index to Caduceus Knowledge): Using the Internist-l/Caduccus knowledge base as
an electronic textbook of medicine.Comput. and Biomedical Research 18( 2): 137- 165, April, 1985.
Forgy, C. L.OPSS U s e r ’s M a n u a l .
Technical Report CMU-CS-81-135. Carnegie-Mellon U., Dept. of Comp. Sci.. July, 1981.
Fox, J. Alvey, P.. and Myers, C.Decision Technology and Man-Machine Interaction: The PROPS Package.In Expert Systems 83. Cambridge, November. 1983.
Fox, M. S. and Smith, S. F.ISIS-A knowledge-based system for factory scheduling.Expert Systems l( 1):25-49. 1984.
Fox. 1.. Myers. C.D.. Grcavcs. M.F. and Pcgram. S.Knowledge acquisition for expert systems: experience in Icukacmu diagnosis.Methods lrq: ,\led 2-!(2):65-72, [1pril, 1985.
Fox. M.S.Reasoning with incomplete knowledge in a resource-limited eni ironment: integrating reasoning and
knowledge acquisition.In Proc. l./C,W8/. pages 313. lJG\I. 1951.
31
WI
11121
V 131
WIL
[ 1201
Fox, MSConstraint-direcled search: a case study ofjob-shop scheduling.PhD thesis, Carnegie-Mellon University, 1983.
Freiling, M., Alexander, J., Messick, S., RehfUss. S. and Shulman. S.Starting a Knowledge Engineering Project: A Step-by-Step Approach.AI Magazine 6( 3): 150-164. Fall. 1985.
Friedland. P. E.Acquisition of proccdurnl kno\j Icd_r~~ from dom‘lin cxpcrts.In IQ-N. IJO 1-N. p;~gcs GO-M 1. \‘,lncoui cr. ,\ugust, 19s 1.
G‘tdsdcn. J.A.:\n Expert System for E\,aluating Flcctronic W,lr-f‘lrc Tasking Pl;ms for the i<oh;~i Sv,l\ y.In The F’irsl C’otlferetlce otl Arlificiul lt//eiiigetxe ~fpplicntrotls. pages 86-9 1. I f-I%. I Et% Compu tcr
Society Press. Dcccmbcr. 1984.
Gaglio, S.. Minciardi. R. and Puliafto. P.P.1Multiperson decision aspects in the construction of expert systems.IEEE Transactions on System. Man and Cybernetics ShIC-15(4):536-539. July/August. 1985.
Gale, W.A., and Pregibon, D.Using Expert Systems for Developing Statistical Strategy.In Joint S/a!istical Meetings. Toronto, 1983.
Gale, W.A. and Pregibon, D.Artificial intelligence research in statistics technology transfer.AI Magazine 5(4):72, Winter, 1985.
Gammack, J.G. and Young, R.M.Psychological Techniques for Eliciting Knowledge.In M.A. Bramer (editor), Research and Developmenr in E.uperl SJ*srems, pages . Cambridge University
Press, Cambridge, 1984.
Ganascia. J.G.Using an expert system in merging qualitative and quantitative data analysis.Int. J. ,Ilan-Machine Studies 20(3):319-330. March. 1984.
Gaschnig, J.Preliminary Performance Analysis of the Prospector Consultant S!stcm for Mineral Exploration.In Proc. /JC,l/-79, pages 308-310. Tokyo. Japan. 1979.
32
[121] Gaschnig, J., Klahr, P., Pople, H., Shortliffe. E. and Terry, A.Evaluation of Expert Systems: Issues and Case Studies.In F. Hayes-Roth, D.A. Waterman and D.B. Lenat (editor), Building Expert .yyslerrts, pages. Addison-
Wesley, Reading, MA, 1983.
[122] Gelernter, H., Miller, G.A., Larsen, D.L. and Berndt. D.J.Realization of a Large Expert Problem-Solving System -- SYNCHEMZ: A Cast Study.In The First Conference on Arrificial ltz~eliiger~ce Applica/iom. pages 92-106. IEEE. IEEE Computer
Society, December. 1984.
[ 1231 Genesercth, M.R.Automated Comulla/iotl for Cotrrplex Corrlpuler S’Jwettls.PhD thesis, Harvard L’niicrsity. Scptcmbcr, 1978.
[124] Genescrcth, M. K.Diagnosis using hierarchical design models.In Proc. AAM-82, pages 278-283. Pittsburgh, PA. Aug. 1982.
[125] Genesereth. Michael R.An Inrroducbon to MRS for AI Experts.Technical Report HPP-82-27, Stanford University, November, 1982.
[126] Genesereth, M. R.The use of design descriptions in automated diagnosis.Arhjicial Inrelligence 24( l-3):411-436, 1984.
.[127] Georgeff, Michael, and Umberto Bonolio.
Procedural Expert Systems. ’In Proc. /JCA/-83, pages 151-157. IJCAI. Karlsruhe. West Germany, August. 1983.
[128] Gevarter, W.Expert systems: limited but powerful.IEEE Specrrum 20:39-45, 1983.
[ 1291 Goldberg, M., Goodenough, D.G., Alvo, M. and Karam. G.M.L A hierarchical expert system for updating forestry maps with Landsat data.
Proc. IEEE 73(6): 1054- 1063, June. 1985.
[ 1301 Gordon, J. and Shortliffe, E.H.A method for managing evidential reasonin,0 in a hierarchical hypothesis space.,drtificiai Inrelligence 26( 3):323-357, July. 1985.
[ 13 l] Gerry. G.A..Modelling the diagnostic process.J. Med. E&car. 45:293-302. 1970.
33
w21
WI
[134]
w51
[136 1
WI
’ [138]
WI
Gerry, G.A., Silverman, H., Paukcr. S.G.Capturing Clinical Expertise: A Computer Program that Considers Clinical Responses to Digitalis.Am. Jnl. Medicine 64:452-460, 1978.
Gottinger, H.W.HAZARD: An expert system for screening environmental chemicals on carcinogenecity.Experl Syslems l(2): 169- 176, October. 1984.
Goyal. SK.. Prcrau. D.S.. lcmmon. AC’., Gunderson. AS. and Rcinkc. R.E.Compass: An cxpcrt cystcm for tclcphonc switch maintenance.l:‘.tper/ Systettls Z(3): 112-126. Jul). 1985.
Gricsmer. J.H.. S.J. Hong. Ll. Karnaugh, J.K. Kdstner. h1.1. Schor. R.I.. Ennis. 1).!1. Klein. K.R.M illiken. and H.Jt. Van Wocrkom.Y ES/MVS: A Continuous Real ‘lime Expert System.In Proc. /1/f ,ff/-(94. AAAI, (Austin. ‘IX. ,2ugust. 198-l.
Gupta. N.K. and Seviora. R.E.An Expert System Approach to Real .l‘ime System Debugging.In The First Cotrference on Artificial Intelligence Applica/iotls. pages 336-343. IEEE. IEE
Society Press, December. 1984.E Computer
Hahn, G.J.More Intelligent Statistical Software and Statistical Expert Systems: Future Directions.The American Slatisrician 390: l-8, , 1985.
Hakami, B. and Newborn, J.Expert system in heavy industry: an application of ICLX in a British Stell Corporation works.ICL Technical Journal 3(4):347-359, November, 1983.
Hamscher, Walter, and Randall Davis.Diagnosing Circuits With State: An Inherently Underconstrained Problem.In Proc. AAA/-84. AAAI, Austin, TX, August. 1984.
Harmon, P.Overview of Small Expert Systems Building Tools.Expert Sysfems Straregies l( 1): l-10. September. 1985.
Harmon. P. and King D.Expert SJwems: A rtlficial ln teliigetlce in Busrness.John Wiley & Sons, New York, 1985.
Harrar. G.Thomas J. Watson Research Center. For the sake of IB&l dnd science.Cotttputenwrid 19( 14): 102-l 18. April, 1985.
34
[143] Hart, P.E.Progress on a Computer Based Consultant.In IJCAWS. IJCAI, Tbilisi, Georgia, USSR, 1975.
(1441 Hart, P.E., Richard 0. Duda, and M.T. Einaudi.PROSPECTOR -- A Computer-based Consultation System for IMineral Exploration.Mathematical Geology lO( 5). 1978.
r[ 1451 Hasling, D.W.
Abstract Explanations ofstratcgy in a l>iagnostic Consultation System.In 1’1-0~. AAA/-S3. p‘rgcs 157-161. ,\:\,.\I. 1\‘,\shinston, 11.C.. :\llgust. 1083.
[146] Hasling. D.W.. Clanccy. W.J.. Rcnncls. G. I<.Strategic explanations for in dingncjstic consultation ststcm.Irlt.J.ltlanlllachit7e.~tudies” LO( 1):3- 19. January. 1984.special issue on expert s!,stcms.
[ 1471 Hayes- Roth. F.The Knowledge-Based Expert System: A Tutorial.Computer 0: 1 l-28. September. 1984.
[ 148) Hayes-Roth, F., Donald A. Waterman, and Douglas B. Lenat (eds.).Building Expert Systems.
Addison- Wesley, Reading, MA, 1983.
(1491 Hayes-Roth, F.Rule-based systems.
.CACI\! 28(9):921-932, September, 1985.
[ISO] Hayes. J.E. and Michie, D.Intelligent Systems: The Unprecedented Opportunit):Halsted Press. New York. 1983.
[ 1511 Helly. J.J., Jr.A distributed expert system for space shuttle flight control.PhD thesis, UCLA, 1984.
.
[lSZ] Hickam. D.H., Shortliffe, E.H.. Bischoff, M.B.. Scott. [l.c.. Jacobs. C.D.A study of the treatment advice of a computer-based cancer chemotherapV\l protocol ad\*isor,Technical Report KSL 85-21, Stanford University. Comp. Sci. I>cpt.. 1985.acccptcd for publication in the ‘Annals of Internal Medicine’.
[153] Hollan. J.D.. Hutchinson. EL. and Weitzman. L.Steamer: An interactive inspectable simulation-based training sytcm..-L I .Vagazine 5( 2): 15. Summer. 1984.
35
[154] Horn, W;, W. Buchstaller, and R. Trapp.Knowledge Structure Definition for an Expert System in Primary Medical Care.In Proc. IJCA1-81, pages 850-852. IJCAI, UBC, Vancouver, British Columbia. August, 1981.
[155] Horn, K.A., Compton. P., Lazarus, L. and Quinlan. J.R.An expert computer system for the interpretation of thyroid assays in a clinical laboratory.The Australian Compurer Joumal17( 1):7-11, February, 1985.
[156) Horvitz. E.J.. D.E. Hcckcman. R.N. Nathwani and l..M. Fmm.cDiagnostic Stratcgics in the Hi pothcisi-Dircctcd P,\-I’HFIS I)ER Shstcm.In f’roc. lI:I:‘I:‘/,.I ,,I .,I I C’ot!t: 011 .4pylicaliorts of.4 rIific*itil I~~Icili~tw~~. p,~ph h.WL%. 19I3-I.IEEE Computer Society.
[ 157) Hudlicka. Eva. and ITictor Ii. Lesser.Meta-Level Control Through Fault Detection and Diagnosis.In Pruc. ,&&Il-R4. ,\&\I. ,\ustin. ‘f-x’, August, 1984.
[158] Hudson. D.L. and Cohen. M.E.EM ERG E. A Rule- Based Clinical Decision &Making ilid.In The First Conference on Artificial Intelligence Applications, pages 617-623. IEEE. 1EF.F. Computer
Society Press. December, 1984.
[LSS] Ishizuka, M., K-S. Fu, and J.T.P. Yao.Inexact Inference for Rule-Based Damage Assessment of Existing Structures.In Proc. IJCM-81, pages 837-842. IJCAI, UBC, Vancouver, British Columbia, August. 1981.
.[160] Ishizuka, M. ..
Japanese work in expert systems.Experf Systems l( 1):51-56, July, 1984.
[ 1611 Jackson, P. and P. Lefrere.On the application of rule-based techniques to the design of advice-giving systems.Int. J. Man-Machine Studies 20( 1):63-86, January. 1984.
[162] Jackson. P.L[ntroduction to Expert Systems.Addison-Wesley, Reading, MA. 1985.
[163] Johnson. P. E., Duran. ;\. S.. Has&rock. F.. Mollcr. J.. Prictula. M. Feltovich. P. J.. and Swanson.D. 1%Expcrtlse and error in diagnostic reasoning.C’ognlrive Scietlce 5( 3):235-233. 198 1.
36
WI
Ml
WI
WI
WI
w11
VI
i173]
1174
Johnson,_P. E.What kind of expert should a system be?Jnl. Medicine and Philosophy 8177-97.1983.
Johnson, T.The Commercial Application of Expert System Technology.Knowledge Etlgineering Review 1( 1): 15-25, , 1984.
Johnson, W. Lewis. and Elliot Soloway.Intention-Based I)i;lgnosis of Programming Errors.In Proc. /I,,/ ~1 I-,Scl. [\(\A I. I\ustin, TX, August. 198-I.
Johnson, WI . . and Soloway. I+.PROUST: Knowlcdgc-based program understanding.IEEE Trans. Svjiware Etlgineering SE- 1 l( 3). . 1985.
Joshi. Aravind. Bonnie Wcbbcr. and Ralph Wcischcdcl.Living Up to Expectations: Computing Expert Responses.In Proc. AA AI-84. AAAI[, Austin, TX, August, 1984.
Kahn, G. and J. McDermott.The MUD System.In Proc. IEEE/AAAI Cot-$ on Applications of Artificial Intelligence, pages 116- 122. Denver, 1984.IEEE Computer Society.
Kahn, G., Nowlan, S. and McDermott, J.Strategies for knowledge acquisition.IEEE Tram on Pattern Analysis and Machine Intelligence 7(5):511-522, September, 1985.
Kaihara. S., T. Koyama. T. Minamikawa, and T. Yasaka.A Rule-Based Physicians’ Consultation System for Cardiovascular Diseases.In Itlt. Conf: on Cybernetics and Society, pages 85-88. 1978.
Kaplan, S.J.The industrialization of artificial intelligence.A I Magazine 5( 2):5 1, Summer, 1984.
Kassirer, J. P., Kuipers, B. J., and Gorry, GA.Toward a theory of clinical expertise.Am. Jnl. Medicine 73:251-259, 1982.
Kastner, J.. S. Weiss, and C. Kulikowski.,\n Efficient Scheme for ‘rime-Dependent Consultation Systems.In Proceedings ,\/ ED I,VFO 53: Fowth World Conference otl :l/edical ft$miurics. pages 6 19-622.
North-Holland, August 1983.
37
[175] Kastner, J.K., Weiss, S.M., Kulikowski, CA., Dawson. C.R.Therapy Selection in an Expert Medical Consultation System for Ocular Herpes Simplex.Computers in Biology and Medicine, 1983.
[176] Kastner, J.K.Slrategies for Expert Systetn Comultation in Therapy Selection.PhD thesis, Rutgers University. 1983.
[177] Keen. M.J.R.
[17S] Kelly. V. and Stcinbcrg I..‘I’hc CRI’ITER syctcm -- nnal! ;ling digital circuits by propagating bch;lk iors ,Ind cpccificxions.In l’roc. Acirll-82. pages 283-289. ;\lItZI. August. 1952.
[179] Kelly. V.The CRITTER .J:r~stern--auromared critiquing of digiral circuit designs.PhD thesis. Rutgers University, 1984.Also, ‘Proc. 21st Design Automation Conference IEEE’, June 1984.
[180] Kent. D.L.. Shortliffe, E.H, Carlson, R.W., Bischoff. M.B. and Jacobs, C.D.Improvements in data collection through physician use of a computer-based chemotherapy treatment
consultant.J. of Clinical Oncology 3: 1409- 1417, 1985.also Stanford Comp. Sci. Report KSL 85-22..
[181] Kidd. A.L. and Cooper, M.B.Man-Machine Interface for an Expert System.In Expert Sysfenzs 83. Cambridge. London, November. 1983.British Computer Society.-
[182] Kidd. AL. and M.B. Cooper.Man-machine interface issues in the construction and use of an expert system.Int. J. Man-Machine Studies 22( 1):246-259, January, 1985.
(183 Kim, Jin H.. and Judea Pearl.A Computational ‘Model for Causal and Diagnostic Reasoning in Inference Systems.In Alan Bundy (editor). f’roc. IJCA/-(?3. pages 190-193. IJCI\l. Kdrlsruhc. West Germany, [iugust,
1983.
[18-r) Kim. J.ILt.Explorting Domain Knowledge in IC Ceil Layout.
IEEE Desigrl urld Test of C’wpurers l( 3):52-63. [August.. 198-I.
[is9 1
WI
WI
[1941
[ 1951
38
King, J.J..An Investigation of Expert Systems Technology for Automated Troubleshooting of Sciermfic
Instrumentation.Technical Report CSL-82-012, Hewlett Packard Co. Comp Sci Lab, 1982.
Kingma. Wheri.KITE: Knowledge and inference in troubleshooting electronics.Master’s thesis. Univ. of Washington. 1985.
Knight. 13.. Endcr\;b>. K. ;lnd C’ollcr. V.R.‘I’hc use of cxpcr[ s!, s[cms in industrial control.Aleasuremerzts wd (‘orltrol 17; 11):409-413. December-January, 1984.
Kolodncr. J.1..Towards an understanding of the role of experience in the evolurion from noi ice to expert.lnt. J. .Ilarl-,llclchille Studies 19:497-5 18. 1983.
Kowalski, T.J. and Thomas, DE‘I’hc VLSI design automation assistant: first steps.In Proc. Twenty-sixth IEEL’ Computer Society lnt. Conj?rence, pages 126- 130. IEEE. 1983.
Kowalski, T.J.The FLSI design automation assistant: a knowledge-based expert system.PhD thesis, Carnegie-Mellon Univ., September, 1984.
Kubel, E.J.Expert systems tackle forging problems.Materials Engineering 139-42, ,March, 1985.
Kuipers. B.J. and J.P. Kassirer.How to Discover a Knowledge Representation for Casual Reasoning by Studying an Expert Physician.In Proc. INCA/-83. Karl&he. Germany, August 1983.
Kukich, K.Knowledge-based report generation: a knowledge engineering approach to natural language report
generation.PhD thesis. U. of Pittsburgh, 1983.
Kulikowski. C.A.Artificial Intelligence Methods and Systems for Medical Consultation.IEEE Transuctions on Pattern rlnai>‘sis and ,Ilachine Intelligence 2(5):&V476. Scptcmbcr. 1980.
Kulikowski. C. ~1.. and Weiss. Shalom Fvl.Rcprcsentarion of expert knowledge for consultation: The CGN ET clnd EXPERT projects.In Peter Szolovits (editor). .4rtificial Intelligence in ,Ifedicine. pages 21-55. Westview Press, lhulder,
Cola.. 1982.
WI
WI
1‘19s I
WA
WI
WI
WI
PO31
WI
[205]
PO61
39
Kunz. J. C., Fallat, R. J., McClung, D. H., Osborn. J. J., Votteri. B. A., Nii, H. P.. Aikins. J.S.. Fagan.L.M., and Feigenbaum, E.A.A Physiological Rule-Based System for Interpreting Pulmonary Function Test Results.In hoc. Computers in %riticai Care and Pulmonary Medicine, pages 375-379. IEEE Press, 1979.
Kunz, J. C.he of A I, simple r,lathematics, urtd a physiological model for ruakirlg medical diagnoses and trealruenl
plans.Phi> chcsis. Stanford Heuristic Programming Project, Stanford Lnilcrsitj. 1W.
Kuw.. J.C.. Kchlu. ‘I’.P. .lnd \Villi;lms. h1.I).Applications dcvclopmcnt using cl hybrid AI development system.A/ .Iftlgazille 5( 3):4 1. Fall. 1984.
Lambird. B.A.. D. I.avine. and L.N. Kanal.Distributed architecture and parallel non-directional starch for kncju icdgc-b;lscd cnrtogr;iphic t’caturc
extraction systems.lrlt. J. Milan-Jlachirle Studies 20( 1): 107- li0. January, 1984.
Langlotz, Curtis P. and Shortliffe, Edward H.Adapting a Consultation System to Critique User Plans.lnt. J. Man-Machine Studies 19(5):479-496. November, 1983.
Larkin, J.H.. McDermott. J., Simon, D.P., and Simon, HA.Expert and Novice Performance in Solving Physics Problems.Science 208(20): 1335-1342, June. 1980.
Lavrac, N., Bratko, I., Mozetic. I., Cercek, B., Grad. A. and Horvat. :M.KARDIO-E-An expert system for electrocardiographic diagnosis of cardiac arrhythmias.Expert Systems 2( 1):46-50. January , 1985.
Lee, R.M.Expert vs. management support systems: semantic issues.Cybernetics and Systems 14(2-4): 139-157, April-December, 1983.
Lenat, Douglas B., Alan Boming. David McDonald, Craig -Taylor. and Steven Meyer.Knoesphere: Building Expert Systems With Encyclopedic Knowledge.In Proc. IJCAk83, pages 167469. IJCAI, Karlsruhc. West Germany. f1ugust. 1983.
Lcnat, Douglas B., Albert Clarkson, and Garo Kiremidjian.An Expert System for Indications and Warning Analysis.[n /‘rot. [Jc,,1/-1(3, pages 259-262. IJCXI. Karlsruhc, Wtst Germany, i\ugust. 1983.
Lindberg, D.. G. Sharp, L. Kingland, S. Weiss. S. Hayes. I-f. Ueno and S. Har.elwood.Computer-Based Rheumatology Consultant.In Pm. Third IV&.j C’orlfhrerlce on ,\ledicai lrlfbnrratlcs. pages 13 1 I- 13 15. North-Holland, 1980.
WI
PO81
[ 2091
[X0]
WI
WI
.
WI
(2 141
a
WI
,
[X6]
40
Lindsay, R. K., Buchanan, B.G., Feigenbaum, E. A., and Lederberg, J.Applications of Artificial Intelligence for Organic Chemistry: The DENDR AL Project.McGraw-Hill, New York, 1980.
Lindberg, D.A.B.. Gaston. L.W., Kingsland, L.C. and Vanker, A.D.AI/COAG. A Knowledge-Based System for Consultation About Human Hemostasis Disorders:
Progress Report.In Ffth ,4rlr1ual S~rywsiur~l WI Computer Applications in Medical Care. pages 253-257. 198 1.
IL1 ;ihcr. M . I . .HI- I< 1.5’1:‘: ..I kw )\ Ic~tigc- btmd e.vpert JJ*sterrl jbr the prelimimr)* strwtura[ tiusig,l oj’high rise buildirrgs.PhD thesis. Carnegie-Mellon University. Dept. of Civil Engineering, 1984.
hlamdani, E.H. and Ffstathiau. J.An Analysis of Formal Logics as Inference Mechanisms in Exert Systems.lrlt. J. ,llurl-JILtchine Studies 21(3):213-227. Scpt, 1984.
Matsumoto, K., Sakaguchi. T. and Wake. T.Fault diagnosis of a power system based on a description of the structure and fbu.ion of the relay
system.Expert Systems 2(3): 134-138, July, 1985.
McCarthy. J.Some expert systems need common sense.Annals of the New York Academy of Science, 1983.Invited presentation for the New York Academy of Sciences Science Week Symposium on Computer
Culture, April 5-8, 1983.
McDermott, J.Rl: The Formative Years.The AI Magazine 2(2):21-29, 1981.
McDermott, J.Rl: A rule-based configurer of computer systems.Artificial Intelligence 19( 1): 39-88. Scptembcr. 1982.Also CMU Tech Report CMU-CS-80- 119. April 1980.
McDermott, 1.XSEL: a computer sales person’s assistant.In I.E. Hayes, D. Michie and Y-H Pao (editors). Mzchine lnteiiisence /O. pages 325-337. Ellis
Hoi-wood Ltd., Chichcster, England. 1982.
McDermott. J.Extracting Knowledge From Expert Systems.In Proc. l&11-83. pages 100-107. IJCAI. Karlsruhe. West Germany, f\ugust. 1983.
41
WI
Wl
[‘19]
[I’O] IL1 ichaclscn. R.H.An expert system for federal tax planning.l::uperr Sjwtns 1( 2): 149- 167. October. 1984.
[221
[222
.
[223 1
[224l
L
[225]
[226]
McDermott, J.Rl revisited: Four years in the trenches.A I Magazine 5( 3):21-24, Fall, 1984.
McKeown, D.J. Jr., Harvey, W.A. Jr. and McDermott, J.Rule-based interpretation of aerial imagery.IL’EL Trans. on Pattern Attal~~sis and Machine inrelligence 7(5):570-555. Scptcmbcr, 1985.
,ClcShcrry. I>. and Fullerton. K.Prcccptor: [\ shell for medical cxpcrt systems and its application in ;1 stud! of prognostic indicts in
stroke.Kspert .J’?nslems 2(2): 140- 147. July. 1985.
Michalski. R.S.. and Chilausky. R.L.Knowledge acquisition by encoding expert rules versus computer by induction from examples: a case
study involving soybean pathology.
,
Inr. J. of Man-Machine Studies 12( 1):63-87, 1980.
Michalski. R.S., and Richard L. Chilausky.Learning by being told and learning from examples: An experimental comparison of the two methods
of knowledge acquisition in the context of developing an expert system for soybean diseasediagnosis.
International J. of Policy Analysis and Info. Systems 4( 2). 1980.
Michalski, R.S., and A.B. Baskin.Integrating Multiple Knowledge Representations and Learning Capabilities in an Expert System.In Alan Bundy (editor), ptoc. IJCA/-83, pages 256-258. IJCfN K;lrlsruhe. West Germany, August,
1983.
Michie, D. (ed.).Expert Systems in rhe Micro- Electronic Age.Edinburgh University Press. Edinburgh, Scotland. 1979.
M ichie. D.Expert Systems.Cotnputer Journal 23( ): 369-376. . 1980.
51 ichlc. D. (cd.).lnlroduclor?/ Readings in Expert Systems.Gordon & Breach. ,Uew York. 1982.
42
(2271 Michie, D,, Muggleton, S., Riese, C. and Zubrick, S.RULEMASTER: A Second-Generation Knowledge-Engineering Facility.In The First Conference on Artificial Intelligence Applicarions, pages 591-597. IEEE. IEEE Computer
Society Press, December, 1984.
[228] Michic. D.Towards a knowlcdgc accelerator.In f’t~c. of Impucr-84. SPL-Insight, Abingdon. UK. 1984.
(2291 \li!!cr. F.l>., Copp. 11.H.. Vcsondct-. G..!‘. iind Zielinksi. J.E.THE ACE EXPERIMEN’I’: Initial Evaluation ofan Expert System for Preventive Maintenance.In Artificial lnrelligence in ~\lainrenance: Proc. Join/ Sen?ces Workshop. pages 421-427. [\ir Force
Systems Command. . June. 1984.Publication # AFH RL-TR-84-25.
[230] Miller. P.B.An Expert System Which Critiques Patient Workup: Modclling Conflicting Fxpcrtisc.Cotnpulers and Biomedical Research 171554-569. 1984.
[231] Miller, R.A., Pople, H.E., Meyers, J.D.INTERNIST-l, an Experimental Computer-Based Diagnostic Consultant for General Internal
Medicine.NEJM 307(8):468-476,1982.Reprinted in ‘Readings in Medical Artificial Intelligence: The First Decade’: edited by W.J. Clancey
and E.H. Shortliffe; published in Reading, MA by Addison-Wesley: pages 190-209..
[232] Milne, R.A framework for electronic diagnosis expert systems.In Prvc. 1984 Conference on Inlelligenl Syslems and Machines. 1984.
[233] Mitchell, T.. Steinberg, L., and Shulman, J.VEXED: A Knowledge- Based VLSI Design Consuiranr.Rutgers AI/VLSI Project Working Paper 17. Rutgers University, 1984.
[234] Mitchell, T.M., Steinberg, L.I. and Shulman, J.S.L
A knowledge-based approach to design.IEEE Transactions on Parrern Analysis and Machine lnrelligence 7(5):502-5 10. Scptcmbcr, 1985.
[235] 1fittal. S.. Bobrow. D.G. and de Klcer. J.DARN: X Community Memory for a Diagnosis and Rcpalr I’dsk.:I / ,tlagazrne 6( 2):32-36. Summer. 1985.
43
[236] Mittal. S.,-Dym, C, and Morjaria, M.PRIDE: An expert system for the design of paper handling systems.In C.L. Dym (editor), Proc. Winier Annual Meeting of ASME. ASME, ASME Press, Florida,
November, 1985.
(2371 Moore, R.L., Hawkinson, L.B., Knickerbocker, C.G. and Churchman, I-.M.A Real-Time Expert System for Process Control.In The Firs/ Conferetlce on A rrificial intelligence Applicariotls. pages 569-576. I EFE. IEEE Computer
Society Press. Dcccmbcr, 1984.
[238] Moore, R.L.Adding real-tirnc cxpcrt qstcm capabilities to large distributed control cbstcms.Cou/t-01 l:‘tlgirleertrty 32(-I): 118-121. (\pri!. 1985.
[239] ~MOSCS. J.Symbolic integration: The stormy decade.Cotnmunicalions JC,\/ 8:548-560, 1971.
(2401 Mulsant, B. and Servan-Schreiber. D.Knowledge engineering: A daily activity on a hospital ward.Compulers and Biomedical Research 17:71-91,1984.
[241] Murphy, T.Artificial Intelligence topics at IBM.Sitnulation 44( 1):33-37, January , 1985.
.(2421 Myers, C.D., Fox, J., Pegram, S.M., and Greaves. M.G.
Knowledge Acquistion for Expert Systems: Experience Using EMYCIN for Leukaemia Diagnosis.In British Compuler Society, Exper! Systems 53. Churchill CO!!C~C. Cambridge, 1983.
[243] Narain, S.Mycin: implementing the expert system in Loglisp.IEEE Sofrware 2(3):83-88, May, 1985.
[2#] Nau, D.Expert Computer Systems.Compuler 16:63-85.1983.
12451 Neches, R., Swartou& W.. and Moore, J.Enhanced -Maintenance and Explanation of Expert Systems through Explicit lClodcls ofTheir
Development.In Proc. /EE,E Workshop otl Principles of Knowledge- Based $‘sIetns. December. 1984.
44
WI
WI
[248]
w4
WI
125 11
WI
WI
WI
[ZSS]
Neches, R., Swartout. W. R., and Moore, J.Explainable (and maintainable) expert systems.In Proc. IJCA1-8.5. IJCAI, 1985.
Nii, H. P., E. A. Feigenbaum, J. J. Anton, and A. J. Rockmore.Signal-to-Symbol Transformation: HASP/SIAP Case Study.A/ Magazine 3( 2):23-35, spring, 1982.
Norkin. D.D.Expert system fr;lmcwrk for geotechnical site characterization.M;wYs thesis. Carncgic-Mellon University. 1985.Dept. of Civil Engineering.
Novak. G. S., Jr.GLISP: A Lisp-based Programming System with Data Abstraction.A. /. Magazine 4( 3). August. 1983.
Nowak, E.J. and Szablowski. B.F.Expert systems in scientific information exchange.J. of lrlforrna~ion Science 8( 3): 103- 111, April, 1984.
O’Connor. Dennis E.Using Expert Systems to Manage Change and Compexity in IManufacturing.In W. Reitman (editor), AI Applicationsfor Business, . Ablex Publishing Corp., 1984.
Ogawa, H., Fu, K.S. and Yao, J.T.P.An Expert System for Damage Assessment of Existing Structures.In The First Conference on Artificial Inlelligence Applicarions. pages 33 l-335. IEEE, IEEE Computer
Society Press, December, 1984.
O’Keefe, R.M.Expert systems and operational research.Journal Optical Research Sociely 36( 2): 125- 129, February, 1985.
Orciuch, Ed and Frost, John.ISA: Intelligent Scheduling Assistant.In Proc. IEEE/AAAI Conf: on Appiicarions of Arrificial lnrelligence. pages 314-320. Denver, 1984.IEEE Computer Society.
Owen. K.Progress in IKBS.Alvey Xews ( 11): lo- 11. June, 1985.
WI
12571
(258)
[259l
[16OJ
126 11
.
WI
-[263]
L
j264
[265 1
45
Patil, KS-., Szolovits, P.. and Schwartz, W. B.Causal Understanding of Patient Illness in Medical Diagnosis.In Ptoc. IJCA1-81, pages 893-899. IJCAI, Vancnuver. British Columbia, Canada, August 1981.
Patil, R.S.Causal Represenlatiotr of Patient Illness for Elecrrolyre and Acid- Base Diagnosis.PhD thesis, MIT. Cambridge. MA, 1981.
Pattcn, C.Knowlcdgc cnginccring: tapping the experts.l:‘lcclrot~ic Design 33( 10):93-98. 100. bla>. 1985.
Paul. 1 ..I--.Failure dctcction proccsscs bk an expert system and 1~) brid pattern recognition.In Proc. 6rh I:‘uroyem (‘~4: on Artificial lnrelligence. Lisa. lr;lly. Sept.. 1984.
Pipitonc. F.An Expert System for Electronics Troubleshooting Bused on Function and Connectivity.In The Firsr C’onjkrence on A rlificial Inleiligence Applicurions. pages 133- 138. IEEE, IEEE Computer
Society Press. December. 1984.
Pisano, A.D. and Jones. H.L.An Expert Systems Approach to Adaptive Tactical Navigation.In The Firsl Conference on Arrificial lnfeiligence Applications, pages 460-464. IEEE, IEEE Computer
Society Press. December, 1984.
Pittman. J.A.Predicting the usefulness of expert consultant systems wilh’informarlon-rheorerlc measures.PhD thesis, Virginia Polytechnic Institute and State University. 1984.
Politakis, P G.Using Empirical Analysis to Refine Expert System Knowledge Bases.PhD thesis, Computer Science Research Laboratory. Rutgers University. October, 1982.Rep.No. CBM-TR-130.
Politakis. P. and Hickson. W.Dcvclopmcntal facilities in an expert system for network troubleshooting.In Proc. European Conf: on Arlificial Inrelligence. Italy. September. 1984.
Politakis. P. and Weiss, S. M.Using empirical analysis to refine cxpcrt system knowlcdgc bases.,Irr$cia/ Intelligence 22( 1):23-38, 198-t.
WI
WI
. (268 I
P691
WI
WI
12721
WI
L
WI
[2751
46
Pollack, M.. Hirschberg, .J., Webber, B.User Partrcipation in the Reasoning Processes of Expert Systems.Technical Report Technical Report CIS-82- 10, University of Pennsylvania. 1982.A short version appears in the Proc. AAAI-82.
Pople, H.E., Ir., Myers, J. D., and Miller, R. A.DIALOG (INTERNIST): A Model of Diagnostic Logic for Internal Medicine.In Proc. K’n1-75, pages 849-855. Tbilisi, Georgia. USSR, Septcrnber, 1975.
Pople, H.E.The Formation of Composite Hypotheses in Diagnostic Problem-sol1 ing: ;\n Fxercisc in Sy nthctic
Reasoning.In Proc. IJCil I-77. pages 1030-1037. IJCAI, Cambridge, Mass. August. 1977.
Pople, H-E., J.D. iMyers and R.A. Miller.DIALOG: a model of diagnostic logic for internal mcdicinc.In Proc. IJCA I-77. [JCAI, Computer Science Dept.. CMU. 1977.
Pople, H.E.. Jr.Heuristic methods for imposing structure on ill-structured problems: the structuring of medical
diagnostics.In P. Szolovits (editor), Arfificial Intelligence In Medicine. pages 119-190. Westview Press, Boulder,
Cola., 1982.AAAS Symposium Series.
Prade, H.A computational approach to approximate and plausible reasoning with applications to expert systems.IEEE Transactions on Pattern Analysis and Machine Inrelligence. PAMl-7 7(3):260-283, May, 1985.
Prerau, D.S. *
Selection of an appropriate domain. ’A/ Msg. 6( 2):26-31. Summer, 1985.
Quinlan, J.R.Practical Applications of Expert Systems.In Proc. of the f&h Australian Computer Conference. Melbourne, Australia, 1983.
Quinlan, J.R.Internal consistency in plausible reasoning systems.New Generation Computing (Japan) 3( 2): 157- 180. 1985.
Radford. c1.D.. Hung, P. and Gcro, J.S.New rules of thumb from computer-aided structural design -- acquiring knowledge for expert systems.In Proc. C,-IfI-X4, pages 558-566. Buttcrworths. Guildford. Surrey. 1984.
WI Rauch, H;E.Probability concepts for an expert system. Used for data fusion.A I Magazine 5(3):55, Fall, 1984.
WI Rauch-Hindin, W.B.ArGjkial Inrelligence in Business. Science and Industry.Prentice Hall, New Jersey, 1985.
WI Reboh, R.Knowledge engineering lechniyues and tools in /he Prospector envlrotltnent.Technical Report 243. SRI International. .Lfcnlo Park, CA. June. 198 1.
P791 Reboh, Rene.Extracting Useful Advice From Conflicting Expertise.In Proc. IJCAl-83, pages 145-150. IJCiII, Karlsnrhe, West Germany. t\ugust. 1983.
P801 Reggia, J.A.Diagnostic Expert Systems Based on a Set Covering Model.IN. J. Man-Machine Studies 19:437-460. 1983.
128 11 Reggia, J.A., Dana S. Nau, and Pearl Y. Wang.A New Inference Method for Frame-Based Expert Systems.In Proc. AAAI-83. AAAI, Washington, D.C., August. 1983.
(2821
.
WI
Reggia, J.A., B.T. Perricone, D.S. Nau and Y. Peng.Answer justification in diagnostic expert systems -- Part I: abductive infcrcnce and its justification.IEEE Transactions on Biomedical Engineering, BME-32 32(4):263-267, April. 1985.
Reggia, J.E., B.T. Perricone, D.S. Nau, and Y. Peng.Answer justification in diagnostic expert systems -- Part II: supporting plausible justifications.IEEE Transactions on Biomedical Engineering, BME-32 32(4):268-272. April. 1985.
WI Reinstein. H.C., and J.S. Aikins.Application Design: Issues in Expert System Architecture.In Proc. IJCAI-81, pages 888-892. IJCAI. UBC, Vancouver. British Columbia. August. 1981.
[2i51 Rich, C., Shrobe, H.E. and Waters. R.C.Overview of the programmer’s apprentice.In Proc. IJCAI-79. pages 827-828. Tokyo, Japan. 1979.
LW Richer. iMark H.
47
Evaluating the Existing Tools for Developing Knowledge-Bused SWetns.KSL85- 19, Knowledge Systems Laboratory, Stanford University. May, 1985.
WI
W81
[789 I
WI
P 11
WI
LW
c
[296
.
[197
48
Rivlin, J.&l., M.B. Hsu, and P.V. Marcal.Knowledge Based Consullalion for Fit&e Elemenr SIruclural Analysis.Technical Report, MARC Analysis Research Corp., Palo Alto, CA, 1980.
Roach, J., Lee, S., Wilcke, J. and Ehrich, M.An Expert System That Criticizes Decisions in Combination Drug Therapy.In The First Conference on Arrificial lnrelligence .+lpplicarions, pages 344-349. IEEE, IEEE Computer
Society Press, December, 1984.
Roach. J.W.. Virkar. R.S.. Weaver, M.J. and Drake. C.R.POIMME: A computer-based consultation system for apple orchard management using Prolog.Experr Systetns 2(2):56-69, April. 1985.
Rosenberg, Steven.HPRL: A Language for Building Expert Systems.In Proc. IJCAI-83, pages 215-217. lJC[\ 1, Karlsruhe, West Germany, August. 1983.
Rub&s, C.E.A rule-based poison control expert system.SIGBIO Newsletler 7( 1):33-36, March, 1985.
Rychener, M.D.Expert systems for engineering design.Experl Sysrems 2( 1):30-44, January, 1985.
Sagalowicz, D.Development of an expert system.Experr Sysrems 1(2):137-147, October, 1984.
Sage, A.P. and C.C. White.Ariadne: a knowledge-based interactive system for planning and decision support.IEEE Transactions on Sysrems. Man. and Cybernerics WC-14 14(1):35-47. January/February, 1984.
Sammut, C.A.Concept development for expert system knowledge bases.Ausrralian Compuring Journal 17( 1):49-55, February. 1985.
Sauers. Ron, and Rick Walsh.On the Requirements of Future Expert Systems.In Proc. IJCA I-83. pages 110-115. IJCAI. Karlsruhc West Germany. :\ugust. 1983.
Savory. S.Faultfinder and Conad: 2 working expert systems at Nixdorf Computer AG.In Proc. GDl Chf: otl Artificial in/elligence. North Holland. Rueschlikon. 1984.to be published.
49
P981
WI
WOI
WI
WI
13031
13041
[3051
13061
[3071
[308 I
Savory, SE.Artificial Intelligence - State of the Art 1984.In S.E. Savory (editor), Kunstliche Inrelligenz und Experrensysleme. chapter 1. pages 13-34.
Oldenbourg Verlag, Munich, 1985.
Schank, Roger C.The Current State of Al: One Man’s Opinion.AI Magazine 4(4):3, 1983.
Schindlcr. M.Artificial Intelligence Begins to Pa> Off u irh Expert S> stems for l-kginccring.Elecrronic Design 0: 106- 146, [August. 1984.
Schindlcr. M.Expert systems -- 1985 technology forecast.Electronic Design 33( 1): 112 ff.. January, 1985.
Sedimeyer, Robert L.. William B. Thompson, and Paul E. Johnson.Diagnostic Reasoning in Software Fault Localization.In Proc. IJCAl-83. pages 29-31. IJCAI, Karlsruhe, West Germany, August. 1983.
Sheehan, G.M.An application to payroll.In R.N. Anthony (editor), Proc. of rhe Aufomaric Da/a Processing Conference. Boston: Harvard
Graduate School of Business Administration,, Sept., 1956.
Shibahara, Tetsutaro, John K. Tsotsos, John Mylopoulos, and H. Dominic Covvey.CAA: A Knowledge Based System Using Causal Knowledge to Diagnose Cardiac Rhythm Disorders.In Proc. IJCAI-83, pages 242-245. IJCAI, Karlsruhe, West Germany. August. 1983.
Shortliffe, E. H., and Davij, R.Some Considerations for the Implementation of Knowledge-Based Expert Systems.SIGART Newslelter 5519-12. December, 1975.HPP-75-16.
Shortliffe, E. H., Scott A. C., Bischoff, M. B., Campbell. A. B.. Van Mtlle. W.. and Jacobs. C. D.ONCOCIN: An Expert System for Oncology Protocol Managcmcnt.In Proc. IJCAI-81, pages 876-88 1. IJCAI, Vancouver, B C.. iiugust- 1981.
Shrager, J., Finin. T.An Expert System That Volunteers Advice.In Proc. AA AI-82. XAAI, 1982.
Sizcmore, R.C.Al improving soup cooking at Campbells.Aulotnalion News :35. August. 1985.
13091
13 101
P111
[3 121
WI
13 141
WI
P 161
13 171
13 181
50
Slagle, J.R. and Michael Gaynor.Expert System Consultation Control Strategy.In Proc. AAAI-83. AAAI, Washington, D.C., August 1983.
Slagle, J.R. and Hamburger, H.An expert system for a resource allocation problem.C&Z1 28(9):994- 1004. Septcmbcr. 1985.
Smith, Reid G.. and James D. Baker.‘l’hc Dipmctcr Advisor System - A Cast Study in Commercial Expert Slstcm Dcvclopmcnt.In Proc. IJCA/-83. pages 122-129. lJC,\l. Karlsruhc, West Gcrrnany, August. 1983.
Smith, Reid G.On the dekclopmcnt of commercial expert systems.AI Magazine 5(3):61-73, Fall, 1984.
Smith, S.B. and Lane. R.S.An intelligent operator aid for dynamic route planning.Expert Systems l(2): 143-148. October. 1984.
Sriram, D.DESTINY: A knowledge based approach lo inregraled smctural design.PhD thesis, Carnegie-Mellon University, Dept. of Civil Engineering, 1985.
Sriram, D.Expert Systems in Engineering.SIGART Newsletrer92, April, 1985.(Survey, with notes from many contributors).
Stansfield. S.A.ANGY: A Rule-Based Expert System for Identifying and Isolating Coronary Vessels in Digitial
Angiograms.In The Firsl Conference on Arlificial Inrelligence .4pplicarions, pages 606-609. IEEE. IEEE Computer
Society Press, December, 1984.
Stefik, M.. Aikins, J., Balzer, R., Benoit, J., Birnbaum, L., Hayes-Roth. F.. Sacerdori. E.The Organization of Expert Systems, A Tutorial.Arlificial Inrelligence 18( 2): 135-173, March. 1982.
Steinberg, L. and Mitchell. T.r\ knowledge based approach to VI-S1 CAD: The redesign system.In Proc. 2Isr Design Au/otnalion Conference, IEEE. IEEE, 1984.Also Rutgers Al/VLSI Project Working Paper No. 11.
5 1
(3191 Steinberg, L.1 and Mitchell, T.M.The Redesign system: A knowledge-based approach.IEEE Design and Test. Compuring 2( 1):45-54, February, 1985.
[320] Subrahmanian, E.An analysis of a welfare eligibility determination interview: A planning approach.In Proc. AAAI-83, pages 398-401. AAAl, Washington, DC., 1983.
[321] Swanson, D. B.. Feltovich, P. J.. and Johnwn, P. F.Psychological analysis of physician cxpcrtisc: Implications for design ofdccisiun support s+crns.In MEDIIVI~~O~~. pages 161-164. Sur-th-HulLmd. r\mstcrdclm. 1977.
(322) Swartout W. R.Explaining and justifying in cxpcrt consulting programs.In Proc. IJCAl-Xl. Vatlcouver. /Kc’.. pages 815-822. IJCIZI, August. 1981.Also in Clancey and Shortliffc (cds.). ‘Readings in Medical Artificial Intclligcncc: the Firct 1 >ccadc’.
Addison- Wesley. 1984.
[323] Swartout, W.R.Producing Explanations and Jusrificariotls of Experr Cotlsulritlg Programs.PhD thesis, Laboratory for Computer Science. Massachusetts Institute of Technology, January, 1981.
[324] Swartout. W.R.Explainable expert systems.In Proc. IEEE Conference, MEDCOMP ‘83. IEEE. 1983.
. [325] Swartout, W.R.XPLAIN: A system for creating and explaining expert consulting programs.Arlificial Inlelligence 21(3):285-325, September. 1983.
[326] Sweet, L.Research in Progress at General Electric.A I Magazine 6( 3):220-227, Fall, 1985.
[327] Szolovits, P.L Toward more perspicuous expert system organization.
In W. Swartout (editor), SIGART Newslelrer. ACM. 1983.in Report on Workshop on Automated Explanation Production.
[328] SLolovits, P. and S.G. Pauker.Research on a Medical Consultation System for Taking the Present lllncss.In Proc. 3rd lllinols Conference on Medical ltlforma~iotl -5) slems. University of Illinois at Chicago
Circle. 1976.
13293
13301
.
[331 1
WI
WI
WI
WI
.
WI
52
Szolovits, P. and Long, W.J.The Development of Clinical Expertise in the Computer.In P. Szolovits (editor), ArtiJiciai Intelligence in Medicine, pages 79-l 18. Westview Press, 1982.
Szolovits, P. and S.G. Pauker.Categorical and Probabilistic Reasoning in Medical Diagnosis.Artificial Intelligence 11: 115 144, 1984.Reprinted in ‘Medical Artificial Intelligcncc: the First Decade’: W.J. Clancey and E.H. Shortliffc
(cds.), 1984.
Taylor. E.Developing a knowlcdgc cnginccring capability.AI Msg. 6(2):58-63. Summer. 1985.
Teach, Randy I,. and Shortliffe, Edward H.An ,\nalysis of Physician Attitudes Regarding Computer-Based Clinical Consultation Sjstcms.C’omput. in Riorned. Research 141542-558. 1981.
Tengvald, E.A.The design of expert planning systems: arl experimental operations plarming system for turtling.PhD thesis, Linkoping University, September, 1984.
Thompson. B.A. and Thompson, W.A.Inside an expert system.Byre :315-330, April, 1985.
Tou, J.T., Huang, C.L. and Li, W.H.Design of a Knowledge-Based System for Understanding Electronic Circuit Diagrams.In The First Conference on Artificial Intelligence Applications. pages 652-66 1. IEEE, IEEE Computer
Society Press, December, 1984.
Travelers Tribune.Travelers develops first expert system.(Corporate Newspaper), August 1985.
Trigoboff, M.IRIS: A framework for the construction of clinical consultation systems.PhD thesis. Department of Computer Scicncc, Rutgers University. 1978.
Tsotsos. J. K.Computer assessment of left ventricular wall motion: The ALU3 expert sytstem.Comput. and Biomedical Research 18( 3):254-277, June, 1985.
5 3
13391
[3401
[3Jl]
(342
[343
1
1
WI
[345 1.
WI
I3471
[3481
Tsuji, S. and Shortliffe, E. H.Graphical Access to the Knowledge Base of a Medical Consultation System.In Proc. AAMSI Congress ‘83, pages 551-555. AAMSI, May, 1983.
Van Melle, W.J.MYCIN: A Knowledge-Based Consultation Program for Infectious Disease Diagnosis.I~I. J. Man-Machine J’tudies 10:313-322. October, 1979.
Van Mcllc. W.J.A rrijicial lnleiligerrce. .Vo. I I: .Y~3tettl ,,I itis itr Corrsltwlitrg Cotmcltafiuri PtvgrattIs.UhlI kscmh Press. [\nn Arbor, Ml, 1981.Based on PhD dissertation, Stanford University. 1980.
C’an Mcllc. W.. Shortliffc. E. H. and f3uchar-m. l&G.EMYCIN: A Dornairl-independent LJj,sterrl that Aids iti (‘onstnrctlng Knowledge-based Consultation
Programs.Pergamon-lnfotech. New York. 1981..
Vesondcr. Gregg T.. Salvatore 1. Stolfo, John E. Ziclinski. Frcdcrick D. *Miller. and David H. Copp.ACE: An Expert System for Telephone Cable Maintenance.In Proc. IJCA1-83, pages 116-121. IJCAI, Karlsruhe, West Germany, August 1983.
Waller. L.AI brings smarts to PC-board assembly.Electronics : 17- 18, July, 1985.
Waterman, D.A.User-Oriented Systems for Capturing Expertise: A Rule-Based Approach.In Michie. D. (editor), Expert Systems in the Micro-Electronic Age. pages. Edinburgh University
Press, Scotland, 1979
Waterman, D., Anderson, R. H., Hayes-Roth. F.. Klahr. P.. Martins, G. and Rosenschein, S. J.Design of a n&oriented system for implementing expertise.Technical Report N-115%l-ARPA, The Rand Corporation. May, 1979.
Waterman, D.A. and Peterson. M.A.Evaluating civil claims: an expert systems approach.Expert SJ)stems l( 1):65-76. July, 1984.
Waterman. D.A Guide to I:‘xperr $-sterns.,1ddison-Wesley. Reading. MA. 1985.
[349 1
(3501
WI
[352]
WI
WI
WI
PW
WI
L
[358
(359
5 4
Weiss, SM.A System for Model- Based Computer Aided Diagnosis and Therapy.PhD thesis. Computers in Biomedicine, Department of Computer Science, Rutgers University, June,
1974.
Weiss, S.M., Kulikowski, C. and Safir. A.A model-based consultation system for the long-term management of glaucoma.In Proc. IJCN-77, pages 826-832. lJC\I. IClassclchusetts Institute Technology. 1977.
Weiss. S.. C.A. Kulikowski. and [\. Sin%.Glaucoma consultation by computer.Cornput. Biol. Med. 8:25-40. 1978.
Weiss. S.M., K.B. Kern and C.A. Kulikowski.A Guide to the Use of /he EXPLR T Consultation System.Technical Report CBM-TR-94. Dept. of Computer Science, Rutgers IJnivcrsity. Novcmbcr. 1978.
Weiss, S.M.. Kulikowski. C.A., Amarcl. S. and Safir. A.A model-based method for computer-aided medical decision-making.A I Magazhe 11: 145- 172, 1978.
Weiss, S.M., C. Kulikowski, and R. Galen.Developing Microprocessor-Based Expert Models for Instrument Interpretation.In Proc. IJCA I-81, pages 853-855. Vancouver, Canada. 1981.
Weiss, S. M. and Kulikowski, C. A.A Practical Guide to Designing Expert Systems.Rowman and Allanheld, Totowa, NJ, 1984.
White, A.P.Inference Deficiencies in Rule-Based Expert Systems.In M.A. Bramer (editor), Research and Development in Expert Sjstenrs, . Cambridge University Press,
Cambridge, 1984.
Wilkins, D. C., Buchanan, B.G.. Clancey. W.J.Inferring an Expert’s Reasoning by Watching.Proc. 1984 Conference on Iurelligent Systems and L\lachines, 1984.(also, Stanford KSL Report HPP-84-29).
Wilkinson. P.‘Humanized’ system tracks troubles (AT&T automated cable expertise system.Telephone Engineering and ,llanngernent 89(9): 86-91, flpril. 1985.
Williams, Thomas L., Paul J. Orgren. and Carl L. Smith.Diagnosis of Multiple Faults in a Nationwide Communications Network.In Pruc. IJCU-83. pages 179-181. IJCAI. Karlsruhc. West Germany, August. 1983.
55
[360] W illiams,C.Software Tool Packages the Expertise Needed to Build Expert Systems.Electronic Design : 153-167, August, 1984.
[361] Winston, P.H. and Predergast. K.A. (eds.).The A I Business: The Commercial Uses of Artificial Intelligence.MIT Press, Cambridge, MA, 1984.
[362] Yagcr, R.R.Approximate Reasoning as a Basis for Rule- f&cd Expert Systems.lL’1:‘1:’ Trmls. otl S:rwcvtls. .\l(~tl cltd Cj~benretics SLIC- l-4(4):636-643. Jul> /,\upust. 1984.
[363] Yamada, Naoyuki. and Hiroshi Motoda.A Diagnosis Method of Dynamic System Using the Knowledge on System Description.In Proc. IJCAI-83, pages 225-229. IJCAl, Karlsruhe. West Germany. August. 1983.
* [364] Young. R.M.Human Interface Aspects of Expert Systems.In J. Fox (editor), State of the Art Report in Expert Systems, pages. Pergamon lnfotcch. , 1985.
[365] Zarri, GbP.Expert iystems and information retrieval: an experiment in the domain of biographical data
management.Int. J. Man-Machine Studies 20( 1):87- 106. January. 1984.
[366] Zivy, G.M.The role of expert systems in producing log interpretation software.Expert Systems l( 1):57-63, July, 1984.