ontology development to support aqua question-answering richard fikes jessica jenkins bill...
TRANSCRIPT
Ontology DevelopmentOntology DevelopmentTo SupportTo Support
AQUAAQUA Question-Answering Question-Answering
Richard FikesRichard Fikes
Jessica Jenkins Bill MacCartney Rob McCool Deborah McGuinnessJessica Jenkins Bill MacCartney Rob McCool Deborah McGuinness
Knowledge Systems LaboratoryKnowledge Systems LaboratoryStanford UniversityStanford University
www.ksl.stanford.eduwww.ksl.stanford.edu
Knowledge Systems Laboratory, Stanford University
2
KSL and the WMD CoalitionKSL and the WMD Coalition Tools for ontology creation, evolution, and maintenanceTools for ontology creation, evolution, and maintenance
Coalition teams have adopted KSL’s Ontolingua and Chimaera as Coalition teams have adopted KSL’s Ontolingua and Chimaera as a standard for ontology development, maintenance, and analysis a standard for ontology development, maintenance, and analysis
(Additionally, we use other internal tools – JTP, DQL, IW)(Additionally, we use other internal tools – JTP, DQL, IW)
Initial evaluation and ongoing supportInitial evaluation and ongoing support KSL evaluated initial stage of WMD ontology using our Chimaera KSL evaluated initial stage of WMD ontology using our Chimaera
tools, reviewed findings with other teams, and taught others how tools, reviewed findings with other teams, and taught others how to use the tools themselvesto use the tools themselves
Knowledge Systems Laboratory, Stanford University
3
KSL and the WMD CoalitionKSL and the WMD Coalition
Tools for ontology creation, evolution, and maintenanceTools for ontology creation, evolution, and maintenance
Initial evaluation and ongoing supportInitial evaluation and ongoing support
Knowledge representation and consulting workKnowledge representation and consulting work KSL providing some core new KB development for CNS core – KSL providing some core new KB development for CNS core –
Russian naval facilities and Newly Independent States facilities – Russian naval facilities and Newly Independent States facilities – and is augmenting this with extraction resultsand is augmenting this with extraction results
Providing consultation on sources / merging opportunities -Providing consultation on sources / merging opportunities -Counter-terrorism KBs built for DARPA (HPKB, ISX-Cyladian- Counter-terrorism KBs built for DARPA (HPKB, ISX-Cyladian- HORUS, SAIC-Cycorp, … ) semantic web for the military (SWMU HORUS, SAIC-Cycorp, … ) semantic web for the military (SWMU ontology tutorial), ontologies for information fusion, etc.ontology tutorial), ontologies for information fusion, etc.
Knowledge Systems Laboratory, Stanford University
4
KSL and the WMD CoalitionKSL and the WMD Coalition Tools for ontology creation, evolution, and maintenanceTools for ontology creation, evolution, and maintenance
Initial evaluation and ongoing supportInitial evaluation and ongoing support
Knowledge representation and consulting workKnowledge representation and consulting work
Knowledge extractionKnowledge extraction KSL knowledge extraction tools for RNF and NISKSL knowledge extraction tools for RNF and NIS
New focus on utilizing other important useful KB sources New focus on utilizing other important useful KB sources SUMO is a core ontology for ontology sharing: ~3,900 axioms; SUMO is a core ontology for ontology sharing: ~3,900 axioms;
relations and sets; processes and objects; temporal, spatial, and relations and sets; processes and objects; temporal, spatial, and mereological relations; agents, etc.mereological relations; agents, etc.
Domain ontologies: WMDs, terrorism, biological viruses, … Domain ontologies: WMDs, terrorism, biological viruses, … Written in SUO-KIF, a proprietary dialect of KIFWritten in SUO-KIF, a proprietary dialect of KIF Published by Teknowledge under GNU public license as part of Published by Teknowledge under GNU public license as part of
IEEE SUO working group (ontology.teknowledge.com/)IEEE SUO working group (ontology.teknowledge.com/)
Knowledge Systems Laboratory, Stanford University
5
SUMOSUMO
SUMO requires translation to be used with any reasonerSUMO requires translation to be used with any reasoner
KSL has successfully translated SUMO to plain-vanilla KIFKSL has successfully translated SUMO to plain-vanilla KIF Full translation: complete semantic content retainedFull translation: complete semantic content retained Highly portable: should be fully compatible with most FOL reasonersHighly portable: should be fully compatible with most FOL reasoners Accurate: most test queries demonstrably answerable from translated Accurate: most test queries demonstrably answerable from translated
axiomsaxioms
However, the result is not yet fully usableHowever, the result is not yet fully usable Most test queries not answered from full axiom set in reasonable timeMost test queries not answered from full axiom set in reasonable time SUMO was not designed for efficient automated reasoningSUMO was not designed for efficient automated reasoning Solution: a smarter translator, and some SUMO “brain surgery”Solution: a smarter translator, and some SUMO “brain surgery”
Translation to DAML may provide another path…Translation to DAML may provide another path… The existing translation is quite lossy, but enables some query-answeringThe existing translation is quite lossy, but enables some query-answering Further work will enable a more complete and accurate translationFurther work will enable a more complete and accurate translation
Knowledge Systems Laboratory, Stanford University
6
DAML Versions of SUMO, WMD, and TerrorismDAML Versions of SUMO, WMD, and Terrorism
DAML translations of SUMO, WMD, and 5 terrorism-related ontologies DAML translations of SUMO, WMD, and 5 terrorism-related ontologies and knowledge bases (KBs) provided by Teknowledgeand knowledge bases (KBs) provided by Teknowledge
ClassesClasses PropertiesProperties InstancesInstances TriplesTriplesDroppedDropped
AxiomsAxioms
SUMOSUMO 577577 188188 6868 35243524 ~800~800
WMDWMD 186186 99 6969 10211021 5959
TerrorismTerrorism
OntologyOntology8484 00 00 200200 4444
TerrorismTerrorism
KBsKBs22 11 28922892 96689668 29192919
TotalTotal 849849 198198 30293029 1441314413 38223822
Knowledge Systems Laboratory, Stanford University
7
DAML Versions of SUMO, WMD, and TerrorismDAML Versions of SUMO, WMD, and Terrorism
A few simple translations were used by Teknowledge to translate A few simple translations were used by Teknowledge to translate original KIF content to DAMLoriginal KIF content to DAML
Example: subrelationExample: subrelation
(subrelation father parent)(subrelation father parent)<daml:ObjectProperty rdf:ID=“father”><daml:ObjectProperty rdf:ID=“father”> <rdfs:subPropertyOf rdf:resource=“#parent”/><rdfs:subPropertyOf rdf:resource=“#parent”/></daml:ObjectProperty></daml:ObjectProperty>
Example: KIF triples to RDF triplesExample: KIF triples to RDF triples
(part MadridSpain Spain)(part MadridSpain Spain)<rdf:Descriptioin rdf:ID=“MadridSpain”><rdf:Descriptioin rdf:ID=“MadridSpain”> <part rdf:resource=“#Spain”/><part rdf:resource=“#Spain”/></rdf:Description></rdf:Description>
Knowledge Systems Laboratory, Stanford University
8
DAML Versions of SUMO, WMD, and TerrorismDAML Versions of SUMO, WMD, and Terrorism
Teknowledge’s DAML files provide a great starting point, but there are Teknowledge’s DAML files provide a great starting point, but there are a few problemsa few problems Syntactic errors and issues with resolving references across files -- these Syntactic errors and issues with resolving references across files -- these
problems are easy to fix.problems are easy to fix. A large amount of the original KIF content was dropped in the translation A large amount of the original KIF content was dropped in the translation
to DAML. Reincorporating some of this content is trivial, but it is generally to DAML. Reincorporating some of this content is trivial, but it is generally nontrivial.nontrivial.
(instance part TransitiveRelation)(instance part TransitiveRelation)
Example (trival): TransitiveRelationExample (trival): TransitiveRelation
<daml:TransitiveProperty rdf:ID=“part”><daml:TransitiveProperty rdf:ID=“part”> … …</daml:TransitiveProperty></daml:TransitiveProperty>
Knowledge Systems Laboratory, Stanford University
9
KIF -> DAML Example (nontrivial)KIF -> DAML Example (nontrivial)
(=> (and (instance ?SUBSTANCE BiochemicalAgent)(=> (and (instance ?SUBSTANCE BiochemicalAgent)(possesses ?AGENT ?SUBSTANCE))(possesses ?AGENT ?SUBSTANCE))
(capability BiochemicalAttack agent ?AGENT))(capability BiochemicalAttack agent ?AGENT))
Original KIFOriginal KIF
Translation of Translation of capabilitycapability [ternary to binary relation][ternary to binary relation]<rdfs:Class rdf:ID=“Capability”/><rdfs:Class rdf:ID=“Capability”/><rdf:Property rdf:ID=“capabilityRole”><rdf:Property rdf:ID=“capabilityRole”> <rdfs:domain rdf:resource=“#Capability”/><rdfs:domain rdf:resource=“#Capability”/> <rdfs:range rdf:resource=“#CaseRole”/><rdfs:range rdf:resource=“#CaseRole”/></rdf:Property></rdf:Property><rdf:Property rdf:ID=“capabilityProcess”><rdf:Property rdf:ID=“capabilityProcess”> <rdfs:domain rdf:resource=“#Capability”/><rdfs:domain rdf:resource=“#Capability”/> <rdfs:range><rdfs:range> <daml:Restriction><daml:Restriction> <daml:onProperty rdf:resource=“&rdfs;#subClassOf”/><daml:onProperty rdf:resource=“&rdfs;#subClassOf”/> <daml:hasValue rdf:resource=“#Process”/><daml:hasValue rdf:resource=“#Process”/> </daml:Restriction></daml:Restriction> </rdfs:range></rdfs:range></rdf:Property></rdf:Property><rdf:Property rdf:ID=“capability”><rdf:Property rdf:ID=“capability”> <rdfs:range rdf:resource=“#Capability”/><rdfs:range rdf:resource=“#Capability”/></rdf:Property></rdf:Property>
Knowledge Systems Laboratory, Stanford University
10
KIF -> DAML Example (nontrivial) cntd.KIF -> DAML Example (nontrivial) cntd.
(=> (and (instance ?SUBSTANCE BiochemicalAgent)(=> (and (instance ?SUBSTANCE BiochemicalAgent)(possesses ?AGENT ?SUBSTANCE))(possesses ?AGENT ?SUBSTANCE))
(capability BiochemicalAttack agent ?AGENT))(capability BiochemicalAttack agent ?AGENT))
Original KIFOriginal KIF
Translation of “?AGENT has BiochemicalAttack capability if it Translation of “?AGENT has BiochemicalAttack capability if it possesses a BiochemicalAgent”possesses a BiochemicalAgent”
<sumo:Capability rdf:ID=“BiochemicalAttackAgentCapability”><sumo:Capability rdf:ID=“BiochemicalAttackAgentCapability”> <sumo:capablityRole rdf:resource=“&sumo;#agent”/><sumo:capablityRole rdf:resource=“&sumo;#agent”/> <sumo:capabilityProcess rdf:resource=“#BiochemicalAttack”/><sumo:capabilityProcess rdf:resource=“#BiochemicalAttack”/></sumo:Capability></sumo:Capability><rdfs:Class rdf:ID=“AgentsWithBiochemicalAttackCapability”><rdfs:Class rdf:ID=“AgentsWithBiochemicalAttackCapability”> <rdfs:subClassOf><rdfs:subClassOf> <daml:Restriction><daml:Restriction> <daml:onProperty rdf:resource=“&sumo;#capability”/><daml:onProperty rdf:resource=“&sumo;#capability”/> <daml:hasValue rdf:resource=“#BiochemicalAttackAgentCapability”/><daml:hasValue rdf:resource=“#BiochemicalAttackAgentCapability”/> </daml:Restriction></daml:Restriction> </rdfs:subClassOf></rdfs:subClassOf> <daml:unionOf rdf:parseType=“daml:collection”><daml:unionOf rdf:parseType=“daml:collection”> <daml:Restriction><daml:Restriction> <daml:onProperty rdf:resource=“&sumo;#possesses”/><daml:onProperty rdf:resource=“&sumo;#possesses”/> <daml:hasClass rdf:resource=“#BiochemicalAgent”/><daml:hasClass rdf:resource=“#BiochemicalAgent”/> </daml:Restriction></daml:Restriction> … …
Knowledge Systems Laboratory, Stanford University
11
Query-Answering Example 1Query-Answering Example 1 ““What has the capability of being the agent of a biochemical attack?”What has the capability of being the agent of a biochemical attack?”
Query pattern: (capability ?agt Biochemical-Attack-Agent-Capability)Query pattern: (capability ?agt Biochemical-Attack-Agent-Capability)
Knowledge in the ontology:Knowledge in the ontology: A thing is an “Agent-With-Biochemical-Attack-Capability” if and only if it –A thing is an “Agent-With-Biochemical-Attack-Capability” if and only if it –
> Has a capability “Biochemical-Attack-Agent-Capability” orHas a capability “Biochemical-Attack-Agent-Capability” or> Possesses a “Biochemical-Agent”Possesses a “Biochemical-Agent”
An “Agent-With-Biochemical-Attack-Capability” has capability “Biochemical-Attack-Agent-An “Agent-With-Biochemical-Attack-Capability” has capability “Biochemical-Attack-Agent-Capability”Capability”
A “Nerve-Agent” is a “Biochemical-Agent”A “Nerve-Agent” is a “Biochemical-Agent”
If AGT has capability “Biochemical-Attack-Agent-Capability”, then AGT is capable of being If AGT has capability “Biochemical-Attack-Agent-Capability”, then AGT is capable of being an “agent” in a “Biochemical-Attack”an “agent” in a “Biochemical-Attack”
If C is the capability of playing role R in processes of type PT, and AGT is known to have If C is the capability of playing role R in processes of type PT, and AGT is known to have played role R in a process of type PT, then AGT has capability Cplayed role R in a process of type PT, then AGT has capability C
Knowledge from documents:Knowledge from documents: ““Al-Qaida” is a “Foreign-Terrorist-Organization” that possesses a “Nerve-Agent”Al-Qaida” is a “Foreign-Terrorist-Organization” that possesses a “Nerve-Agent”
““Aum-Supreme-Truth-Chemical-Attack-27-Jun-94 is a “Chemical-Attack” whose agent is Aum-Supreme-Truth-Chemical-Attack-27-Jun-94 is a “Chemical-Attack” whose agent is “Aum-Supreme-Truth”“Aum-Supreme-Truth”
Knowledge Systems Laboratory, Stanford University
12
Query-Answering Example 1Query-Answering Example 1
““What has the capability of being the agent of a biochemical attack?”What has the capability of being the agent of a biochemical attack?” Query pattern: (capability ?agt Biochemical-Attack-Agent-Capability)Query pattern: (capability ?agt Biochemical-Attack-Agent-Capability)
Answer: “Al-Qaida”Answer: “Al-Qaida” ““Al-Qaida” is a “Foreign-Terrorist-Organization” that possesses a “Nerve-Agent” Al-Qaida” is a “Foreign-Terrorist-Organization” that possesses a “Nerve-Agent”
[from documents][from documents]
A thing is an “Agent-With-Biochemical-Attack-Capability” if and only if it –A thing is an “Agent-With-Biochemical-Attack-Capability” if and only if it –> Has a capability “Biochemical-Attack-Agent-Capability” orHas a capability “Biochemical-Attack-Agent-Capability” or> Possesses a “Biochemical-Agent” [from ontology]Possesses a “Biochemical-Agent” [from ontology]
An “Agent-With-Biochemical-Attack-Capability” has capability “Biochemical-Attack-An “Agent-With-Biochemical-Attack-Capability” has capability “Biochemical-Attack-Agent-Capability” [from ontology]Agent-Capability” [from ontology]
If AGT has capability “Biochemical-Attack-Agent-Capability”, then AGT is capable of If AGT has capability “Biochemical-Attack-Agent-Capability”, then AGT is capable of being an “agent” in a “Biochemical-Attack”being an “agent” in a “Biochemical-Attack” [from ontology][from ontology]
A “Nerve-Agent” is a “Biochemical-Agent” [from ontology]A “Nerve-Agent” is a “Biochemical-Agent” [from ontology]
Knowledge Systems Laboratory, Stanford University
13
Query-Answering Example 1Query-Answering Example 1
““What has the capability of being the agent of a biochemical attack?”What has the capability of being the agent of a biochemical attack?” Query pattern: (capability ?agt Biochemical-Attack-Agent-Capability)Query pattern: (capability ?agt Biochemical-Attack-Agent-Capability)
Answer: “Aum-Supreme-Truth”Answer: “Aum-Supreme-Truth”
““Aum-Supreme-Truth-Chemical-Attack-27-Jun-94 is a “Chemical-Attack” whose Aum-Supreme-Truth-Chemical-Attack-27-Jun-94 is a “Chemical-Attack” whose agent is “Aum-Supreme-Truth” [from documents]agent is “Aum-Supreme-Truth” [from documents]
Playing the role “agent” in a “Biochemical-Attack” requires the capability Playing the role “agent” in a “Biochemical-Attack” requires the capability “Biochemical-Attack-Agent-Capability” [from ontology]“Biochemical-Attack-Agent-Capability” [from ontology]
If playing role R in a process of type PT requires capability C, and Agt plays role R in If playing role R in a process of type PT requires capability C, and Agt plays role R in a process of type PT, then Agt has capability C [from ontology]a process of type PT, then Agt has capability C [from ontology]
““Aum-Supreme-Truth” has capability “Biochemical-Attack-Agent-Capability”Aum-Supreme-Truth” has capability “Biochemical-Attack-Agent-Capability”
Knowledge Systems Laboratory, Stanford University
14
Query-Answering Example 2Query-Answering Example 2
““Who are the agents of attacks that used the same type of weapons as Who are the agents of attacks that used the same type of weapons as “Recent-Attack-001?”“Recent-Attack-001?” Query pattern: (type Recent-Attack-001 ?res) (onProperty ?res instrument) Query pattern: (type Recent-Attack-001 ?res) (onProperty ?res instrument)
(hasClass ?res ?inst-type) (type ?attack ?res) (agent ?attack ?agt)(hasClass ?res ?inst-type) (type ?attack ?res) (agent ?attack ?agt)
Must-bind variables: ?agt ?attackMust-bind variables: ?agt ?attack
Knowledge in the ontology:Knowledge in the ontology: A “Mortar-Attack” has an instrument of type “Mortar”A “Mortar-Attack” has an instrument of type “Mortar”
Knowledge from documents:Knowledge from documents: ““Recent-Attack-001” is a Thing that has an instrument of type “Mortar”Recent-Attack-001” is a Thing that has an instrument of type “Mortar”
““Revolutionary-Armed-Forces-Of-Colombia-Mortar-Attack-1-Jul-00” is a “Mortar-Revolutionary-Armed-Forces-Of-Colombia-Mortar-Attack-1-Jul-00” is a “Mortar-Attack” that has agent “Revolutionary-Armed-Forces-Of-Colombia”.Attack” that has agent “Revolutionary-Armed-Forces-Of-Colombia”.
Answer: “Revolutionary-Armed-Forces-Of-Colombia”Answer: “Revolutionary-Armed-Forces-Of-Colombia”
AQUA Program PlanAQUA Program Plan
Overview of the projectOverview of the project Goal is to create a system that can answer complex Goal is to create a system that can answer complex
questionsquestions With plus up funding, we now have an end-to-end With plus up funding, we now have an end-to-end
system. Makes use of KSL’s Ontolingua Knowledge system. Makes use of KSL’s Ontolingua Knowledge Server and Java Theorem Prover (JTP) to develop Server and Java Theorem Prover (JTP) to develop answers to queriesanswers to queries
Uses SAIC and other technology to automatically Uses SAIC and other technology to automatically populate KBs with information from new text sourcespopulate KBs with information from new text sources
Uses multiple extractors from multiple sources to Uses multiple extractors from multiple sources to answer queriesanswer queries
> KSL extractorKSL extractor> UMBC/NMSU extractorUMBC/NMSU extractor> IBM extractorIBM extractor
SAIC
KIF TMR
Mapper/Translator
NLQUESTION
CNSTEST DATA
MOQA
Text TMR
Translator
IBM
Text KIF
Translator
KIF-Formatted
Question
KIF Answer/
Proof tree
OntolinguaKnowledge Server
----------------JAVA Theorem
Prover
KSL generated
explanation
A
D
SAIC
TMR KIF
Mapper/Translator
CMOQA
NL TMR
Query Processor
B
MOQA
TMR NL
Answer Processor
NLANSWER
E
DATASAIC (A-F)MOQAKSLIBM
SAIC- Team
CNS Data
GenerationF
KSL
Extractor
AQUA Current PlansAQUA Current Plans
AQUA Initial ConceptAQUA Initial Concept
QUESTION
NL QueryInterlingua Query
KIF Q
uery
KIF Answer
Interlingua Answer
NL Answer
ANSWER
NMSU Query Processor
SAIC Interlingua KIF Translator
KSL Java Theorem Prover
SAIC KIF Interlingua Translator
NMSU NL Generator
Key Tasks - SAIC Key Tasks - SAIC Perform translation of Onyx/UMBC Perform translation of Onyx/UMBC
extracted TMRs to KIF (Item A)extracted TMRs to KIF (Item A) Align two disparate ontologiesAlign two disparate ontologies Translate terms once alignedTranslate terms once aligned
Both formalized queries and extracted text Both formalized queries and extracted text need to be translatedneed to be translated
Develop CNS WMD ontologyDevelop CNS WMD ontologyCo-ordinate subcontractors and develop Co-ordinate subcontractors and develop
system interfacessystem interfaces
Key Tasks - OnyxKey Tasks - OnyxProvide formalized translation of NL queries Provide formalized translation of NL queries
(MOCA – item B)(MOCA – item B)Perform extraction of CNS data into text Perform extraction of CNS data into text
(MOCA – item A)(MOCA – item A)
Key Tasks - IBMKey Tasks - IBMAssist in relations extraction from text into Assist in relations extraction from text into
WMB ontologyWMB ontology
KSL’s Current ActivitiesKSL’s Current Activities JTPJTP – Hybrid reasoning for query answering – Hybrid reasoning for query answering
Includes a temporal reasonerIncludes a temporal reasoner
Is a DQL (DAML Query Language) serverIs a DQL (DAML Query Language) server
Knowledge Base PartitioningKnowledge Base Partitioning – Enabling Q-A from large scale KBs using parallel – Enabling Q-A from large scale KBs using parallel heterogeneous reasoners heterogeneous reasoners
Inference WebInference Web – Providing understandable explanations for derived query answers – Providing understandable explanations for derived query answers
Knowledge extractionKnowledge extraction from semi-structured documents from semi-structured documents Tables, lists, outlines, property-value pairs, etc.Tables, lists, outlines, property-value pairs, etc.
SAIC Current Activities SAIC Current Activities SAICSAIC
In-house Ontolingua server with JTP now In-house Ontolingua server with JTP now installed and in use in development effortsinstalled and in use in development efforts
Ontology is available as part of demonstration Ontology is available as part of demonstration in the demo roomsin the demo rooms
Please visit the SAIC/KSL demo standPlease visit the SAIC/KSL demo stand
SAIC Current Activities (cont.)SAIC Current Activities (cont.)SAIC spearheading a federation of a WMD SAIC spearheading a federation of a WMD
ontology development effort, assisted by ontology development effort, assisted by Stanford KSLStanford KSL
Begun development of CNS ontology. Begun development of CNS ontology. Ontology is currently 700 terms and Ontology is currently 700 terms and viewable in our in-house version of viewable in our in-house version of Ontolingua. (Demo available)Ontolingua. (Demo available)
SAIC Current Activities (cont.)SAIC Current Activities (cont.) Discussions underway with Sergei to put Onyx Discussions underway with Sergei to put Onyx
under subcontract to SAIC. Subcontract to go out under subcontract to SAIC. Subcontract to go out as soon as possible.as soon as possible. Labor division is defined and agreed toLabor division is defined and agreed to Major issue – Due to subcontract issues Onyx is still not Major issue – Due to subcontract issues Onyx is still not
under subcontract. This affects Q?A ayatem under subcontract. This affects Q?A ayatem development rates as this task is on the critical path for development rates as this task is on the critical path for system development.system development.
Distributed ontology to KSL and IBM. Distributed ontology to KSL and IBM. Development of the ontology is critical in order to Development of the ontology is critical in order to
allow the extractors to function appropriatelyallow the extractors to function appropriately
WMD Ontology Creation Initial -Confederation WMD Ontology Creation Initial -Confederation AssignmentsAssignments
Stanford/KSL: NIS-Facilities (439 terms) and Stanford/KSL: NIS-Facilities (439 terms) and Russian-Naval-Facilities (365 terms)Russian-Naval-Facilities (365 terms)
IBM: MPT-Topic (771 terms)IBM: MPT-Topic (771 terms)Xerox-Parc: Missiles-Topic (765 terms)Xerox-Parc: Missiles-Topic (765 terms)Tecknowledge: NIS-Nuclear-Weapons-Tecknowledge: NIS-Nuclear-Weapons-
Aggregate (219 terms)Aggregate (219 terms)Battelle: Nuclear-Safety-Assistance (36 Battelle: Nuclear-Safety-Assistance (36
terms)terms)
Knowledge Systems Laboratory, Stanford University
26
Year Two Project GoalsYear Two Project GoalsComplete CNS ontology developmentComplete CNS ontology developmentParticipate in TREC Participate in TREC
System is still immature System is still immature Novel appoachNovel appoach Significant potential for further developmentSignificant potential for further development
Refine interfaces and determine system Refine interfaces and determine system metrics to ensure maximum performance in metrics to ensure maximum performance in future system iterationsfuture system iterations
Knowledge Systems Laboratory, Stanford University
27
TREC participationTREC participation SAIC is signed up for TREC participation this year.SAIC is signed up for TREC participation this year. A multi-pronged approach is possible with the A multi-pronged approach is possible with the
current architecturecurrent architecture With the SAIC/Onyx route and NL interface, gives With the SAIC/Onyx route and NL interface, gives
the initial capability for an end-to-end system with the initial capability for an end-to-end system with restricted domain and rangerestricted domain and range
Formatted queries possible for IBM extractionFormatted queries possible for IBM extraction System will be very immature in year 1 and likely System will be very immature in year 1 and likely
achieve poor TREC scores, but will mature in achieve poor TREC scores, but will mature in multiple and novel directions over timemultiple and novel directions over time
Knowledge Systems Laboratory, Stanford University
28
Future PlansFuture PlansContinue multi-pronged approach (running Continue multi-pronged approach (running
multiple extractors over a uniform KB)multiple extractors over a uniform KB)Plan further enhancements (Possibly add Plan further enhancements (Possibly add
more extractors or reasonersmore extractors or reasonersLeverage multiple KB approach to optimize Leverage multiple KB approach to optimize
research in multi-partition reasoningresearch in multi-partition reasoningDevelop effective metrics to determine Develop effective metrics to determine
efficacy of this approach and which efficacy of this approach and which pathways are optimalpathways are optimal
Knowledge Systems Laboratory, Stanford University
29
Future plans (Cont)Future plans (Cont)Work on implementing latter Proof tree to Work on implementing latter Proof tree to
NL mocha interface in the future (Reverse NL mocha interface in the future (Reverse TMR to KIF)TMR to KIF)
Transition from KIF to DAML format where Transition from KIF to DAML format where possiblepossible
Extend range and capabilities of question Extend range and capabilities of question answering. Initial participation will be answering. Initial participation will be limited in terms of domain and range of limited in terms of domain and range of questions.questions.