saic system architecture
DESCRIPTION
SAIC ArchitectureTRANSCRIPT
Presentation AgendaPresentation Agenda
- SAIC Introduction- Stanford (KSL)- SRI International - Stanford (Formal Reasoning Group)- NWU- MIT- CMU- TextWise- SAIC Summary
SAIC Integrated Knowledge SAIC Integrated Knowledge Environment (SIKE) Environment (SIKE)
ArchitectureArchitecture
Architecture exists at two levels - System Level Architecture
Transport Layer Syntactic Layer
Knowledge Architecture Semantic Layer
HPKB Integrated Knowledge HPKB Integrated Knowledge Environment (HIKE) Environment (HIKE)
ArchitectureArchitecture
Architecture exists at two levels - System Level Architecture
Transport Layer Syntactic Layer
Knowledge Architecture Semantic Layer
System Level ArchitectureSystem Level ArchitectureFeaturesFeatures
A distributed heterogeneous environment to solve Crisis Management Challenge Problem.
Federation of OKBC(Open Knowledge Base Connectivity) servers
Added power of component-based approach for the distribution of knowledge content
Web based graphical user interface
STARTSTART
Analyst
OntolinguaOntolingua
HIKEGUI
HIKEGUI
Ocelot&
PERK
Ocelot&
PERK
ATPATP
SNARKSNARK
SMEMAC/FAC
SMEMAC/FAC
GKB Editor
GKB Editor
TextWiseTextWise
WebKBWebKB
JOTJOT
ATPLATPL
Crisis Management -Crisis Management -Knowledge Level ArchitectureKnowledge Level Architecture
Knowledge Architecture design is an output of the Knowledge Architecture working group convened by SAIC
Includes the SAIC merged ontology The SAIC merged ontology contains the year 1
knowledge bases from KSL, NWU, FRG, SRI, SAIC, and CMU
Ontology merging effort led by Stanford KSL led to development of the KB merging tool
SAIC CM CP KnowledgeSAIC CM CP KnowledgeArchitectureArchitecture
HPKB Upper Level
SAIC Merged Ontology (Y1)
PQ CasesActionsInterests
Year 2 Domain Specific
Analogy ...
SAIC Merged Ontology (Y1) SAIC Merged Ontology (Y1) DomainsDomains
Capability Analysis Benefits/Risks analysis Terrorism World Fact Book International economics model A National interests model A model of economic, military, and diplomatic
support/opposition. World oil flow
SAIC Merged Ontology (Y1) SAIC Merged Ontology (Y1) DomainsDomains
Properties of multilateral organizations Capabilities and Resources International Organizations, Companies Military weapons, artillery, personnel Strike Capabilities EIA pages (oil quotas, etc) International Organizations,
memberships, goals Geographical information
Common Knowledge Common Knowledge ComponentsComponents
PQ Ontology Ontology used to define the vocabulary available for
the user to query the system. Actions
A model of international actions described in the International System Framework Document (ISF).
Interests A model of national interests and strategic interests
defined by the ISF.
Common Knowledge Common Knowledge Components (Cont’d)Components (Cont’d)
Analogy Ontology Case Library
Year 1 Scenario Year 2 Scenario 1998 Iranian-Taliban Crisis Abu Musa Incident Caspian Pipeline Consortium (CPC) Operation Desert Shield 1990-1 1984-8 Tanker War
…
Knowledge Base Knowledge Base Development StrategyDevelopment Strategy
Shared upper structure and SAIC merged ontology
Common components across developers Periodic KB merging into common
components
Knowledge ArchitectureKnowledge Architecture
Currently available in Ontolingua HPKB upper level SAIC merged Ontology (Y1) PQ Ontology Knowledge Components …..
http://ontolingua.stanford.edu
SAIC Crisis Management SAIC Crisis Management Year 2 PQ distribution Year 2 PQ distribution
Different technology developers assume responsibility for specific PQs, but make use of shared knowledge structures
PQ distribution as shown (next slide)
Parameterized Question Distribution
200 SRI 220 SRI 240 SAIC201 SRI 221 SRI202 SRI 222 SRI 251 SAIC203 SRI 223 NWU 252 KSL204 SRI 224 NWU 253 KSL205 FRG 225 NWU 254 SRI206 SRI 226 NWU 255 SAIC207 FRG
228 NWU 124 KSL, MIT209 SRI 125 KSL, MIT210 SRI 230 FRG 126 KSL, MIT211 KSL 231 FRG 127 KSL, MIT212 KSL 232 KSL 128 KSL213 KSL 233 KSL214 SAIC 234 SRI
216 MIT/START 236 SAIC217 MIT/START 237 SAIC
238 SAIC219 SRI 239 SAIC
Critical Component Critical Component Experiments (CCEs)Experiments (CCEs)
Theory Merging CCE Led by KSL. Merges CMU, FRG, KSL, NWU, SAIC and
SRI Knowledge Bases. Develops merging tools and techniques Merging evaluation (TBD)
Critical Component Critical Component Experiments (CCEs)Experiments (CCEs)
Knowledge Extraction (TextWise) TextWise parses a multi-year multi-source
corpus to produce output that populates terrorism templates defined by SAIC.
Phased approach Terrorist Group Template definitions loaded into
SNARK KB (currently available) Post January: Population of Terrorist Event and
Supporting Action templates
Critical Component Critical Component Experiments (CCEs)Experiments (CCEs)
Natural language interface to selected Parameterized Questions using START/SNARK MIT START team parses natural language and
converts this text into KIF formalizations that are then input to SRI SNARK theorem prover.
Server used for START queries also used by SAIC GUI interface.
Critical Component Critical Component Experiments (CCEs)Experiments (CCEs)
Analogical Reasoning Led by NWU NWU will answer the analogical reasoning PQs
for the SAIC integration team. The questions will be answered as follows
Analogy Ontology (NWU) SME, MAC/FAC (Analogical Reasoner) (NWU) Case Library (SAIC)
All Ontologies stored in Ontolingua
SAIC Crisis Management SAIC Crisis Management User InterfacesUser Interfaces
GUI interface to SNARK (live) remote version (Server at SRI) local (server on laptop)
GUI interface to ATP Lisp translator to facilitate batch interface
processing of PQs
Stanford KSLStanford KSL
Stanford KSLStanford KSL
Richard Fikes
Deborah McGuinness
James Rice
Gleb Frank
Yi Sun
Stanford KSL-ATP & ATPLStanford KSL-ATP & ATPL ATP is supported and in use for challenge
problem work Providing ATP for use by FRG ATP has been upgraded to handle larger KBs ATP client side listener developed for remote
building and testing of KBs (see demo!) ATPL available for SAIC challenge problem
use offered knowledge server support to NWU
KSL-Challenge Problem KSL-Challenge Problem WorkWork
PQ answers (over 1/4 of questions) KB diagnostics differential questions
Merging CCE Led merge of Y1 KBs Developed initial merging tool Providing knowledge library of individual and
merged Y1 (and Y2) KBs
Explanation Approach IExplanation Approach I Break queries and answers into components based
on their logical form
conjunctive antecedents are separated
follow-up queries are generated for those that are not
directly asserted
query bindings may be presented
Explanation Approach IIExplanation Approach II Present in pseudo natural language
Use documentation strings and internal templates
Axiom: Diplomatic-Opposition-Propagation-Due-To-Group-Membership
(=> (and (Opposed-Diplomatically ?group ?enemy ?time-range)
(Group-Members ?group ?member))
(Opposed-Diplomatically ?member ?enemy ?time-range))
Doc String: ?member diplomatically opposed ?enemy because
?member is a member of ?group, which opposed ?enemy.
Explanation Approach IIIExplanation Approach III Prune (and/or rewrite) internal axioms
delete internal axioms such as “if a class is known to be
non-primitive, its primitiveness is false” by setting
explanation-visibility to be internal
generate abstract presentation strings for axioms such as
taxonomic inheritance
Explanation Approach IVExplanation Approach IV
Present abstractions for multiple answers “members of the UN-Security Council opposed Iraq”
rather than listing all of the members
Provide meta language for contextual and
domain-oriented pruning explanation visibility, slots to use for abstraction,
“interesting” slots, etc.
TAA68 TAA68 What countries diplomatically opposed What countries diplomatically opposed Iraq after the Persian Gulf War?Iraq after the Persian Gulf War?
Incremental ExplanationsIncremental Explanations
Incremental Explanations IIIncremental Explanations II
Status and PlansStatus and Plans Status
Implemented for ATP Tested on KSL Y1 and some Y2 queries
Plans Implement pruning meta language based on description logic
foundation
Expand to other reasoners (e.g., SNARK)
Demonstrations available
SRISRI
SRI’s Contribution to SRI’s Contribution to IntegrationIntegration
Helped conceptualize the HIKE GUI Delivered a PC-based SNARK server Helped produce the SAIC merged ontology START/SNARK interface Loading information extracted by Textwise
Merging with Team SAICMerging with Team SAIC
Syntactic merge Semantic merge Computational merge
Syntactic MergeSyntactic Merge
KBs translated into the same language Different ways to write the same thing
(person ?x) or (instance-of ?x person) We converted our KBs into a syntax that
will be readable by KSL
Most (95%) of the work can be automated
Semantic MergeSemantic Merge
Semantic merge Identical terms should have the same
definitions Differences in representational choices
Mostly manual, but some tools possible
(Supporting-Terrorist-Attack ?action) =(and (instance-of ?action action) (supports ?action terrorist-attack))
Computational MergeComputational Merge
Merged KB can be as efficiently reasoned with as the original
Sorted vs unsorted language Consider (father ?x ?y)
The first argument must be a male
The second argument must be a person
In a sorted language, ?x will unify with only males
CMCP Knowledge BaseCMCP Knowledge Base
HPKB Upper Level
SAIC Merged Ontology (Y1)
PQ AgentsActionsInterests
ReadingComprehension
Option Generation
Option Evaluation
Cases
CMCP Knowledge BaseCMCP Knowledge Base
Responsibility for about 20 PQs Actively co-developing content with SAIC
Interface withInterface withProject Genoa Project Genoa
Structured Argumentation
Publish Arguments
Direct entry by SMEs
Fusion Fusion Fusion Fusion
Fusion
Q1.1.1 Q1.1.2 Q1.1.3 Q1.4.1 Q1.4..2 Q1.4.3Q1.2.1 Q1.2.2 Q1.2.3 Q1.3.1 Q1.3.2 Q1.3.3
A1.1 A1.2 A1.3 A1.4
A1
Final Conclusion
Is the project being managed according to the project plan?OK Caution Warning
Evidence:
Will the effort be completed on or ahead of schedule?
Will this effort be completed within the budget?
Will the technical solution be developed according to plan?
Will project resources for this effort be available according to plan?
Will operations be satisfied by the results of the project? OK Caution Warning
Evidence:
Will the projected capital & operating costs meet requirements?
Will the projected operating performance meet requirements?
Do projected operating benefits justify expected expenditures?
Are communications between project & operations staff satisfactory?
Argument
Templates
Interface with Project GenoaInterface with Project GenoaAccomplishments for 1998Accomplishments for 1998
OracleDB
SEAS Server
OKBC
WWW Browser
HTTP/HTML
Fusion Fusion Fusion Fusion
Fusion
Q1.1.1Q1.1.2Q1.1.3 Q1.4.1Q1.4..2Q1.4.3Q1.2.1Q1.2.2Q1.2.3 Q1.3.1Q1.3.2Q1.3.3
A1.1 A1.2 A1.3
A1
Ontology Manager
Grasper
Gister Engine
CL-HTTP Server
Oracle DBMSServer
SQL
Ocelot KBMS
Perk StorageSystem
Arg./Sit.Ontology
CWEST
SEAS HTML Generator
OKBC GKB-Browser
Interface with Project GenoaInterface with Project GenoaPlans for 1999Plans for 1999
Integration at content level Use situation ontology from HPKB for argument
indexing Multi-user editing of arguments
Use collaboration system for asynchronous editing Domain-specific GUI for editing argument
ontology Enhance GKB-Editor to be more accessible to SMEs
MIT - STARTMIT - START
MIT (START): MIT (START): Y2 Integration PlansY2 Integration Plans
Link START to other HPKB systems by translating English queries into PQ specifications, then forwarding the translated queries
Extend the START Server’s KB with background knowledge to support analyst’s activities
Support answering selected Parameterized Questions for the Y2 Crisis Management Challenge Problem
Increase START’s access to “live” information from the World Wide Web by incorporating robust access interfaces
MIT (START): New Coverage for Y2
• Material from the International System Framework and Agent-Specific Background Information documents, supporting PQs 216, 217, 124, 125, 126 and 127
• Background information on terrorist groups, including membership, activities, funding and locations
• Weapon strike capabilities between Persian Gulf regions and countries
• Information on Fortune 500 companies, including locations of headquarters, CEOs, assets, profits and stock prices
• Information on 30,000 U.S. cities, including areas, populations, coordinates, time zones and weather
MIT (START): New MIT (START): New Coverage for Y2Coverage for Y2
Material from the International System Framework and Agent-Specific Background Information documents, supporting PQs 216, 217, 124, 125, 126 and 127
Background information on terrorist groups, including membership, activities, funding and locations
Weapon strike capabilities between Persian Gulf regions and countries
Information on Fortune 500 companies, including locations of headquarters, CEOs, assets, profits and stock prices
Information on 30,000 U.S. cities, including areas, populations, coordinates, time zones and weather
NWUNWU
CMUCMU
CMU CM PlansCMU CM Plans Extract relevant ground facts from the Web
company instances name locations of operations economic sector products produced and raw materials consumed
(especially those on export-control lists) relations with other companies pieces of infrastructure
instances of <EconomicActionType>
CMU CM PlansCMU CM Plans
Deliver extracted facts to integration teams via OKBC.
Use facts to support PQs 200, 201, 203, 211, 216, etc. by representing economic interests, capabilities and actions of international agents, and links among agents.
Integration of Text ExtractionIntegration of Text Extractionwith SAIC Terrorism DBwith SAIC Terrorism DB
Ian Niles
TextWise, LLC
SAIC Terrorism DBSAIC Terrorism DB
(defobject ABU-NIDAL-ORGANIZATION"International terrorist organization led by Sabri al-Banna. Split from PLO in 1974. Made up of various functional committees, including political, military, and financial.(Source: 1996 Patterns of Global Terrorism:App. B: Background on Terrorist Groups, http://www.iet.com/Projects/HPKB/Web mirror/GLOB_terror/appb.html)”
(own-slot-value nick-name ABU-NIDAL-ORGANIZATION "ANO")
(individual ABU-NIDAL-ORGANIZATION)
(instance-of ABU-NIDAL-ORGANIZATION terrorist-group)
(residence-of-organization ABU-NIDAL-ORGANIZATION libya))
Integration of CRCs into DBIntegration of CRCs into DB
Terrorist Group template instances were automatically generated from KNOW-IT output in three steps:
A base template instance is created for each example of the proper noun category 54 (terrorist groups)
CRCs referencing terrorist groups are mapped to slots of the terrorist group template.
The automatically generated slots are inserted into the appropriate template instances.
Automatically Generated Template Automatically Generated Template InstancesInstances
(defobject HAMAS
"(Source: 1998 TextWise LLC Terrorism Database)"
(individual HAMAS)
(instance-of HAMAS terrorist-group)
(affiliated-with Palestine-Liberation-Organization)
(own-slot-value nick-name HAMAS Hamas)
(own-slot-value nick-name HAMAS Islamic-Resistance- Movement)
(residence-of-organization HAMAS Israel)
(residence-of-organization HAMAS United-States)
(residence-of-organization HAMAS West-Bank))
Automatically Generated Template Automatically Generated Template Instances (con’t)Instances (con’t)
(defobject Hizballah
"(Source: 1998 TextWise LLC Terrorism Database)"
(individual Hizballah)
(instance-of Hizballah terrorist-group)
(affiliated-with Islamic-Jihad)
(own-slot-value nick-name Hizballah Islamic-Jihad-for-the-Liberation-of-Palestine)
(own-slot-value nick-name Hizballah Lebanese-Hizballah)
(own-slot-value nick-name Hizballah Party-of-God)
(own-slot-value nick-name Hizballah Hezbollah)
(own-slot-value nick-name Hizballah Hizbollah)
(own-slot-value nick-name Hizballah Organization-of-the-Oppressed-on-Earth)
(own-slot-value nick-name Hizballah Revolutionary-Justice-Organization)
(residence-of-organization Hizballah Lebanon))
Future Integration WorkFuture Integration Work Crafting more rules to extract instances of the 54 (terrorist
group) proper name category
Automatic generation of instances of the two other Terrorism DB templates
Mapping more relations and combinations of relations to template slots
Making the ouput KIF 3.0 Compliant
•Carnegie Mellon University
•TextWise
•SRI International
•North Western University
•Stanford University (Knowledge Systems Laboratory)
•Stanford University (Formal Reasoning Group)
•Stanford University (Scaleable Knowledge Composition)
•Massachussets Institute of Technology
•Carnegie Mellon University
•TextWise
•SRI International
•North Western University
•Stanford University (Knowledge Systems Laboratory)
•Stanford University (Formal Reasoning Group)
•Stanford University (Scaleable Knowledge Composition)
•George Mason University
•Massachussets Institute of Technology
•Information Sciences Institute
•Stanford Medical Informatics
BackupsBackups
KB Development Time (Exluding TextWise)
0
5000
10000
15000
20000
25000D
ec
Jan
Feb
Mar
Apr
May
June
(SQ
)
TQ
A
TQ
B
TQ
C
TQ
D
Aug
Months
Axi
oms
SRI(SAIC)KSLNWUCMU
SAIC Crisis Management only
KB Development Time
010000
200003000040000
500006000070000
8000090000
Dec
Jan
Feb
Mar
Apr
May
June
(SQ
)
TQ
A
TQ
B
TQ
C
TQ
D
Aug
Months
Axi
oms
SRI(SAIC)KSLNWUTextWiseCMU
SAIC Crisis Management only
TextWiseTextWise 1. Create a terrorism database partition by retrieving a large multi-year, multi-
source corpus of documents which mention the terms"terrorism", "terrorist" or "terrorists" and running the document processing system over these documents (date of deliverable: 11/27).
2. Create an index from every canonicalized PN in the version of PNDBin /home/chess/CYC to all of its non-canonicalized variants (date ofdeliverable: 11/27).
3. Implement the pseudo-code for the Template Instance Generator (TIG)(date of deliverable: 12/31).
4. Design and implement component which will convert sets of CRCs intoinstances of Supporting Actions and Terrorist Attacks templates.
CreditsCMU - webKB• Tom Mitchell• Mark Craven
MIT - START• Boris Katz• Gary Borchardt
NWU - Flow Model• Ken Forbus• Jeff Usher
SRI - SNARK, GKB Editor• Vinay Chaudhri• Richard Waldinger• Mark Stickel
Stanford KSL - Ontolingua, ATP• Richard Fikes• Deborah McGuinness• James Rice• Gleb Frank
Stanford - FRG• John McCarthy• Tom Costello
TextWise - Know-IT• Liz Liddy• Woojin Paik
USC ISI - LOOM/EXPECT• Yolanda Gil• Jim Blythe
CreditsUSC ISI - LOOM• Bob Mcgregor• Hans Chalupsky• David Moriarty
Cycorp - Cyc• Doug Lenat• Ben Rode
Teknowledge - TFS• Adam Pease• John Li• Cleo Condoravdi
GMU - Disciple• George Tecucci• Katy Wright
SMI - Protege• Mark Musen• Natalya Fridman Noy• Bill Grosso
SAIC - SIKE• Dave Easter• Albert Lin• Barbara Starr• Don Henager• Henry Gunthardt• Ben Good• Brian Truong• Bryner Pancho• Lei Wang
HIKE N-tier ArchitectureHIKE N-tier Architecture
HIKEClient
HIKEClient
HIKEClient
HIKEClient
HIKEClient
HIKEClient
HIKEServer
HIKEServer
HIKEServer
HIKEServer
HIKEServer
HIKEServer
HIKEStub
HIKEStub
HPKBTechnologyComponent
HPKBTechnologyComponent
HPKBTechnologyComponent
HPKBTechnologyComponent
HPKBTechnologyComponent
HPKBTechnologyComponent
HIKEStub
HIKEStub
HPKBTechnologyComponent
HPKBTechnologyComponent
LoomOKBCServer
LoomOKBCServer
OKBC
OKBC
Java RMI
HTTP
Sockets (TCP/IP)
Three Levels of IntegrationThree Levels of Integration
There are 3 levels at which integration can occur: Transport layer (e.g. Sending information from
one server to another) Syntactic layer (Ensuring that information is in the
same syntax as that defined by another system) Semantic layer (Ensuring that all concepts and
theories are aligned) defined last year by Adam Pease
STARTSTART
OntolinguaOntolingua
HIKEGUI
HIKEGUI
OcelotOcelot
ATPATP
SNARKSNARK
SMEMAC/FAC
SMEMAC/FAC
GKB Editor
GKB Editor
TextWiseTextWise
WebKBWebKB
Analyst
Knowledge ArchitectureKnowledge Architecture
Currently available in Ocelot (Via GKB editor) HPKB upper level Actions Ontology Interests Ontology SAIC/SRI Y1 Ontology
lajolla.ai.sri.com:8000
Knowledge ServersKnowledge Servers A federation of OKBC Knowledge Servers
LOOM (USC ISI) Ontolingua (Stanford KSL) Ocelot (SRI) Cyc (Cycorp) ATP
Manual Knowledge Acquisition Tools GKB Editor (SRI) Ontolingua (Stanford KSL) JOT ATPL Ontosaurus (USC ISI) Expect (USC ISI)
Knowledge Servers (Cont’d)Knowledge Servers (Cont’d)
Semi- Automatic Knowledge Acquisition KNOW-IT (TextWise)
Text extraction from the web, newsfeeds and other sources
webKB (CMU) Knowledge Extraction (and discovery) from web
based sources.
Expect (USC ISI) Automatic generation of rules
Question AnsweringQuestion Answering
Natural Language Understanding START (MIT)
Parses natural language queries. Multimedia web based answers from annotated web sources.
TextWise Parses natural language queries. Returns answers from web
based sources by parsing textual information.
Theorem Provers SNARK (SRI) ATP (Stanford KSL)
Problem SolversProblem Solvers
Machine Learning Disciple Learning Agent (GMU)
multi-strategy learning methods Problem Solving Methods
Problem Solving Methods Stanford Medical Informatics (SMI)
Three layered PSM to detect, classify, and monitor battlefield activities.
Information Science Institute (ISI) Course of Action Generation problem solvers to
create alternative solutions to workarounds problems.
Problem Solvers (Cont’d)Problem Solvers (Cont’d)
Bayesian Networks SPOOK (Stanford Robotics Laboratory)
System for Probabalistic Object Oriented Knowledge - supports reasoning with uncertainty
Qualitative Reasoning NWU/KSL
supports construction of certain types of models such as flow models, e.g. :
World Oil flow model Common Sense reasoning about the battlespace, focusing on the
trafficability/terrain suitability task.
Problem Solvers (Cont’d)Problem Solvers (Cont’d) Monitoring Process
Massachusetts Institute of Technology (MIT) provides tools for constructing and controlling
networks of distributed monitoring processes
Crisis Management -Crisis Management -Knowledge Level ArchitectureKnowledge Level Architecture
Knowledge Architecture design is an output of the Knowledge Architecture working group convened by SAIC
Includes the SAIC merged ontology The SAIC merged ontology contains the year 1
knowledge bases from KSL, FRG, SRI/SAIC, and CMU
Ontology merging effort led by Stanford KSL led to development of the KB merging tool