June 11-13, 2002 AQUA Question-Answering System 1

AQUA: AQUAINT QuestionAQUA: AQUAINT QuestionAnswering SystemAnswering SystemProject Progress Report Project Progress Report

SAIC, San DiegoKSL, Stanford

June 11-13, 2002 AQUA Question-Answering System 2

Project SummaryProject Summary

June 11-13, 2002 AQUA Question-Answering System 3

Collaboration with NMSU TeamCollaboration with NMSU Team

This spring, NMSU has been added as an AQUAINT contractorSynergies between the NMSU Team and the SAIC Team have led to a collaborative effort for AQUAINT

June 11-13, 2002 AQUA Question-Answering System 4

SAIC-KSL-NMSU Collaborative SystemSAIC-KSL-NMSU Collaborative System

QUESTION

NL QueryInterlingua Query

KIF Q

uery

KIF Answer

Interlingua Answer

NL Answer

ANSWER

NMSU Query Processor

SAIC Interlingua KIF Translator

KSL Java Theorem Prover

SAIC KIF Interlingua Translator

NMSU NL Generator

June 11-13, 2002 AQUA Question-Answering System 5

Key Tasks—KSL Key Tasks—KSL

Providing knowledge tools for teamOntolingua Knowledge Server (OKS)Background ontologies and lexiconsJTP deductive answer determination system

Developing new methods forAnswer determination from large, complex KBs

• Using interoperating special-purpose reasoners • KB partitioning so that most reasoning occurs within partitions• Using Semantic Web representation languages (e.g., DAML+OIL)

Providing understandable explanations for deduced answers• Reasoning steps, data sourcessources, reasoning method, data

conversionsconversions, …• Use-specific and user-specific customization

Detecting and resolving conflicts in KBs• Proactive background testing for both likelylikely and provable

conflicts• Interactive tools for helping analyst correct or annotateannotate conflicts

June 11-13, 2002 AQUA Question-Answering System 6

Progress to Date—KSL Progress to Date—KSL

Developing JTP – An answer determining reasonerDeveloping a suite of special-purpose reasoners

E.g., for determining time-dependent answers

Reasoner for QA from ontological knowledge (in DAML+OIL)Produces and caches reasoning results during KB loadingWill be able to accept a series of queries, without waiting for answers

Usability improvements to JTPFor KB development and query-answering testing and debuggingSupport for rapid reloading and changing of developing KBs

E.g., Checkpointing and “untell”

Hierarchical presentation of reasoning explanationsDQL – Standard query language and QA protocol for DAML+OIL

Basic framework for client/server deductive query answering• Answers generated a batch at a time• Formal semantics

Applicable to other representation languagesBeing developed jointly with the DAML language design committeeWill be implemented for JTP

June 11-13, 2002 AQUA Question-Answering System 7

Key Tasks—SAIC Key Tasks—SAIC Ontolingua Translator

Automatic translation of information from NMSU team into KIF knowledge representations

• Includes dynamic semantic alignment of TMR Ontology and Ontolingua Ontology• Includes inferring relations that span multiple sentences• Includes flagging relations with document contexts (time, location)

Pre-reasoning to pre-answer known useful questions• Modular, interchangeable KB systems to determine specific types of relations

that are likely to be highly relevant and useful• Current system is event-based, but architecture valid for other types of

representations

OKS InterfaceTransfer, load, and store KIF representations into Ontolingua KBsMaintain DB of translated documents with info about contents of each

• Defines what documents are relevant to specific queries• Reduces burden of KB partitioning

– Each document has a separate KB file– Query TMR matched to contents of known documents to determine relevant KBs– Only KBs for relevant documents loaded into JTP to determine the answer

Source Credibility AssessmentDevelop historical records of source reliability for sourcesAssessment of estimated source credibility for incoming knowledgeDevelop methodologies for dynamic changes to credibility ratings

June 11-13, 2002 AQUA Question-Answering System 8

SAIC-KSL-NMSU Collaborative SystemSAIC-KSL-NMSU Collaborative System

QUESTION

NL QueryInterlingua Query

KIF Q

uery

KIF Answer

Interlingua Answer

NL Answer

ANSWER

NMSU Query Processor

SAIC Interlingua KIF Translator

KSL Java Theorem Prover

SAIC KIF Interlingua Translator

NMSU NL Generator

June 11-13, 2002 AQUA Question-Answering System 9

Converting Semantic Info to KnowledgeConverting Semantic Info to Knowledge

Ontologies (and ontological philosophies) differ between Ontolingua and TMR

SAIC dynamically aligns the two ontologies as part of the knowledge formation processMust be dynamic process since both TMR and Ontolingua ontologies are actively changing during system development

An automated mechanism for translating information in TMR into KIF relations is needed.

TMR breaks text into smallest piecesSAIC must re-unify the pieces to produce more meaningful knowledge representationsNext slide shows a typical example of this problem

June 11-13, 2002 AQUA Question-Answering System 10

Re-Unification of Knowledge Re-Unification of Knowledge

TMR includes separate references for this sentence fragment for:

united-states-soldiersoldier-human-adultunited-states-human-adult

We re-unify this into a single definition that captures that these are all the same and plural.

(defobject united-states-soldier (instance-of united-states-soldier person) (has-country united-states-soldier united-states)(member-of united-states-soldier us-army-special-

forces))(defobject group-of-united-states-soldier

(group group-of-united-states-soldier united-states-soldier)

(cardinality group-of-united-states-soldier 36))

SAMPLE TEXT: About 36 US Special Forces troops started a month of anti-terrorism training…

June 11-13, 2002 AQUA Question-Answering System 11

KIF Formulations from NarrativesKIF Formulations from Narratives

Often, useful text sources are narratives rather than unrelated compilations of factsStory comprehension must extend across sentences to the entire text of the narrativeSAIC’s experience developing knowledge representations from narratives in HPKB and RKF programs provides unique and powerful capability in this areaOur “event descriptor templates” provide relations needed to generate a series of event descriptors for a narrative

These allow us to answer highly complex questions about complicated, real-world situationsDeveloped and proven successful in HPKB ProgramNot trying to generate all possible relations from the text—only those relations that are in the event descriptors (i.e., known useful relations)

June 11-13, 2002 AQUA Question-Answering System 12

Why Event Templates?Why Event Templates?

Using event templates dramatically decreases the work load on JTP for each query

Distributes the analysis across multiple small reasoners specialized to answer specific types of questionsPre-query analysis anticipates common questions that may be asked about this document and pre-determines the answers automatically

• JTP’s set of multiple reasoners includes forward chaining, which may add other relations at load time rather than waiting for a specific query against the events

Result should be to dramatically improve answer response time for many queriesAlso provides extensibility of the system because each set of relations is handled by a separate, modular KB reasoner

• Support for other types of inputs than narratives may replace the mini-reasoners but doesn’t change the architecture

June 11-13, 2002 AQUA Question-Answering System 13

Progress to Date Progress to Date

Application of general-purpose rules that apply across a broad spectrum of instancesInitial processing of event basics in place

Identification of type of event, agent performing event, basic identification of object/agent acted on in event

• Subject-verb-object

Automatic definition of specific objects from textPlaces, people, groups

• Including cardinality of groups if available in original text

Location and time of eventDetermination (from raw text dateline, etc.) of event context location and event context timeAll events in this context are tagged relative to event-context-location and event-context-time

June 11-13, 2002 AQUA Question-Answering System 14

Automatic Knowledge Representation Status Automatic Knowledge Representation Status (cont.)(cont.)

More sophisticated event processingInterests relations

• General rules about actions supporting interests of agents performing them

• Citizenship relations

Action-object events• Dealing with verbs that are objects of actions in action-object-

is relations• Example: “pretending to do something” action-object-is

“doing something”• Representation is an event for “doing-something-event” and a

separate event for “pretending-the-doing-something-event”

Object re-unification• Recognition of previously referred to objects (from other

sentences or within longer sentences) as the same object• Pronoun dereferencing• Multiple phrasings of the same object

June 11-13, 2002 AQUA Question-Answering System 15

Status of Knowledge Representation SystemStatus of Knowledge Representation System

Preliminary demo of automatic translation from TMR to KIF is up and running

Our demonstration system is limited at the moment• Limited ontology• Limited relations• Limited lexical terms

The Demo System is improving quickly in its capabilities; in the meantime…

June 11-13, 2002 AQUA Question-Answering System 16

Demonstration SystemDemonstration System