1

DAML Tools for Intelligent Information Annotation, Sharing and Retrieval

UMBC

Johns Hopkins University Applied Physics LabMIT Sloan School

Tim FininJames MayfieldBenjamin Grosof

DAML PI meeting, 30 November 2004

2/16 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval Overall Program Summary

OWL enables agents in open, dynamic environments to share knowledge and cooperate on tasks while managing privacy, commitments, trust, and security. To do this, we must ensure that:

– OWL integrates with common agent frameworks and standards– OWL integrates with other KR paradigms for real world reasoning -- rule based

systems, Bayesian reasoning, etc.– OWL permits IR systems to index and retrieve documents with text, multimedia

and knowledge

Travel Agents

Auction Service Agent

Customer Agent

Bulletin BoardAgent

Market Oversight Agent

Request

Direct Buy

Report Direct Buy Transactions

BidBid

CFP

Report Auction Transactions

Report Travel Package

Report Contract

Proposal

Web Service Agents

3/16 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval Technical Problem and Approach

• Current agent systems are not set up to use OWL– Develop ontologies, conventions, software and tools to support using OWL in

FIPA and similar systems

– Demonstrate and investigate the use of OWL & Agents in realistic applications • Policies need to be grounded in OWL content

– Develop Rei as a declarative, RDF based policy language– Demonstrate via several testbeds

• OWL needs to be extended/integrated with rule-based reasoning– Work with groups to develop specifications for RuleML, SWRL, etc– Build prototype translation systems and demonstrations

• OWL needs to be extended/integrated with uncertainty reasoning– Annotate OWL content with Bayesian info and generate BBN– Demonstrate via support for ontology mapping

• We need information retrieval systems that work with RDF content– Augment existing IR systems like Google with swangle terms– Develop Swoogle as a native RDF search engine

4/16 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval Technical Progress

Here are the comments on the template slide• What technical problems were there and when/how did you overcome them?• Are there any metrics that are relevant to your program? How did you measure technical

progress and success? What were your intermediate goals?• Did you meet your original or revised programmatic goals?

• We need to come with a slide for this

5/16 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval Milestones and Accomplishments

Agent &Systems

KR

IR

specs

2001 2002 2003 2004 2005

ITTALKS TAGA

Vigil Soupa Cobra

SweetJess SweetRules

Rei

F-OWL reasoner

OWLIR Swoogle

Swangler

Bayes OWL

DAML+OIL Joint Committee

W3C WebOnt Working group SWSI & SWSA

Papers/MS/PhD

5+/7?/2?24/4/525/4/120/1/04/0/0

RGB

s

s

s

ss

s

s

s

s

s

Legend

Softwareor service

Ontology

Ph.D. Dissertation

Spec impact

Distributed IDS

Mogatu

O

OO O

O

O

O

RuleML

O

O

O

6

DAML Tools for Intelligent Information DAML Tools for Intelligent Information Annotation, Sharing and RetrievalAnnotation, Sharing and Retrieval

http://daml.org/ UMBC, JHU/APL, and MIT/Sloan are working together on a set of issues to be integrated into agent-based applications involving search and using rule-based reasoning: UMBC: Integrating communicating agents, DAML and the Web; JHU APL: DAML and information retrieval; and MIT Sloan School: DAML, rules based technology and distributed belief

Features•Generation from DB to DAML and HTML mediated by MySQL, Java servlets, and JSP.

•Generation of DAML descriptions and user profiles from HTML forms.

•Creation & use of DAML-encoded user modelsdescribing interests and ontology extensions.

•Ontologies for events, people, places, schedules,topics, etc.

•Automatic HTML form (pre) filling from DAML.

•Syncing of talks with Palm calendars via Coola.

•Automatic classification of talks into topicontology

•A XSB-based DAML/RDF reasoning engine.

•Agent-based services:• ITTALKS agent with KQML API using DAMLas content language

• Intelligent matching of people and talks based on interests, locations and schedules.

• Agents using both Jackal and FIPA’s Java Message Service

• Notification via email and mobile devices via SMS and WML.

• Discovery of relevant background papers from NEC CiteSeer

• Automatic generation and maintenance of user models

• Talk recommendations via collaborative filtering• Integration with STP (Smart Things and Places) ubiquitous computing project

UMBCAN HONORS UNIVERSITY IN MARYLAND

Webserver + Javaservlets

DAMLreasoning

engine

DAMLreasoning

engine

<daml></daml>

<daml></daml>

<daml></daml>

<daml></daml>

DAML files

Agents

Databases

People

RDBMSRDBMSDB

Email, HTML, SMS, WAP

FIPA ACL, KQML, DAML

SQLHTTP, KQML, DAML, Prolog

MapBlast, CiteSeer, Google, …HTTP

HTTP, WebScraping

WebServices

Ittalks.org a database driven web site of IT related talks at UMBC and other institutions. The database contains information on

– Seminar events– People (speakers, hosts, users, …)– Places (rooms, institutions, …)

This database is used to dynamically generate web pages and DAML descriptions for the talks and related information and serves as a focal point for agent-based services relating to these talks. To add your organization to ittalks.org and receive a domain (e.g., mit.ittalks.org) contact info@ittalks.org. See http://daml.org/ for more information on DAML and the semantic web.

Acknowledgements: Funded by DARPA, contract F30602-00-2-0591. Students: H. Chen, F. Perich, L. Kagal, S. Tolia, Y. Zou, J. Sachs, S. Kumar and M. Gandhe. Faculty: T. Finin, A. Joshi, Y. Peng, R. Cost, C. Nicholas, R. Masuoka

http://ittalks.org/

7

Travel Agent Game in Agentcities

http://taga.umbc.edu/

TechnologiesTechnologiesFIPA (JADE, April Agent Platform)

Semantic Web (RDF, OWL)

Web (SOAP,WSDL,DAML-S)

Internet (Java Web Start )

FeaturesFeaturesOpen Market FrameworkAuction ServicesOWL message contentOWL OntologiesGlobal Agent Community

Acknowledgements: DARPA contract F30602-00-2-0591 and Fujitsu Laboratories of America.Students: Y. Zou, L. Ding, H. Chen, R. Pan. Faculty: T. Finin, Y. Peng, A. Joshi, R. Cost. 8/03

MotivationMotivationMarket dynamicsAuction theory (TAC)Semantic webAgent collaboration (FIPA & Agentcities)

Travel Agents

Auction Service Agent

Customer Agent

Bulletin BoardAgent

Market Oversight Agent

Request

Direct Buy

Report Direct Buy Transactions

BidBid

CFP

Report Auction Transactions

Report Travel Package

Report Contract

Proposal

Web Service Agents

OntologiesOntologieshttp://taga.umbc.edu/ontologies/

travel.owl – travel concepts

fipaowl.owl – FIPA content lang.

auction.owl – auction services

tagaql.owl – query language

FIPA platform infrastructure services, including directory facilitators enhanced to use DAML-S for service discovery

8Contributors include Tim Finin, Anupam Joshi, Yun Peng, R. Scott Cost, Jim Mayfield, Joel Sachs, Pavan Reddivari, Vishal Doshi, Rong Pan, Li Ding, and Drew Ogle. Partial research support was provided by DARPA contract F30602-00-0591 and by NSF by awards NSF-ITR-IIS-0326460 and NSF-ITR-IDM-0219649. November 2004.

http://swoogle.umbc.edu/

Swoogle uses four kinds of crawlers to discover semantic web documents and several analysis agents to compute metadata and relations among documents and ontologies. Metadata is stored in a relational DBMS. Services are provided to people and agents.

SWD RankA SWD’s rank is a function of its type (SWO/SWI) and the rank and types of the documents to which it’s related.

SWOs

SWIs

HTMLdocuments

Images

CGI scripts

Audiofiles

Videofiles

The web, like Gaul, is divided into three parts: the regular web (e.g. HTML), Seman- tic Web Ontologies (SWOs), and Semantic Web Instance files (SWIs)

SWD = SWO + SWI

Swoogle Statistics

Swoogle puts documents into a character n-gram based IR engine to compute document similarity and do retrieval from queries

SWD IR Engine

SWOOGLE 2

SWD Metadata

Web Service

Web Server

SWD Cache

The Web

The WebCandidate

URLs Web Crawler

SWD Reader

IR analyzer SWD analyzer

Human users

Intelligent Agents

discovery

digest

analysis

service

OntologyDictionary

SwoogleSearch

SwoogleStatisticsOntology Dictionary

Swoogle Search

Swoogle provides services to people via a web interface and to agents as web services.

SWDs 310,000 Classes 95,000

Triples 45,000,000 Properties 53,000

Ontologies 4,200 Individuals 6,800,000

Statistics as of November 2004

9

OWL Search usingOWL

document SwangleTool

JENA

OWLdocument

+Swangled

triples

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

SwangleSearchQuery Triple

OWLdocument

+Swangled

triples

OWL Triples

<j.0:SwangledTriple> <rdfs:comment>Swangled text for [<http://www.xfront.com/owl/ontologies/camera/#Digital>, <http://www.w3.org/2000/01/rdf-schema#subClassOf>, <http://www.xfront.com/owl/ontologies/camera/#PurchaseableItem> ] </rdfs:comment> <j.0:swangledText>BE52HVKU5GD5DHRA7JYEKRBFVQ</j.0:swangledText> <j.0:swangledText>WS4KYRWMO3OR3A6TUAR7IIIDWA</j.0:swangledText> <j.0:swangledText>2THFC7GHXLRMISEOZV4VEM7XEQ</j.0:swangledText> <j.0:swangledText>HO2H3FOPAEM53AQIZ6YVPFQ2XI</j.0:swangledText> <j.0:swangledText>6P3WFGOWYL2DJZFTSY4NYUTI7I</j.0:swangledText> <j.0:swangledText>N656WNTZ36KQ5PX6RFUGVKQ63A</j.0:swangledText> <j.0:swangledText>IIVQRXOAYRH6GGRZDFXKEEB4PY</j.0:swangledText></j.0:SwangledTriple>

WebSearchEngine

FiltersSemanticMarkup

InferenceEngine

LocalKB

SemanticMarkup

SemanticMarkup

Extractor

Encoder

RankedPages

SemanticWeb Query

EncodedMarkup

TextQuery

TextFiltersText

[<http://www.xfront.com/owl/ontologies/camera/#Digital>, <http://www.w3.org/2000/01/rdf-schema#subClassOf>, <http://www.xfront.com/owl/ontologies/camera/#PurchaseableItem>]

Inference

[<http://www.xfront.com/owl/ontologies/camera/#Digital>, <http://www.w3.org/2000/01/rdf-schema#subClassOf>, <http://www.xfront.com/owl/ontologies/camera/#PurchaseableItem>] [<http://www.xfront.com/owl/ontologies/camera/#Digital>, <http://www.w3.org/2002/07/owl#Thing>, <http://www.xfront.com/owl/ontologies/camera/#PurchaseableItem>] [<http://www.xfront.com/owl/ontologies/camera/#Digital> ,

< http://www.w3.org/2002/07/owl#Thing >, < http://www.w3.org/2002/07/owl#Thing >]

Swangler

Vision

Indexing and Search

Swangler

10Student: Z. Ding. Faculty: Dr. Y. Peng, Dr. T. Finin, Dr. A. Joshi. Ack: DARPA Contract F30602-00-2-0591. 09/03.

An example of translated Bayesian network ontologyAn example of translated Bayesian network ontology

Unified Ontology Support using Bayesian NetworksUnified Ontology Support using Bayesian Networks

MotivationMotivationReasoning within single ontologies is uncertain due to noisy or incomplete dataDL based reasoning is overgeneralized and inadequateRelations between concepts in different ontologies are inherently uncertainOWL is the new standard proposed by W3C for ontology representation

Translation ProcessTranslation ProcessExtending OWL for probability annotationDefining a set of structural translation rules to translate an OWL RDF graph to the directed acyclic graph (DAG) of Bayesian network (BN)Constructing conditional probability tables (CPTs) for individual variables in the DAG of BN

ApproachApproachTranslating OWL ontologies to Bayesian networks (BNs)Concept mapping as conditional probabilitiesJoining translated BNs (by probabilistic mappings) dynamicallySupport ontology reasoning (within and across ontologies) as Bayesian inference

Context Broker Architecture

CoBrA FeaturesCoBrA Features[1] Use Semantic Web languages for context modeling and reasoning

[2] Use logic inference to detect and resolve inconsistent context knowledge

[3] Extend the REI policy language for privacy protection

[4] Adopt the FIPA standards for communication & knowledge sharing

EasyMeeting an intelligent meeting room prototype that provides services for speakers, audience & organizers based on their situational needs.

http://cobra.umbc.edu

12/16 UMBC/JHU/MIT 11/19/2004

• A slide on SweetRules, SWRL and RuleML

Intelligent Information Annotation, Sharing and Retrieval Adding Rules to OWL

13/16 UMBC/JHU/MIT 11/19/2004

RGB: Research Group in a Boxeating our own dog food

• The UMBC ebiquity lab’s portal exposesits content in RDF using a set of OWL ontologies for papers, people, projects, photos, etc.

• Lab members can add arbitrary RDF assertions about any of the portal’s objects.

• It’s designed as a modular, configurable package using O/S software that others can use to create their own portals.

14/16 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval Transition/Handoff

• Specifications– Participated in WEBONT, SWSA, SWSI, RuleML

• Software products– Damlator, Swangler, F-OWL, SweetRules, Rei

• Major demonstration systems– ITTALKS, TAGA, ebiquity site

• Services– Swoogle

• Papers– ~100 papers in journals, books, conferences and workshops, ~10 MS

theses, 5 PhD dissertations

• Users– Swoogle has many users, Rei used by ~4 other groups, … more

needed here

15/16 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval Remaining Issues

Issue Remediation

SweetRules plan not completed

Complete design and implementation…

Rei policy langu-age needs rules

Incorporate SWRL/RuleML languages. Can we compile Rei policies to OWL+SWRL?

Swoogle needs instance data

Extend Swoogle’s database to include all instance data. But, how well will it scale?

Swangler lacks query I/F

Build a query interface for swangled Semantic Web documents.

IR over mixed RDF and text

Develop tools to enable Swangler and Swoogle to work with documents that embed RDF in XHTML.

Bayes OWL demonstration

Application to ontology mapping will require additional basic research. Can probabilities be estimated from Swoogle data?

SweetRules toolkit

Populate the SweetRules toolkit with open sourced implementations

16/16 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval Summary

We’ve demonstrated that• OWL enables agents in open, dynamic environments to

share knowledge and manage privacy, commitments, trust, and security.– Demonstration: ITTALKS system, TAGA multiagent system, SOUPA

ontology for pervasive computing

• OWL can be integrated with other KR paradigms for real world reasoning– Demonstration: SweetJess, SweetRules, Bayes OWL, Rei

• OWL is compatible with the information retrieval paradigm– Demonstration: Swoogle as an RDF search engine, Swangling

permits IR systems to index and retrieve documents with text, multimedia, and knowledge.

• OWL facilitates the sharing of knowledge thru web portals– Demonstration: ebiquity web pages

17/16 UMBC/JHU/MIT 11/19/2004

Intelligent Information Annotation, Sharing and Retrieval Backup