deploying semantic technologies for digital publishing: a case study from logos bible software

Deploying Semantic Technologies for Digital

PublishingA Case Study from

Logos Bible SoftwareSean Boisen

(sean@logos.com)Slides at: http://semanticbible.org/other/presentations/2007-SemTech/

Outline

• Background: application and motivation• Scope and Overview• Technical Challenges:

– Reification for provenance data– Converting legacy data– Tools for knowledge extension

• Future directions

Who Am I?

• 19 years with BBN Technologies– Information extraction, human language

technology– Scientist, technology manager

• Semantic Web hobbyist• Senior Information Architect at Logos• One-man semantic band

The Importance of the Bible as a Semantic Domain

• The most widely distributed book – 35M Bibles and

Testaments in 2005

• The most widely translated work – > 2000 languages– 41 languages at

www.biblegateway.com

• Spans 1000s of years of ancient history

Logos Bible Software

• High-end desktop digital library– > 7000 titles– Resources in a dozen

languages– Users in 180 countries– Extensive cross-indexing

and hyper linking

• Leading publisher and developer of digital resources for Bible study

Logos Value

• Digital library with hyperlinked references and citations

• Information integration for navigation, search

• Support for original languages• Search• New content to enrich Bible study

The Bible Knowledgebase (BK)

• A machine-readable knowledgebase of semantically-organized Bible data– In OWL– Linked to Biblical texts– Search, navigation, visualization

• Relationships support discovery and exploration• Reusable content (unlike prose)• Integration framework for library resources (future)• Today: named people and places, and their

relationships• Tomorrow: chronology, events, concepts, non-

named things, key terms, topics, …

Approach

• Build on Semantic Web standards• Model the domain rather than annotate

texts• Layer knowledge: first entities, then

relationships• Be conservative in what we assert and

provide references as evidence• Try to avoid philosophy and focus on end-

user value

The Semantic Value Proposition

• Identify and disambiguate entities (beyond names)– 30 people named Zechariah– Jesus’ disciple: Peter, Simon, Simeon, Cephas …– Judah: person, tribe, territory

• Link reference information to passages for background

• Provide a rich set of relationships to encourage exploration and discovery

• Provide consistent cross-resource indexing• Leverage third-party tools• Provide scalability• Avoid reinventing the wheel

User Benefits

• Disambiguation makes search work better• Passage guide displays relevant entities to

provide background information• Relationships encourage browsing and

exploration• Visualization makes complex information

easier to grasp

Development Tools

• Ontology development and instance creation with Protégé

• Legacy data conversion and data merging through XSLT

• Storage in Sesame• Some integration code in Python for

loading and querying RDF• TBD

Most Important BK Classes

• > 60 classes in all (not counting reified relationships)

• Many upper classes are not instantiated

• General coordination of class names with SUMO– But not true re-use

BK Classes for Places

BK Abstract Classes

BK Instances

• ~100k triples• ~3000 people instances

– Aaron to Zurishaddai– Names (various

languages)

• ~20k passage references for assertions

• 90 cities, other places• Ethnicities, belief systems,

languages, social roles, organizations

Major BK Relationships

Family Relationships

Human

Human

Domain RangeProperty

Knows, collaborates, antagonist, enemy

Member ofGroup

Region

Native, resident, visited place

And inverse relationships …

(attributes)

Social role, Ethnicity, Belief

Region Subregion

Geolocation data Latitude, longitude,

etc.

Challenge: Assertions about Properties

• Provenance is important to the domain and application• Problem: how to make assertions about properties

– <#John.3, isFatherOf, #Peter>: says who?

#John.3

#Peter

isFatherOf

hasFather

#Andrew.1

hasFather

isFatherOf

Reification

• Merriam-Webster:– “to regard (something abstract) as a material

or concrete thing “

• Model the relationship between instances as an instance itself

Reified Relationships

• Solution: make the relationship an object about which we can make assertions– All “simple” properties get more complex

#John.3

#John.3_parent_Andrew.

1

isFatherOf

hasFather

#Andrew.1

isSonOf

hasSon

“bible.64.1.42”

reference

#John.3_parent_

Peter

isFatherOf

hasFather

#Peter

isSonOf

hasSon

“bible.64.21.15”

“bible.64.21.16”

“bible.64.21.17”

Some Consequences of Reification

• Class and property instance overhead– 2 simple inverse properties become 4

properties and 1 class– Abstract hierarchy of classes of reified

relationships– Add overhead as well to ontology

development, query construction, etc.

• Symmetric and transitive properties• Challenges for reasoning

– Restrictions come from a combination of properties and reified classes

Reified Classes (Family)

• All binary relationships with appropriate restrictions on their arguments (max 2, range restrictions, etc.)

Other Reified Classes

Properties Between Reified Properties

• Beyond OWL• Defined with respect to particular reified classes• Automatically derivable from the ontology

reif:isFatherOf

reif:isSonOf

reif:hasSon

reif:hasFather

owl:inverseOf

owl:inverseOf

reif:inverseOf

reif:pairedProperty

reif:onsetOf

reif:codaOf

Reified Relationships: Names

• Appellations (names) are class instances– An Appellation instance has string

representations (in various languages)– Keeps all the facts about a name (different

language versions, pronunciation, literal meaning, etc.) in one place

• An individual has a (reified) NamingRelation to an Appellation instance– Mentions of the individual are properties of the

NamingRelation

Reified Names Example

#Barnabas

isNamedBy hasAppellation

“Barnabas”@en

"Βαρναβᾶς"@el

"Bernabé"@eshasName

#Joseph2.1

bär'nə-bəs

hasPhoneticRepresentation

www.libronix.com/

bkaudio/barnabas.wav

hasPronunciation

bk:Man

rdf:type

bk:NameRel

#BarnabasnamedByBarnabas

#BarnabasnamedBy

Joseph

#Joseph2.1

namedByJoseph

bk:Appellation

#NameOfBarnabas

#NameOfJoseph

“bible.61.1.16”

reference

“bible.61.1.18 Etc.

And all the right-to-left equivalents …

Challenge: Converting Legacy Data

• Strategy: use XSL to generate RDF matching the ontology– Legacy XML data organized by name and by person– Generate reified relations from simple ones

• Lookup table for reified inverse properties (but kb query would be cleaner)

– Both sides of family relationships are defined independently

• URI Naming– Map different XML names to a single URI– Generate shared URIs for reified relations like

#<personURI>_<relation>_<personURI>– RDF merging connects them in the kb

• Why not owl:sameAs?– Additional complexity but no practical benefit for internal-

only data

Converting Legacy Data (2)

• Other OWL data with different URIs and non-reified relations– Map entities to common URIs (shared across

both legacy datasets)– Adopt same URI construction principles– Expand out reified relations– RDF merge in the kb

Legacy Data Pipeline

XSLAaron.xmlAaron.xml

Aaron.xmlAaron.xml

Biblical People XML

Aaron.xmlAaron.xml

Aaron.xmlAaron.rdf

… in RDF

Loader

NTNames (OWL)

BK ontology

XSL BK-NTNames

merge map

• Query (SeRQL, SPARQL)

• Extract

• API

• Web service

Other data (OWL)

Challenge: Maintenance and Extension

• How to lower the skill threshold for extending the data?

• Approach: – Distinguish different operations

1. Adding new instances of relationships (easy)2. Adding non-relational attributes (easy)3. Adding new instances of basic entities (a little harder)4. Fixing bad data, extending the ontology (hard)

– Get the core entities right first (enables #1)– Develop specialized tools that

• Are constrained in scope• Provide simple choices• Hide complications (like reification)

How Do We Deliver Semantics?

• Part of a consumer software application: not on the open web

• Not practical to ship an RDF store• Likely: combination of

– Some static results shipped with product– Some web service support for dynamic

information– A web portal with richer search capabilities

Open Architecture Issues

• Visualization – Likely: custom MFC

• End-user query – Likely: at most, templated queries

• Reasoning– Necessary, but …

Future Extensions to BK

• Place names and related properties• Brief descriptions for entities• Place people in Biblical eras• Narrative role (greetings in epistles, scene

participants, background)• Key events from narratives• Concepts• Unnamed things (descriptions, pronouns)• Headwords and lexical relationships

References

• Weaving the New Testament into the Semantic Web, http://semanticbible.org/other/presentations/2006-sbl/Weaving.xhtml

• Suggested Upper Merged Ontology (SUMO), http://www.ontologyportal.org/

• Defining N-ary Relations on the Semantic Web, http://www.w3.org/TR/swbp-n-aryRelations