the ockham / nsdl digital library services registry

emory ▪ notre dame ▪ oregon state ▪ virginia tech

The OCKHAM / NSDLDigital Library Services Registry

A Distributed Approach

To Enable End-to-End

Digital Service Resolution

emory ▪ notre dame ▪ oregon state ▪ virginia tech

emory ▪ notre dame ▪ oregon state ▪ virginia tech

•Funded by the National Science Foundation

•National Science Digital Library Program

•2 Year Project Funding

emory ▪ notre dame ▪ oregon state ▪ virginia tech

Goals1. Create a Registry for all possible Digital Library Services2. Enable End-to-End Digital Library Service Resolving

Sub-Goals

1. Ensure DLSR is Scalable and Redundant2. Ensure Manageability of DLSR is Scalable 3. Use Existing Standards and Technologies

emory ▪ notre dame ▪ oregon state ▪ virginia tech

Use of the DLSR

1. Library Portal Use Case2. Metasearch Use Case3. Personal Digital Library

3 Examples

emory ▪ notre dame ▪ oregon state ▪ virginia tech

Distributed DLSR

•The OCKHAM/NSDL DLSR is Distributed•Many nodes over the network•Scalability•Redundancy

•Approach in part based on DNS model

emory ▪ notre dame ▪ oregon state ▪ virginia tech

A Brief History of DNS

•Hosts.txt file (later just hosts)•ARPAnet started with centralized management

•Inter-NIC•Eventually, a more manageable approach was needed•Current Distributed DNS System was created

•Allows De-centralized administration•Hierarchical design

•Simplifies management•Reduces bandwith, bottlenecks•Reduces duplicate name issue (i.e. .edu, .com, etc.)

emory ▪ notre dame ▪ oregon state ▪ virginia tech

Distributed DLSR

•Reasoning much the same as DNS•Similarities and Differences

emory ▪ notre dame ▪ oregon state ▪ virginia tech

emory ▪ notre dame ▪ oregon state ▪ virginia tech

emory ▪ notre dame ▪ oregon state ▪ virginia tech

Data LayerRelational Db & OJB• Synchronized• Database

Interchangeability• Ease of development

Lucene• Fast indexing• “on the fly” indexing• Flexible query engine

emory ▪ notre dame ▪ oregon state ▪ virginia tech

Interface LayerCurrent Interfaces• Struts powered J2ee

interface • OAICat - OAI-PMH 2.0

Future Interfaces• SRU/W • Z39.50• OpenURL

Output Formats• Html• XML

emory ▪ notre dame ▪ oregon state ▪ virginia tech

Clients• Web browser• OAI-PMH • Z39.50• Any web enabled

application.

emory ▪ notre dame ▪ oregon state ▪ virginia tech

Network Layer• JXTA provides low level

network functionality– Peer identification and

discovery– Transport layer

• PeerManager provides modular application level functionality

– Load Management– Client/Server Modules

• Main Registry Modules– URL Server– Harvester Client

emory ▪ notre dame ▪ oregon state ▪ virginia tech

Using OAI For Propagating Data

• A peer can query any other peer and receive an incremental update

• Queries are based on the latest record modification date for the peer’s local copy of the set

• Subsequent queries will use the new latest record modification date

emory ▪ notre dame ▪ oregon state ▪ virginia tech

Hierarchal Network Topology

Topology Requirements• Data must propagate to all

peers• Compensation for inherent

instability of P2P networks• Scalable structure• Low overhead• DNS for Digital Library

Services

emory ▪ notre dame ▪ oregon state ▪ virginia tech

Put the pieces together

emory ▪ notre dame ▪ oregon state ▪ virginia tech

OCKHAM Future

•Expand the DLSR Community and Use•Explore and build DLSR-aware tools and services•Prototype semi-automated creation of DL’s

emory ▪ notre dame ▪ oregon state ▪ virginia tech

Further Information

OCKHAM Website – http://ockham.org

Martin Halbert, Emory – mhalber@emory.edu

Jeremy Frumkin, Oregon State University – jeremy.frumkin@oregonstate.edu