pete cliff ukoln, university of bath, united kingdom [email protected] uwe müller
DESCRIPTION
Tutorial OAI and OAI-PMH for Beginners An introduction to the Open Archives Initiative and the Protocol for Metadata Harvesting. Pete Cliff UKOLN, University of Bath, United Kingdom [email protected] Uwe Müller Humboldt University Berlin, Germany [email protected]. Agenda. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/1.jpg)
Tutorial OAI and OAI-PMH for Beginners
An introduction to the Open Archives Initiative and the Protocol for Metadata Harvesting
Pete Cliff
UKOLN, University of Bath, United Kingdom
Uwe Müller
Humboldt University Berlin, Germany
![Page 2: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/2.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners
Agenda
Part IHistory and overview
Part IIMain Ideas of the OAI-PMH / Technical introduction
Short break Part III – Breakout Sessions
Implementation issues – data and service provider
Coffee Break Part IV
Implementation issues – XML schema and supporting multiple record formats
![Page 3: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/3.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners
Acknowledgements
Some of the slides presented here are our own! Many of them have been kindly donated by
(taken from!)Herbert Van de Sompel
Carl Lagoze
Michael Nelson
Simeon Warner
Andy Powell
(and others probably!)
![Page 4: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/4.jpg)
Tutorial OAI and OAI-PMH for Beginners
An introduction to the Open Archives Initiative and the Protocol for Metadata Harvesting
Part I: History and overview
![Page 5: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/5.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
A History Lesson - Roots of OAI
Some early activityXXX (arXiv), CogPrints, NCSTRL, RePEc
Web interfaces for peopleNo machine interfaces
Different interfaces for different archives End Users forced to learn diverse interfaces Little or no autonomous metadata sharing
![Page 6: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/6.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
Santa Fe Meeting
“…the joint impact of these and future initiatives can be substantially higher when interoperability between them [e-print archives] can be established…”[Ginsparg, Luce, Van de Sompel, UPS Call, July 1999]
![Page 7: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/7.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
The Problems
Two problems:
End users where/are faced with multiple search interfaces making resource discovery harder.
No machine based way of sharing the metadata
![Page 8: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/8.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
Cross Search?
US Digital Library Experience suggests cross searching doesn’t scale - N > 100 = bad!
Collection description - knowing which target to use
Query language and search attribute variation Rank merging problem Different size and type of target can skew results Performance - limited to slowest target Difficult to build a browse interface
SOLUTION: get all the metadata records in one place
![Page 9: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/9.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
Harvest?
Harvest records out of archives into one place Universal Preprint Service Prototype
So: N = 1 most of the time… One query language, set of search attributes and
ranking algorithm An awareness of the data makes browse
structures easier to build UPS was quickly changed to OAI - the Open
Archives Initiative
![Page 10: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/10.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
Data and Service Providers
Data ProviderCreators and keepers of the metadata and repositories of resources
Service ProviderHarvesters of metadata for the purpose of providing a service such as a search interface, peer-review system, etc.
One ‘service’ can play both roles
![Page 11: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/11.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
The Dawn of a Protocol
To facilitate metadata harvesting there needs to be agreement on:
Transport protocol - HTTP or FTP or … Metadata format - Dublin Core or MARC or … Metadata Quality Assurance - mandatory element
set, naming and subject conventions, etc. Intellectual Property and Usage Rights - who can
do what with what?
Agreement led to (fanfare): the Santa Fe Convention
![Page 12: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/12.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
The Santa Fe Convention
First incarnation of the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH)
Drew upon:The UPS Prototype
RePEc/SODA - the Service/Data provider model
the Dienst Protocol
Work of the Santa Fe group
To “optimise the discovery of e-prints”
![Page 13: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/13.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
The OAI-PMH 1.0
Introduced Dublin Core element set
Drew upon:Santa Fe Convention
Digital Library Federation meetings
Work at Cornell
Feedback from alpha-testers
A new focus to facilitate the discovery of “document-like objects”
![Page 14: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/14.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
The OAI-PMH 1.0 - Summary
Low barrier interoperability specification Based around metadata harvesting model Focus on “document-like objects” HTTP based GET / POST requests XML responses Uses unqualified Dublin Core Not a search protocol! Experimental
![Page 15: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/15.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
The OAI-PMH 1.1
A revision of the 1.0 specification taking account of changes to the emerging XML Schema specification
![Page 16: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/16.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
The OAI-PMH 2.0
Major revision - not compatible with 1.x
Drew upon:OAI-PMH 1.x
Feedback from OAI Implementers List
OAI tech deliberation
Feedback from alpha-testers
“the recurrent exchange of metadata about resources between systems”
![Page 17: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/17.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
The OAI-PMH 2.0 - Summary
Still a low barrier interoperability specification Based around metadata harvesting model Metadata about resources HTTP based GET / POST requests XML responses Uses unqualified Dublin Core Not a search protocol! Stable - OAI has committed to making subsequent
revisions of the protocol backwards compatible
![Page 18: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/18.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
about eprintsdocument
like objectsresources
metadata OAMSunqualifiedDublin Core
unqualifiedDublin Core
transport HTTP HTTP HTTP
responses XML XML XML
requests HTTP GET/POST HTTP GET/POST HTTP GET/POST
verbs Dienst OAI-PMH OAI-PMH
nature experimental experimental stable
modelmetadataharvesting
metadataharvesting
metadataharvesting
Santa Feconvention
OAI-PMHv.1.0/1.1
OAI-PMHv.2.0
![Page 19: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/19.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
Multiple data and service p’s
Data providers
Service providers
Harvestingbased onOAI-PMH
![Page 20: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/20.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
Aggregators
Data providers
Service providers
Aggregator
![Page 21: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/21.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
Can be mixed with x-searching
Data providers
Service providers
Harvestingbased onOAI-PMH
Searchingbased onZ39.50 orSRW
![Page 22: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/22.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
The Benefits of OAI-PMH
Simple Web (and so firewall) friendly Access-control, compression, error codes, etc.
based on HTTP Many toolkits - can hide the protocol from
developers Multiple SPs can harvest from multiple DPs
ensuring a wider spread of metadata A base layer to build other services on Complements search protocols like Z39.50
![Page 23: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/23.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
Summary So Far
Early movers developing separately Need for interoperability Santa Fe Meeting led to OAI OAI promotes interoperability via: OAI-PMH
Low cost
Harvest model
Data Providers / Service Providers
Simple, easy and built on existing technology
An open standard
![Page 24: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/24.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
Resources
OAI Web site:http://www.openarchives.org/
OAI-PMH specification:http://www.openarchives.org/OAI/openarchivesprotocol.html
Implementation guidelines:http://www.openarchives.org/OAI/2.0/guidelines.htm
Discussion lists:http://www.openarchives.org/mailman/listinfo/oai-general
http://oaisrv.nsdl.cornell.edu/mailman/listinfo/oai-implementers
Repository explorer:http://oai.dlib.vt.edu/cgi-bin/Explorer/oai2.0/testoai
Tools: http://oai.dlib.vt.edu/cgi-bin/Explorer/oai2.0/testoai
![Page 25: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/25.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part I
Examples of Service Providers
Citation Indexinghttp://icite.sissa.it
Search Enginehttp://www.ncstrl.org/
Printing on Demand Servicehttp://www.proprint-service.de
Value added Search Enginehttp://www.myoai.com
![Page 26: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/26.jpg)
Tutorial OAI and OAI-PMH for Beginners
An introduction to the Open Archives Initiative and the Protocol for Metadata Harvesting
Part II: Main Ideas of OAI-PMH
Technical Introduction
![Page 27: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/27.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Agenda
1. Protocol Basics
2. Protocol Details
3. Request Types
4. Examples
![Page 28: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/28.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
The Open Archives Initiative (OAI)
Main ideasworld-wide consolidation of scholarly archives
free access on the archives (at least: metadata)
consistent interfaces for archives and service provider
low barrier protocol / effortless implementation
based on existing standards (e.g. HTTP, XML, DC)
Basic functioning
Harvester Repository
Requests (based on HTTP)
Metadata (encoded in XML)
Metadata(Documents)
Metadata
Service Provider Data Provider
„Service”
![Page 29: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/29.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
OAI: General Assumptions
two groups of ‘participants’ Data Providers (Open Archives, Repositories)
free access of metadata
not necessarily: free access to full texts / resources
easy to implement, low barriers
Service Providersuse OAI interfaces of the Data Providers
harvest and store metadata (no live requests!)
may select certain subsets from Data Providers(set hierarchy, date stamp)
may enrich metadata
offer (value-added) service on the basis of the metadata
![Page 30: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/30.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
OAI-PMH: Structure Model
Se
rvic
e P
rovi
der
e-print
Da
ta
Pro
vid
er e-prints
e-print
Da
ta
Pro
vid
er Images
e-print
Da
ta
Pro
vid
er
OPAC
e-print
Da
ta
Pro
vid
er Museum
e-print
Da
ta
Pro
vid
er Archive
Requests:
Identify
ListMetadataformats
ListSets
ListIdentifiers
ListRecords
GetRecord
Responses:
General information
Metadata formats
Set structure
Record identifier
Metadata
Da
ta
Pro
vid
er Harvester
Repository
Repository
Repository
Repository
Repository
![Page 31: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/31.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
OAI-PMH: Protocol Overview
protocol based on HTTP
request arguments as GET or POST parameters
six request types
e.g. http://archive.org?verb=ListRecords&from=2002-11-01
responses are encoded in XML syntax
supports any metadata format (at least: Dublin Core)
logical set hierarchy (definition: data providers)
date stamps (last change of metadata set)
error messages
flow control
![Page 32: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/32.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Agenda
1. Protocol Basics
2. Protocol Details
3. Request Types
4. Examples
![Page 33: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/33.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Protocol Details: Definitions
Harvesterclient application issuing OAI-PMH requests
Repositorynetwork accessible server, able to process OAI-PMH requests correctly
Resourceobject the metadata is “about”, nature of resources is not defined in the OAI-PMH
Itemcomponent of an repository from which metadata about a resource can be disseminatedhas an unique identifier
Recordmetadata in a specific metadata format
Identifierunique key for an item in a repository
Setoptional construct for grouping items in a repository
![Page 34: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/34.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Protocol Details: Definitions (2)
resource
all available metadata about David
item
Dublin Coremetadata
MARCmetadata
SPECTRUMmetadata records
item = identifier
![Page 35: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/35.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Protocol Details: Records
metadata of a resource in a specific format three parts
1. header (mandatory)identifier (1)datestamp (1)setSpec elements (*)status attribute for deleted item (?)
2. metadata (mandatory)XML encoded metadata with root tag, namespacerepositories must support Dublin Core
3. about (optional)rights statementsprovenance statements
![Page 36: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/36.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Protocol Details: Datestamps
date of last modification of a metadata set mandatory characteristic of every item two possible granularities:
YYYY-MM-DD, YYYY-MM-DDThh:mm:ssZ function: information on metadata, selective
harvesting (from and until arguments) applications: incremental update mechanisms modification, creating, deletion deletion: three support levels
no, persistent, transient
![Page 37: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/37.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Protocol Details: Metadata Schema
OAI-PMH supports dissemination of multiple metadata formats from a repository
properties of metadata formatsid string to specify the format (metadataPrefix)metadata schema URL (XML schema to test validity)XML namespace URI (global identifier for metadata format)
repositories must be able to disseminate unqualified Dublin Core
arbitrary metadata formats can be defined and transported via the OAI-PMH
returned metadata must comply with XML namespace specification
![Page 38: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/38.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Protocol Details: Metadata Schema (2)
minimum standard: unqualified Dublin Corehttp://dublincore.org/
Dublin Core Metadata Element Set contains 15 elements
elements are optional
elements may be repeated
The Dublin Core Metadata Element Set:
Title Contributor Source
Creator Date Language
Subject Type Relation
Description Format Coverage
Publisher Identifier Rights
![Page 39: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/39.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Protocol Details: Sets
logical partitioning of repositories optional – archives do not have to define sets no recommendations not necessarily exhaustive not necessarily strictly hierarchical function: selective harvesting (set parameter) applications:
subject gateways, dissertation search engine, … examples (Germany, see http://www.dini.de)
publication types (thesis, article, …)document types (text, audio, image, …)content sets, according to DNB (medicine, biology, …)
![Page 40: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/40.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Protocol Details: Request Format
requests must be submitted using the GET or POST methods of HTTP
repositories must support both methods at least one key=value pair: verb=[RequestType] additional key=value pairs depend on request type example for GET request: http://archive.org/oai?
verb=ListRecords&metadataPrefix=oai_dc encoding of special characters
e.g. “:” (host port separator) becomes “%3A”
![Page 41: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/41.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Protocol Details: Response
formatted as HTTP responses content type must be text/xml status codes (distinguished from OAI-PMH errors)
e.g. 302 (redirect), 503 (service not available) compression: optional in OAI-PMH,
only identity encoding is mandatory response format: well formed XML with markup:
1. XML declaration (<?xml version="1.0" encoding="UTF-8" ?>)
2. root element named OAI-PMH with three attributes(xmlns, xmlns:xsi, xsi:schemaLocation)
3. three child elements1. responseDate (UTC datetime)2. request (request that generated this response)3. a) error (in case of an error or exception condition)
b) element with the name of the OAI-PMH request
![Page 42: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/42.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Protocol Details: Flow Control
four of the request types return a list of entries three of them may reply ‘large’ lists OAI-PMH supports partitioning decision on partitioning: repository response to a request includes
incomplete listresumption token + expiration date, size of complete list, cursor (optional)
new request with same request type resumption token as parameterall other parameters omitted!
response includesnext (maybe last) section of the listresumption token (empty if last section of list enclosed)
![Page 43: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/43.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Protocol Details: Flow Control (2)
Example
Harvester Repository
Service Provider Data Provider
“want to have all your new records”
archive.org/oai?verb=ListRecords&metadataPrefix=oai_dc&from=2003-01-01
“have 267, but give you only 100”
100 records + resumptionToken “anyID1”
“want more of this”
archive.org/oai?verb=ListRecords&resumptionToken=anyID1
“have 267, give you another 100”
100 records + resumptionToken “anyID2”
“want more of this”
archive.org/oai?verb=ListRecords&resumptionToken=anyID2
“have 267, give you my last 67”
67 records + resumptionToken “”
![Page 44: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/44.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Protocol Details: Errors and Exceptions
repositories must indicate OAI-PMH errors inclusion of one or more error elements defined error identifiers
badArgument
badResumptionToken
badVerb
cannotDisseminateFormat
idDoesNotExist
noRecordsMatch
noMetaDataFormats
noSetHierarchy
![Page 45: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/45.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Agenda
1. Protocol Basics
2. Protocol Details
3. Request Types
4. Examples
![Page 46: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/46.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Request Types
six different request types1. Identify
2. ListMetadataFormats
3. ListSets
4. ListIdentifiers
5. ListRecords
6. GetRecord
harvester has not to use all types repository must implement all types required and optional arguments depend on request types
![Page 47: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/47.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Request Type: Identify
functiondescription of an archive
example archive.org/oai-script?verb=Identify
parameters none
errors / exceptionsbadArgument
e.g. archive.org/oai-script?verb=Identify&set=biology
![Page 48: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/48.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Request Type: Identify (2)
Element Example #
repositoryName My Archive 1
baseURL http://archive.org/oai 1
protocolVersion 2.0 1
earliestDatestamp
1999-01-01 1
deleteRecords no, transient, persistent 1
granularity YYYY-MM-DD, YYYY-MM-DDThh:mm:ssZ 1
adminEmail [email protected] +
compression deflate, compress, … *
description oai-identifier, eprints, friends, … *
response format
![Page 49: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/49.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Request Type: ListMetadataFormats
functionretrieve available metadata formats from archive
example archive.org/oai-script?verb=ListMetadataFormats&
identifier=oai:HUBerlin.de:3000218parameters
identifier (optional)
errors / exceptionsbadArgumentidDoesNotExist
e.g. archive.org/oai-script?verb=ListMetadataFormats&
identifier=really-wrong-identifier noMetadataFormats
![Page 50: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/50.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Request Type: ListSets
functionretrieve set structure of a repository
example archive.org/oai-script?verb=ListSets
parameters resumptionToken (exclusive)
errors / exceptionsbadArgumentbadResumptionToken
e.g. archive.org/oai-script?verb=ListSets&resumptionToken=any-wrong-token
noSetHierarchy
![Page 51: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/51.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Request Type: ListIdentifiers
functionabbreviated form of ListRecords, retrieving only headers
example archive.org/oai-script?verb=ListIdentifiers&
metadataPrefix=oai_dc&from=2002-12-01parameters
from (optional)until (optional) metadataPrefix (required)set (optional) resumptionToken (exclusive)
errors / exceptionsbadArgument, e.g. …&from=2002-12-01-13:45:00badResumptionTokencannotDisseminateFormatnoRecordsMatchnoSetHierarchy
![Page 52: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/52.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Request Type: ListRecords
functionharvest records from a repository
example archive.org/oai-script?verb=ListRecords&
metadataPrefix=oai_dc&set=biologyparameters
from (optional)until (optional) metadataPrefix (required)set (optional) resumptionToken (exclusive)
errors / exceptionsbadArgumentbadResumptionTokencannotDisseminateFormatnoRecordsMatchnoSetHierarchy
![Page 53: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/53.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Request Type: GetRecord
functionretrieve individual metadata record from a repository
example archive.org/oai-script?verb=GetRecord&
identifier=oai:HUBerlin.de:3000218&metadataPrefix=oai_dc
parametersidentifier (required)metadataPrefix (required)
errors / exceptionsbadArgumentcannotDisseminateFormatidDoesNotExist
![Page 54: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/54.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Agenda
1. Protocol Basics
2. Protocol Details
3. Request Types
4. Examples
![Page 55: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/55.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Example: http://edoc.hu-berlin.de/OAI-2.0?verb=ListIdentifiers&from=2002-01-06&until=2002-01-08&metadataPrefix=oai_dc&set=doctypes:dissertations
<?xml version="1.0" encoding="UTF-8"?> <OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"> <responseDate>2002-10-22T17:49:49+01:00</responseDate> <request verb="ListIdentifiers" from="2002-01-03" until="2002-01-08" metadataPrefix="oai_dc" set="doctypes:dissertations">http://edoc.hu-berlin.de/OAI-2.0</request> <ListIdentifiers> <header> <identifier>oai:HUBerlin.de:3000819</identifier> <datestamp>2002-01-08</datestamp> <setSpec>doctypes</setSpec> <setSpec>doctypes:dissertations</setSpec> <setSpec>dnb</setSpec> <setSpec>dnb:dnb33</setSpec> </header> <header> <identifier>oai:HUBerlin.de:3000831</identifier> <datestamp>2002-01-07</datestamp> <setSpec>doctypes</setSpec> <setSpec>doctypes:dissertations</setSpec> <setSpec>dnb</setSpec> <setSpec>dnb:dnb27</setSpec> </header> </ListIdentifiers> </OAI-PMH>
![Page 56: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/56.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Example: http://edoc.hu-berlin.de/OAI-2.0?verb=GetRecord&identifier=oai:HUBerlin:3000819&metadataPrefix=oai_dc
<?xml version="1.0" encoding="UTF-8"?> <OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"> <responseDate>2002-11-27T14:57:01+01:00</responseDate> <request verb="GetRecord" metadataPrefix="oai_dc" identifier="oai:HUBerlin.de:3000819">http://edoc.hu-berlin.de/OAI-2.0</request> <GetRecord> <record> <header> <identifier>oai:HUBerlin.de:3000819</identifier> […] </header> <metadata> <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd"> <dc:title>Einfluß genetischer Variationen im Tumor Nekrose […]</dc:title> <dc:creator>Schüttlöffel, Antje</dc:creator> […] </metadata> </record> </GetRecord></OAI-PMH>
![Page 57: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/57.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part II
Technical Introduction: Questions?
OAI – official sitehttp://www.openarchives.org/
protocol specificationhttp://www.openarchives.org/OAI/openarchivesprotocol
.html
general mailing listhttp://www.openarchives.org/mailman/listinfo/OAI-general/
implementers mailing listhttp://www.openarchives.org/mailman/listinfo/OAI-implementers/
![Page 58: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/58.jpg)
Tutorial OAI and OAI-PMH for Beginners
An introduction to the Open Archives Initiative and the Protocol for Metadata Harvesting
Part III: Implementation Issues
Data Provider and Service Provider
![Page 59: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/59.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Agenda
1. General Considerations
2. Data Provider
3. Service Provider
![Page 60: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/60.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
General: First Questions
Data Provider Which data do I want to deliver?
Which service providers do I want to provide with data?
Service ProviderWhich Service do I want to provide?
From which data providers do I get the metadata?
In which way the metadata have to be processed?
Data Provider & Service ProviderWhich aspects do we have to agree upon?
![Page 61: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/61.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
General: Metadata Formats / Sets
required: unqualified Dublin Core special subjects / communities: other metadata
specifications may be requireddescribe resources in a specialised way
definition of an XML schema (publicly available for validation)
define set hierarchysensible partitioning for selective harvesting
agreement between data providers and between data and service providers
![Page 62: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/62.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
General: Organisational Structure
aggregated data providers if harvested by a service provider, “sub data providers” should not be harvested by same SP (duplication ...)
subject gatewaysselective harvesting if corresponding sets have been defined and implemented
![Page 63: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/63.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Agenda
1. General Considerations
2. Data Provider
3. Service Provider
![Page 64: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/64.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Data Provider: Prerequisites
metadata on resources (“items”)should be stored in (SQL) database
possible in case of need: file system …
unique identifier for each item
web server, accessible via the internete.g. apache, IIS
programming interface / APIe.g. Perl, PHP, Java-Servlet
web server extension
access to database (or filesystem)
not needed: session management
![Page 65: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/65.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Data Provider: Prerequisites (2)
archive identifier / base URL unique identifier for items metadata format (at least: unqualified Dublin
Core) datestamps for metadata (created / last modified) logical set hierarchy (may have)
agreement within (subject) communities
flow control / implementation of resumption token (optional, ‘larger’ archives should have that)
![Page 66: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/66.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Data Provider: Architecture
OAI Data Provider
OAI request(HTTP request)
Web server (e.g. Apache, IIS)
Programming extension (e.g. PHP, Perl,JavaServlets)
DB response
OAI response(XML instance)
Script / Programme- parsing arguments
- creating error messages- creating SQL statements
-creating XML output
SQL-Database
SQL request
![Page 67: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/67.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Data Provider: General Structure
Argument Parservalidates OAI requests
Error Generatorcreates XML responses with encoded error messages
Database Query / Local Metadata Extractionretrieves metadata from repository
according to the required metadata format
XML Generator / Response Creationcreates XML responses with encoded metadata information
Flow Controlrealises incomplete list sequences for ‘larger’ repositories
uses resumption token as mechanism
![Page 68: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/68.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Data Provider: Example Flow Chart
HTTP request
verb
metadataPrefix
ListIdentifie
rs G
etRecord
Identify
ListMe
tadata
-F
orma
ts
ListSets
ListRecords
elseerror: badVerb
error: cannotDiss-eminateFormat
elseemptyresumption
Token
error: badResumptionToken
unknown
read parametersfrom local system
valid parse the otherparameters
oai_dc
send SQL requestto database
rows>100
store parameters,store and deliver resumptionToken
yes
XML response
error: badArgument
empty
• verb, metadataPrefix, resump-tionToken … OAI arguments
• rows … size of the result list• 100 … here: maximal list size
for responses
deliver min (rows, 100)record headers
no
![Page 69: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/69.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Data Provider: Resumption Token
should be implemented for “large” lists initiated by data provider store parameters (set, from, …) and number of already
delivered records properties
expiration: expirationDate (optional)completeListSize (optional)already delivered records: cursor (optional)recovery from network errors (possibility to re-issue most recent resumption token)
problemdatabase changestwo possible solutions
duplicate data in a “request table”store date of first request with the other parameters use like additional until argument
![Page 70: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/70.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Data Provider: Resumption Token (2)
Example
Harvester Repository
Service Provider Data Provider
“want to have all your new records”
archive.org/oai?verb=ListRecords&metadataPrefix=oai_dc&from=2003-01-01
“have 267, but give you only 100”
100 records + resumptionToken “anyID1”
“want more of this”
archive.org/oai?verb=ListRecords&resumptionToken=anyID1
“have 267, give you another 100”
100 records + resumptionToken “anyID2”
“want more of this”
archive.org/oai?verb=ListRecords&resumptionToken=anyID2
“have 267, give you my last 67”
67 records + resumptionToken “”
![Page 71: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/71.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Data Provider: Resumption Token (3)
Repository
Data Provider
Example (2)
“want to have all your records”
archive.org/oai?verb=ListRecords&metadataPrefix=oai_dc&from=2003-01-01
Database
“have 267, but give you only 100”
100 records + resumptionToken “anyID1”
“want more of this”
archive.org/oai?verb=ListRecords&resumptionToken=anyID1
“have 268, give you another 100”
100 records + resumptionToken “anyID2”
select dc-datafrom metadata-table
1267 records
2
insert,update,delete
3
select dc-datafrom metadata-table
45
268 records
anyID1 = { from=2003-01-01, until=empty, set=empty, mdP=oai_dc, date= 2002-12-05T15:00:00Z, delivered=100}
![Page 72: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/72.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Data Provider: Data Representation
use recommended data representationdates
2002-12-052002-xx-xx, 2002, 05.12.2002
language codeeng, ger, ...en, de, english, german
multi values: use own XML element for each entityauthor
<dc:creator>Smith, Adam</dc:creator><dc:creator>Nash, John</dc:creator><dc:creator>Smith, Adam; Nash, John</dc:creator>
![Page 73: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/73.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Data Provider: Compression
method to reduce traffic and enhance performance optional for both sides: data and service providers handled on HTTP level harvesters may include an Accept-Encoding header in
their requests –specifying preferences harvesters without Accept-Encoding header always
receive uncompressed data repositories must support HTTP identity encoding repositories should specify supported encodings by
including compression elements in the identify response
![Page 74: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/74.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Data Provider: Test and Registration
create own OAI-PMH requests and send to OAI interface – check results
use the Repository Explorer (VT University)http://oai.dlib.vt.edu/cgi-bin/Explorer/oai2.0/testoai/ provide arguments via HTML formsresponses are validated ‘browsing’ to other requestsautomatic conformance tester
official registration sitehttp://www.openarchives.org/data/registerasprovider.html provide base URLextensive conformance test (incl. error conditions …)information on incorrect behaviourin case of conformance – added to the official listregular checks
![Page 75: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/75.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Agenda
1. General Considerations
2. Data Provider
3. Service Provider
![Page 76: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/76.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Service Provider: Examples
Repository Explorer: http://oai.dlib.vt.edu/cgi-bin/Explorer/oai2.0/testoai/
search engines / subject gatewaysCross Archive Searching Service: http://arc.cs.odu.edu/DINI: http://edoc.hu-berlin.de/oaisearch/ Physnet: http://physnet.uni-oldenburg.de/oai/query.phpNCSTRL: http://www.ncstrl.org
value added servicesProPrint: http://www.proprint-service.de Citation Indexing: http://icite.sissa.it:8888 MyOAI: http://www.myoai.org/
![Page 77: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/77.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Service Provider: Prerequisites
internet connected server database system (relational or XML) programming environment
can issue HTTP requests to web servers
can issue database requests
XML parser
![Page 78: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/78.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Service Provider: Structure (1)
Archive Management
selection of archives to be harvested
enter entries manually or
automatically add / remove archives using the official registry
Request Component
creates HTTP requests and sends them to OAI archives (data provider)
demands metadata using the allowed verbs of the OAI-PMH
possibly selective harvesting (set parameter)
![Page 79: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/79.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Service Provider: Structure (2)
Scheduler
realises timed and regular retrieval of the associated archives
simplest case: manual initiation of the jobs
else: e.g. cron job …
Flow Control
resumption token: partitioning of the result list into incomplete sections – anew request to retrieve more results
HTTP error 503 (service not available) – analysis of response to extract “retry-after” period
![Page 80: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/80.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Service Provider: Structure (3)
Update Mechanism
realises consolidation of metadata which have been harvested earlier (merge old and new data)
easiest case: always delete all ‘old’ metadata of an archive before harvesting it
reasonable: incremental update (from parameter) – insert new metadata and overwrite changed / deleted metadata (assignment using the unique identifiers)
XML Parser
analyses the responses received from the archives
validation: using the XML schema
transforms the metadata encoded in XML into the internal data structure
![Page 81: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/81.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Service Provider: Structure (4)
Normaliser transforms data into a homogenous structure
(different metadata formats) harmonises representation (e.g. date, author,
language code) maps / translates different languages
Database mapping the XML structure of the metadata into a
relational database (multi values …) or: use an XML database
![Page 82: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/82.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Service Provider: Structure (5)
Duplication Checker
merges identical records from different data providers
possibility: unique identifier for the item (e.g. URN, …)
but: often not easily practicable and not risk / error free
Service Module
provides the actual service to the ‘public’
basis: harvested and stored records of the associated archives
uses only local database for requests etc.
![Page 83: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/83.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Service Provider: Architecture
Data Provider Data Provider Data Provider
Scheduler
Flow controlXML Parser
Normaliser
Database
Service module
User Harvester User
OAI Service Provider
Dublication checker
Update mechanism
Administrator
![Page 84: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/84.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Service Provider: Resumption Token
optional from the data provider’s point of view but: mandatory for service providers for complete lists: resume sequences of
incomplete lists1. ‘recognise’ that response contains incomplete list
2. re-issue OAI request to data provider in order to get next part of the list
![Page 85: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/85.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part III
Service Provider: Test and Registration
harvest registered ( OAI complient!) data providers
test behaviour of service provider official registration site
http://www.openarchives.org/service/registerasprovider.html
provide institutional information
web site, email address, ...
![Page 86: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/86.jpg)
Tutorial OAI and OAI-PMH for Beginners
An introduction to the Open Archives Initiative and the Protocol for Metadata Harvesting
Part IV: Implementation issues - XML schemas and support for multiple record formats
![Page 87: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/87.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
The Basics
OAI-PMH uses XML Schemas Any XML with an XML Schema = OK for OAI! OAI-PMH mandates ‘oai_dc’ schema OAI-PMH documentation includes schema for
RFC1807 metadata
MARC21 metadata (Library of Congress)
oai_marc metadata
![Page 88: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/88.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
oai_dc
Simple unqualified DC schema Mandatory ‘Lowest Common Denominator’ Container schema is OAI specific Container schema hosted @ OAI Web site Imports a generic DCMES schema DCMES schema @ DCMI Web site
![Page 89: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/89.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
oai_dc - a record
<?xml version="1.0" encoding="UTF-8"?>
<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/
http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
<responseDate>2003-03-15T16:16:51+01:00</responseDate>
<request verb="GetRecord" metadataPrefix="oai_dc" identifier="oai:HUBerlin.de:3000476">http://edoc.hu-berlin.de/OAI-2.0</request>
<GetRecord>
<record>
<header>
<identifier>oai:HUBerlin.de:3000476</identifier>
<datestamp>1997-07-18</datestamp>
<setSpec>pub-type</setSpec>
</header>
<metadata>
<oai_dc:dc
xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/
http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
<dc:title>Melanchthon in seiner Zeit. In: Philipp Melanchthon 1497-1997</dc:title>
<dc:creator>Selge, Kurt-Victor</dc:creator>
...
![Page 90: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/90.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
oai_dc - a record
three important things to notice:
namespace for the oai_dc formatxmlns:oai_dc=http://www.openarchives.org/OAI/2.0/oai_dc/
namespace for DCMES elementsxmlns:dc=http://purl.org/dc/elements/1.1/
container schema associated with the oai_dc namespace
xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd"
![Page 91: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/91.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
The XML Schemas
The oai_dc “container schema”
Imports DCMES schema
Defines a container element - ‘dc’
Lists the allowed elements within the ‘dc’ container (defined in DCMES Schema)
![Page 92: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/92.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
Other metadata formats
oai_dc is a simple format providing baseline interoperability
It may not be suitable:Not enough (or the required) elements!
Not very precise - it is an “unqualified” MES
(not covered in this talk... Sorry!)
Not the metadata format you need ie. not:
IMS/IEEE LOM - eLearning metadata
ODRL - Open Digital Rights Language
![Page 93: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/93.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
oai_dc is... not enough
Extend the Schema by adding new elements:
Create a name for new schema Create namespaces Create the schema for the new elements Create ‘container schema’ Validate your schema / records Add to repository’s “ListMetadataFormats” Add to repository’s other verbs Test it worked and is valid
![Page 94: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/94.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
oai_dc is... not enough
Simple Scenario: I have test repository containing some photos:
http://homes.ukoln.ac.uk/~lispdc/oaitutorial/petesphotos/oai/
Currently using oai_dc I want to add an “Equipment Used” element (not
part of the DCMES)
![Page 95: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/95.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
Step 1: Name your format
I’m choosing “pp_dc” - following the “oai_dc” convention
Could be anything you like...
![Page 96: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/96.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
Step 2: Create Namespaces
We need two namespaces:Namespace for the new format (pp_dc) that mixes both standard DC elements and any new ones
Namespace for the new (pp_dc) elements
Namespaces are declared as URIs DCMI usage recommends use of Purl, but this is
not required We will use:
http://homes.ukoln.ac.uk/oaitutorial/petesphotos/pp_dc/
http://purl.org/petec/ppterms
![Page 97: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/97.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
Step 3: New Terms Schema
Create an XML Schema for the new termshttp://homes.ukoln.ac.uk/~lispdc/oaitutorial/petesphotos/pp_dc/20030317/ppterms.xsd
(Notice the datestamp - makes it easier to enhance the schema without breaking things using the old one)
Defines the new element “equipmentUsed” Defines a new container type
ppterms:elementContainer
![Page 98: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/98.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
Step 4: Container Schema
Create an XML Schema for pp_dc record formathttp://homes.ukoln.ac.uk/~lispdc/oaitutorial/petesphotos/pp_dc/20030317/pp_dc.xsd
(Another date stamp!)
Imports the pp_terms Schema Defines a container element ‘ppdc’ of type
ppterms:elementContainer
![Page 99: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/99.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
Step 5: Validate
Create some test records (or modify your existing ones)
Validate the records and schema withhttp://www.w3.org/2001/03/webdata/xsv/
![Page 100: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/100.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
Step 6: ListMetadataFormats
OAI-PMH verb ListMetadataFormats Needs an awareness of the new format so: Need to modify your repository software (source
code and/or configuration files) to support the new metadata format
…
<metadataFormat>
<metadataPrefix>pp_dc</metadataPrefix>
<schema>http://homes.ukoln.ac.uk/~lispdc/oaitutorial/petesphotos/pp_dc/20030316/pp_dc.xsd
</schema>
<metadataNamespace> http://homes.ukoln.ac.uk/~lispdc/oaitutorial/petesphotos/pp_dc/
</metadataNamespace>
</metadataFormat>
…
![Page 101: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/101.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
Step 7: Other Verbs
Also need to ensure pp_dc is available via:ListSets
ListIdentifiers
ListRecords
GetRecord
requests Accept metadata prefix “pp_dc” Return the appropriate records
![Page 102: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/102.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
Step 8: Testing
Use the Repository Explorer to test new format Ensure:
All requests work with the new ‘metadataPrefix’
oai_dc still works
appropriate records are returned
responses validate correctly
Congratulations - you’ve got a new format!
![Page 103: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/103.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
Summary - Extending a format
Decide a name and some namespaces Develop XML schema for the container and the
new elements Create test records and validate Modify repository (source code and/or
configuration files) to support new format Test and validate new repository output
![Page 104: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/104.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
oai_dc... is not the MES I’m looking for
Implement a different format eg. IMS/IEEE LOM Very similar steps Already agreed names, XML schema and
namespaces Should, therefore, be easier!
![Page 105: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/105.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
Implementing an existing format
Modify the “ListMetadataFormats” response to include (eg. for IMS):
...
<metadataFormat>
<metadataPrefix>ims</metadataPrefix>
<schema>http://www.imsglobal.org/xsd/imsmd_v1p2p2.xsd</schema>
<metadataNamespace>
http://www.imsglobal.org/xsd/imsmd_v1p2
</metadataNamespace>
</metadataFormat>
...
Extend other verbs to deal with ‘ims’ metadataPrefix
![Page 106: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/106.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners - Part IV
Summary
OAI-PMH allows for any MES so long as... ...it is encoded in XML with an XML Schema All repositories must support oai_dc for... ...minimum level of interoperability If oai_dc is not enough - extend it! If oai_dc is not precise - wait a bit! If oai_dc is not ‘the one’ - use something else as
well!
![Page 107: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/107.jpg)
Tutorial OAI and OAI-PMH for Beginners
An introduction to the Open Archives Initiative and the Protocol for Metadata Harvesting
![Page 108: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/108.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners
Summary
during today’s tutorial we hope that you havegained an overview of the history behind the OAI-PMH and an overview of its key features
been given a deeper technical insight into how the protocol works
learned something about some of the main implementation issues
found some useful starting points and hints that will help you as implementors
![Page 109: Pete Cliff UKOLN, University of Bath, United Kingdom p.d.cliff@ukoln.ac.uk Uwe Müller](https://reader030.vdocuments.site/reader030/viewer/2022032606/56812d15550346895d91fbd8/html5/thumbnails/109.jpg)
3rd OAForum workshop - Berlin - 27th-29th March 2003 - Tutorial: OAI and OAI-PMH for Beginners
Questions
now… feel free to tell us what you didn’t understand and ask general questions (of course!)
Pete CliffUKOLN, University of Bath, United Kingdom
Uwe Müller
Humboldt University Berlin, Germany