new developments in oai michael l. nelson old dominion university mln/ [email protected] oa-forum may...
Post on 21-Dec-2015
213 views
TRANSCRIPT
![Page 1: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/1.jpg)
New Developments in OAI
Michael L. NelsonOld Dominion University
http://www.cs.odu.edu/~mln/
OA-Forum
May 13-14, 2002
Pisa, Italy Many slides borrowed from Herbert Van de Sompel & Carl Lagoze
![Page 2: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/2.jpg)
N.B.
• OAI-PMH 2.0 is not scheduled for public beta release until May 19, 2002– some of the details of this presentation are still
subject to change!– final public release of 2.0 scheduled for June 1
![Page 3: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/3.jpg)
What’s New in 2.0?!• Good news: OAI-PMH is still
Six Verbs + DC
• Incremental improvements– single XML schema– ambiguities removed– more expressive options– cleaner separation of roles & responsibilities
• Bad news: not backwards compatible with 1.1
![Page 4: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/4.jpg)
Open Archives Initiative
The protocol is openlydocumented, and metadatais “exposed” to at least somepeer group (note: rights management can still apply!)
Archive defined as a“collection of stuff” --not the archivist’s definition of “archive”. “Repository” used in most OAI documents.
OAI is happeningat break-neck speed...
![Page 5: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/5.jpg)
The Rise and Fall of Distributed Searching
• wholesale distributed searching, popular at the time, is attractive in theory but troublesome in practice– Davis & Lagoze, JASIS 51(3), pp. 273-80– Powell & French, Proc 5th ACM DL, pp. 264-265
• distributed searching of N nodes still viable, but only for small values of N
• NCSTRL: N > 100; bad• NTRS/NIX: N<=20; ok (but could be better)
![Page 6: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/6.jpg)
The Rise and Fall of Distributed Searching
• Other problems of distributed searching (from STARTS)
– source-metadata problem• how do you know which nodes to search?
– query-language problem• syntax varies and drifts over time between the various nodes
– rank-merging problem• how do you meaningfully merge multiple result sets?
• Temptations:– centralize all functions
• “everything will be done at X”
– standardize on a single product• “everyone will use system Y”
![Page 7: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/7.jpg)
Metadata Harvesting• Move away from distributed searching• Extract metadata from various sources• Build services on local copies of metadata
– data remains at remote repositories
user
. . .
search for “cfd applications”
local copy ofmetadata
metadataharvested offline
metadataharvested offline
metadataharvested offline
metadataharvested offline
each node independently maintained
all searching, browsing, etc. performed on the metadata hereindividual nodes can
still support direct userinteraction
![Page 8: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/8.jpg)
about eprintsdocument
like objectsresources
metadata OAMSunqualifiedDublin Core
unqualifiedDublin Core
transport HTTP HTTP HTTP
responses XML XML XML
requests HTTP GET/POST HTTP GET/POST HTTP GET/POST
verbs Dienst OAI-PMH OAI-PMH
nature experimental experimental stable
modelmetadataharvesting
metadataharvesting
metadataharvesting
Santa Feconvention
OAI-PMHv.1.0/1.1
OAI-PMHv.2.0
![Page 9: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/9.jpg)
Santa Fe Convention [02/2000]
• goal: optimize discovery of e-prints
• input:
• the UPS prototype
• RePEc /SODA “data provider / service provider model”
• Dienst protocol
• deliberations at Santa Fe meeting [10/99]
![Page 10: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/10.jpg)
OAI-PMH v.1.0 [01/2001]
• goal: optimize discovery of document-
like objects
• input:• SFC• DLF meetings on metadata harvesting• deliberations at Cornell meeting [09/00]• alpha test group of OAI-PMH v.1.0
![Page 11: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/11.jpg)
• low-barrier interoperability specification
• metadata harvesting model: data provider / service provider
• focus on document-like objects
• autonomous protocol
• HTTP based
• XML responses
• unqualified Dublin Core
• experimental: 12-18 months
OAI-PMH v.1.0 [01/2001]
![Page 12: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/12.jpg)
pre- 2.0 OAI Timeline Highlights
• October 21-22, 1999 - initial UPS meeting• February 15, 2000 - Santa Fe Convention published in D-Lib Magazine
– precursor to the OAI metadata harvesting protocol• June 3, 2000 - workshop at ACM DL 2000 (Texas)• August 25, 2000 - OAI steering committee formed, DLF/CNI support• September 7-8, 2000 - technical meeting at Cornell University
– defined the core of the current OAI metadata harvesting protocol• September 21, 2000 - workshop at ECDL 2000 (Portugal)• November 1, 2000 - Alpha test group announced (~15 organizations)• January 23, 2001 - OAI protocol 1.0 announced, OAI Open Day in the U.S. (Washington DC)
– purpose: freeze protocol for 12-16 months, generate critical mass• February 26, 2001 - OAI Open Day in Europe (Berlin)• July 3, 2001 - OAI protocol 1.1 announced
– to reflect changes in the W3C’s XML latest schema recommendation• September 8, 2001 - workshop at ECDL 2001 (Darmstadt)
![Page 13: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/13.jpg)
OAI-PMH v.2.0 [06/2002]
• goal: recurrent exchange of metadata about
resources between systems
• input:• OAI-PMH v.1.0• feedback on OAI-implementers• deliberations by OAI-tech [09/01 -]
• alpha test group of OAI-PMH v.2.0 [03/02 -]
![Page 14: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/14.jpg)
• low-barrier interoperability specification
• metadata harvesting model: data provider / service provider
• metadata about resources
• autonomous protocol
• HTTP based
• XML responses
• unqualified Dublin Core
• stable
OAI-PMH v.2.0 [06/2002]
![Page 15: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/15.jpg)
process leading to OAI-PMH v.2.0
pre-alpha phase
alpha-phase
creation of OAI-tech
beta-phase
![Page 16: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/16.jpg)
• created for 1 year period
• charge:
• review functionality and nature of OAI-PMH v.1.0
• investigate extensions
• release stable version of OAI-PMH by 05/02
• determine need for infrastructure to support broad adoption of the protocol
• communication: listserv, SourceForge, conference calls
creation of OAI-tech [06/01]
![Page 17: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/17.jpg)
US representatives
Thomas Krichel (Long Island U) - Jeff Young (OCLC) - Tim Cole - (U of Illinois at Urbana Champaign) - Hussein Suleman (Virginia Tech) - Simeon Warner (Cornell U) - Michael Nelson (NASA) - Caroline Arms (LoC) - Mohammad Zubair (Old Dominion U) - Steven Bird (U Penn.)
European representatives
Andy Powell (Bath U. & UKOLN) - Mogens Sandfaer (DTV) - Thomas Baron (CERN) - Les Carr (U of Southampton)
OAI-tech
![Page 18: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/18.jpg)
• review process by OAI-tech:
• identification of issues
• conference call to filter/combine issues
• white paper per issue
• on-line discussion per white paper
• proposal for resolution of issue by OAI-exec
• discussion of proposal & closure of issue
• conference call to resolve open issues
pre-alpha phase [09/01 – 02/02]
![Page 19: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/19.jpg)
• creation of revised protocol document
• in-person meeting Lagoze - Van de Sompel - Nelson – Warner
• autonomous decisions
• internal vetting of protocol document
pre-alpha phase [02/02]
![Page 20: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/20.jpg)
• alpha-1 release to OAI-tech March 1st
2002
• OAI-tech extended with alpha testers
• discussions/implementations by OAI-tech
• ongoing revision of protocol document
alpha phase [02/02 – 05/02]
![Page 21: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/21.jpg)
• The British Library • Cornell U. -- NSDL project & e-print arXiv • Ex Libris • FS Consulting Inc -- harvester for my.OAI • Humboldt-Universität zu Berlin • InQuirion Pty Ltd, RMIT University • Library of Congress • NASA • OCLC
OAI-PMH 2.0 alpha testers (1/2)
![Page 22: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/22.jpg)
OAI-PMH 2.0 alpha testers (2/2)
• Old Dominion U. -- ARC , DP9 • U. of Illinois at Urbana-Champaign • U. Of Southampton -- OAIA, CiteBase, eprints.org
• UCLA, John Hopkins U., Indiana U., NYU -- sheet music collection • UKOLN, U. of Bath -- RDN• Virginia Tech -- repository explorer
![Page 23: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/23.jpg)
beta phase [05/02]
• beta release on May 1st 2002 to:
• registered data providers and service providers
• interested parties
• fine tuning of protocol document
• preparation for the release of 2.0 conformant tools by alpha testers
![Page 24: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/24.jpg)
What’s new in OAI-PMH v.2.0?
corrections
new functionality
general changes to improve solidity of protocol
quick recap
![Page 25: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/25.jpg)
Overview of OAI Verbs
Verb Function
Identify description of archive
ListMetadataFormats metadata formats supported by archive
ListSets sets defined by archive
ListIdentifiers OAI unique ids contained in archive
ListRecords listing of N records
GetRecord listing of a single record
archivalmetadata
harvestingverbs
most verbs take arguments: dates, sets, ids, metadata formatsand resumption token (for flow control)
![Page 26: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/26.jpg)
Identify
• Arguments– none
• Errors– none
• Arguments– none
• Errors– badArgument
1.1 2.0
![Page 27: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/27.jpg)
ListMetadataFormats
• Arguments– identifier
(OPTIONAL)
• Errors– id does not exist
• Arguments– identifier
(OPTIONAL)
• Errors– badArgument– noMetadataFormats– idDoesNotExist
1.1 2.0
![Page 28: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/28.jpg)
ListSets
• Arguments– resumptionToken
(EXCLUSIVE)
• Errors– no set hierarchy
• Arguments– resumptionToken
(EXCLUSIVE)
• Errors– badArgument– badResumptionToken– noSetHierarchy
1.1 2.0
![Page 29: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/29.jpg)
ListIdentifiers
• Arguments– from (OPTIONAL)
– until (OPTIONAL)
– set (OPTIONAL)
– resumptionToken (EXCLUSIVE)
• Errors– no records match
• Arguments– from (OPTIONAL)– until (OPTIONAL)– set (OPTIONAL)– resumptionToken
(EXCLUSIVE)– metadataPrefix (REQUIRED)
• Errors– badArgument– cannotDisseminateFormat– badGranularity– badResumptionToken– noSetHierarchy– noRecordsMatch
1.1 2.0
![Page 30: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/30.jpg)
ListRecords
• Arguments– from (OPTIONAL)– until (OPTIONAL)– set (OPTIONAL)– resumptionToken
(EXCLUSIVE)– metadataPrefix
(REQUIRED)
• Errors– no records match– metadata format cannot be
disseminated
• Arguments– from (OPTIONAL)– until (OPTIONAL)– set (OPTIONAL)– resumptionToken
(EXCLUSIVE)– metadataPrefix (REQUIRED)
• Errors– noRecordsMatch– cannotDisseminateFormat– badGranularity– badResumptionToken– noSetHierarchy– badArgument
1.1 2.0
![Page 31: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/31.jpg)
GetRecord
• Arguments– identifier
(REQUIRED)
– metadataPrefix (REQUIRED)
• Errors– id does not exist
– metadata format cannot be disseminated
• Arguments– identifier
(REQUIRED)– metadataPrefix
(REQUIRED)
• Errors– badArgument– cannotDisseminateFor
mat– idDoesNotExist
1.1 2.0
![Page 32: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/32.jpg)
general changes
• clear distinction between protocol and
periphery
• fixed protocol document
• extensible implementation guidelines:
• e.g. sample metadata formats, description containers, about containers
• allows for OAI guidelines and community guidelines
![Page 33: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/33.jpg)
general changes
• clear separation of OAI-PMH and HTTP
• OAI-PMH error handling
• all OK at HTTP level? => 200 OK
• something wrong at OAI-PMH level? => OAI-PMH error (e.g. badVerb)
![Page 34: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/34.jpg)
OAI Data Model:
Resources / Items / Records
resource
all available metadata about David
item
Dublin Coremetadata
MARCmetadata
SPECTRUMmetadata records
item = identifier
record = identifier + metadata format + datestamp
![Page 35: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/35.jpg)
general changes
• better definitions of harvester,
repository, item, unique identifier, record,
set, selective harvesting
• oai_dc schema builds on DCMI XML
Schema for unqualified Dublin Core
• usage of must, must not etc. as in
RFC2119
• wording on response compression
![Page 36: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/36.jpg)
general changes
• all protocol responses can be validated
with a single XML Schema
• easier for data providers
• no redundancy in type definitions
• SOAP-ready
• clean for error handling
![Page 37: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/37.jpg)
<?xml version="1.0" encoding="UTF-8"?><OAI-PMH><responseDate>2002-0208T08:55:46Z</responseDate> <request verb=“GetRecord”… …>http://arXiv.org/oai2</request> <GetRecord> <record> <header> <identifier>oai:arXiv:cs/0112017</identifier> <datestamp>2001-12-14</datestamp> <setSpec>cs</setSpec> <setSpec>math</setSpec> </header> <metadata> ….. </metadata> </record> </GetRecord></OAI-PMH>
response no errors
![Page 38: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/38.jpg)
<?xml version="1.0" encoding="UTF-8"?><OAI-PMH><responseDate>2002-0208T08:55:46Z</responseDate> <request>http://arXiv.org/oai2</request><error code=“badVerb”>ShowMe is not a valid OAI-PMH verb</error></OAI-PMH>
response with error
![Page 39: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/39.jpg)
corrections
• all dates/times are UTC, encoded in
ISO8601, Z-notation
1957-03-20T20:30:00.00Z
![Page 40: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/40.jpg)
• idempotency of resumptionToken: return same
incomplete list when rT is reissued
• while no changes occur in the repo: strict
• while changes occur in the repo: all items with
unchanged datestamp
•new attributes for the resumptionToken:
•expirationDate
•completeListSize
•cursor
resumptionToken
![Page 41: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/41.jpg)
• harvesting granularity
• mandatory support of YYYY-MM-DD
• optional support of YYYY-MM-DDThh:mm:ssZ
• granularity of from and until must be the
same
new functionality
![Page 42: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/42.jpg)
• Identify more expressive
new functionality
<Identify>
<repositoryName>Library of Congress 1</repositoryName>
<baseURL>http://memory.loc.gov/cgi-bin/oai</baseURL>
<protocolVersion>2.0</protocolVersion>
<adminEmail>[email protected]</adminEmail>
<adminEmail>[email protected]</adminEmail>
<deletedRecord>transient</deletedRecord>
<earliestDatestamp>1990-02-01T00:00:00Z</earliestDatestamp>
<granularity>YYYY-MM-DDThh:mm:ssZ</granularity>
<compression>deflate</compression>
![Page 43: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/43.jpg)
• header contains set membership of item
new functionality
<record> <header> <identifier>oai:arXiv:cs/0112017</identifier> <datestamp>2001-12-14</datestamp> <setSpec>cs</setSpec> <setSpec>math</setSpec> </header> <metadata> ….. </metadata> </record>
![Page 44: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/44.jpg)
• ListIdentifiers returns headers
new functionality
<?xml version="1.0" encoding="UTF-8"?><OAI-PMH><responseDate>2002-0208T08:55:46Z</responseDate> <request verb=“…” …>http://arXiv.org/oai2</request><ListIdentifiers> <header> <identifier>oai:arXiv:hep-th/9801001</identifier> <datestamp>1999-02-23</datestamp> <setSpec>physic:hep</setSpec> </header> <header> <identifier>oai:arXiv:hep-th/9801002</identifier> <datestamp>1999-03-20</datestamp> <setSpec>physic:hep</setSpec> <setSpec>physic:exp</setSpec> </header> ……
![Page 45: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/45.jpg)
• ListIdentifiers mandates
metadataPrefix as argument
new functionality
http://www.perseus.tufts.edu/cgi-bin/pdataprov?
verb=ListIdentifiers
&metadataPrefix=olac
&from=2001-01-01
&until=2001-01-01
&set=Perseus:collection:PersInfo
![Page 46: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/46.jpg)
• character set for metadataPrefix and
setSpec extended to URL-safe characters
new functionality
A-Z a-z 0-9 _ ! ‘ $ ( ) + - . *
• identifierType = anyURI
•repositoryName = string
![Page 47: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/47.jpg)
• introduction of provenance container to
facilitate tracing of harvesting history
in the periphery
<about> <provenance> <originDescription> <baseURL>http://an.oa.org</baseURL> <identifier>oai:r1:plog/9801001</identifier> <datestamp>2001-08-13T13:00:02Z</datestamp> <metadataPrefix>oai_dc</metadataPrefix> <harvestDate>2001-08-15T12:01:30Z</harvestDate> </originDescription> <originDescription> … … … </originDescription> </provenance></about>
![Page 48: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/48.jpg)
• introduction of friends container to
facilitate discovery of repositories
in the periphery
<description>
<Friends>
<baseURL>http://cav2001.library.caltech.edu/perl/oai</baseURL>
<baseURL>http://formations2.ulst.ac.uk/perl/oai</baseURL>
<baseURL>http://cogprints.soton.ac.uk/perl/oai</baseURL>
<baseURL>http://wave.ldc.upenn.edu/OLAC/dp/aps.php4</baseURL>
</Friends>
</description>
![Page 49: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/49.jpg)
• revision of oai-identifier
• guidelines for collection-level and set-
level metadata
in the periphery
![Page 50: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/50.jpg)
future
adoption
communities
OAI-PMH
![Page 51: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/51.jpg)
• release of OAI-PMH v.2.0 [06/2002]
• no backwards compatibility with v.1.0/1.1
• stable
• migration process for registered repos
• ? formal standardization ?
• ? SOAP version ~ web services framework [SOAP, WSDL, UDDI] ?
the OAI-PMH
![Page 52: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/52.jpg)
• proliferation of community-specific add-ons for:
• collection & set level metadata
• expressive metadata formats (e.g. qualified DC XML Schema)
• shared set-structures
• machine readable rights (about the metadata)
communities
![Page 53: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/53.jpg)
• evolution
• from talking about OAI-PMH
• to talking about projects that use OAI-PMH
• to talking about projects and failing to mention they use OAI-PMH
=> OAI-PMH becomes part of the infrastructure
adoption
![Page 54: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/54.jpg)
indicators of adoption of OAI-PMH
tools
structural support
service providers
data providers
![Page 55: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/55.jpg)
• 49 registered repositories [11/2001]
• 65 registered repositories [03/2002]
• 77 registered repositories [05/2002]
• 5+ million records
• many unregistered repositories
data providers
![Page 56: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/56.jpg)
•Arc : cross-searching of registered
repositories [Old Dominion U]
[ http://arc.cs.odu.edu ]
• OLAC: cross-searching of Language
Archive Community repositories
http://www.language-archives.org/index.html
service providers
![Page 57: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/57.jpg)
• Scirus scientific search engine [Elsevier]
[ http://www.scirus.com ]
• my.OAI : user-tailorable cross-searching
of registered repositories [FS Consulting,
Inc.]
[http://www.myoai.com]
• growing interest from web search
engines
service providers
![Page 58: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/58.jpg)
• Repository Explorer: interactive exploration of
repositories [Virginia Tech]
[ http://www.purl.org/NET/oai_explorer ]
• eprints.org: generic OAI-PMH compliant repository
software [U of Southampton]
[ http://www.eprints.org ]
• ALCME repository and harvester software [OCLC]
[ http://alcme.oclc.org/index.html ]
OAI-PMH tools
![Page 59: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/59.jpg)
• Kepler [Old Dominion U]
• your personal OAI data provider: Kepler archivelet
• the Kepler service provider harvests from
archivelets that register
• archivelet downloadable
•http://www.dlib.org/dlib/april01/maly/04maly.html
exploration
![Page 60: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/60.jpg)
• DP9 [Old Dominion U]• provides entry page to repositories for web-
crawlers
• provides bookmarkable URL for OAI record
• provides resolution of OAI identifier into
metadata
• software downloadable
exploration
![Page 62: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/62.jpg)
Emergency Backup Slides
![Page 63: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/63.jpg)
resumptionToken
harvester RDBMS
ListRecords
Records 1-100, resumptionToken=AXad31
ListRecords, resumptionToken=AXad31
Records 101-200, resumptionToken=pQ22-x
ListRecords, resumptionToken=pQ22-x
Records 201-277
scenario: harvesting277 records in 3 separate100 record “chunks”
![Page 64: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/64.jpg)
Open Archives Initiative Open Archival Information System
http://www.dlib.org/dlib/april01/04editorial.htmlhttp://www.dlib.org/dlib/may01/05letters.htmlhttp://ssdoo.gsfc.nasa.gov/nost/isoas/us/overview.html
exposure of metadata for harvesting insuring long-term preservation of archival materials
OAIS
OAIS w/an OAI interface
![Page 65: New Developments in OAI Michael L. Nelson Old Dominion University mln/ mln@cs.odu.edu OA-Forum May 13-14, 2002 Pisa, Italy Many](https://reader036.vdocuments.site/reader036/viewer/2022062714/56649d595503460f94a391ce/html5/thumbnails/65.jpg)
Field of Dreams• It should be easy to be a data provider, even if it makes more work for
the service provider.– if enough data providers exist, the service providers will come (DPs >>
SPs)
• Open-source / freely available tools– “drop-in” data providers:
• industrial strength: http://www.eprints.org/• personal size: http://kepler.cs.odu.edu/
– tools to make your existing DL a data provider:• http://www.openarchives.org/tools/tools.htm• also: OAI-implementers mailing list / mail archive!
– service providers:• only bits and pieces currently publicly available...