oai workshop extended
Post on 27-Dec-2014
847 Views
Preview:
DESCRIPTION
TRANSCRIPT
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb1
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
OAI workshop:OAI-PMH - Practical
session
Bram VandeputteK.U.LeuvenARIADNE
1
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
2
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
• Harvesting metadata : the simple way
• Binding your database to the OAI-PMH target• Setting up an OAI-PMH target
• Harvesting metadata - take 2
• How OAI works
2
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
• Harvesting metadata : the simple way
• Binding your database to the OAI-PMH target• Setting up an OAI-PMH target
• Harvesting metadata - take 2
• How OAI works
2
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb3
How OAI works
• OAI “VERBS”
• Identify
• ListMetadataFormats
• GetRecord
• ListIdentifiers
• ListRecords
• ListSets
HARVESTER
REPOSITORY
OAI OAI
Service Provider Metadata Provider
HTTP Request
HTTP Response
(OAI Verb)
(Valid XML)
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb4
GetRecord
• Purpose
• Returns the metadata for a single item in the form of an OAI record
• Parameters
• identifier – unique id for item (R)
• metadataPrefix – metadata format for the record (R)
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb5
ListRecords• Purpose
• Retrieves metadata records for multiple items
• Parameters
• from – start date (O)- greater than or equal to
• until – end date (O)- less than or equal to
• set – set to harvest from (O)
• resumptionToken – flow control mechanism (X)
• metadataPrefix – metadata format (R)
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb6
ListRecords – from until
http://localhost:8080/oaicat/OAIHandler?verb=ListRecords&from=1999-01-15&until=2005-12-31&metadataPrefix=oai_lom
UTCdatetimeDates and times are uniformly encoded using ISO8601
and are expressed in UTC throughout the protocol. When time is included, the special UTC designator
("Z") must be used. UTC is implied for dates although no timezone designator is specified. For example,
1957-03-20T20:30:00Z is UTC 8:30:00 PM on March 20th 1957. UTCdatetime is used in both protocol
requests and protocol replies, in the way described in the following sections.
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb7
ListIdentifiers
• Purpose
• List headers for all items corresponding to the specified parameters
• Parameters
• from – start date (O)
• until – end date (O)
• set – set to harvest from (O)
• metadataPrefix – metadata format to list identifiers for (R)
• resumptionToken – flow control mechanism (X)
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb8
ListSets
• Purpose
• Provide a listing of sets in which records may be organized (may be hierarchical, overlapping, or flat)
• Parameters
• None
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
9
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
• Harvesting metadata : the simple way
• Binding your database to the OAI-PMH target• Setting up an OAI-PMH target
• Harvesting metadata - take 2
✓ How OAI works
9
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
• Harvesting metadata : the simple way
• Binding your database to the OAI-PMH target• Setting up an OAI-PMH target
• Harvesting metadata - take 2
✓ How OAI works
9
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Harvesting metadata• the simple way : harvesting through browser
• Identify- http://ariadne.cs.kuleuven.be/oaitarget/OAIHandler?verb=Identify
• ListMetadataFormats- http://ariadne.cs.kuleuven.be/oaitarget/OAIHandler?verb=ListMetadataFormats
• ListRecords- http://ariadne.cs.kuleuven.be/oaitarget/OAIHandler?verb=ListRecords&metadataPrefix=oai_lom
- http://ariadne.cs.kuleuven.be/oaitarget/OAIHandler?verb=ListRecords&resumptionToken=
• GetRecord- http://ariadne.cs.kuleuven.be/oaitarget/OAIHandler?verb=GetRecord
&identifier=oai:ariadne.cs.kuleuven.be:BLKLKP1382&metadataPrefix=oai_lom
• ...
10
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
11
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
✓ Harvesting metadata : the simple way
• Binding your database to the OAI-PMH target• Setting up an OAI-PMH target
• Harvesting metadata - take 2
✓ How OAI works
11
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
✓ Harvesting metadata : the simple way
• Binding your database to the OAI-PMH target• Setting up an OAI-PMH target
• Harvesting metadata - take 2
✓ How OAI works
11
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Setting up a target
12
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Setting up a target
Wiki page with detailed instructions :http://ariadne.cs.kuleuven.be/lomi/index.php/Setting_Up_OAI-PMH
12
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
OAI-PMH result
DYNAMO LOM metadataDYNAMO LOM
metadataDYNAMO LOM metadataRepository LOM
metadata
ARIADNEHarvester
1. Get metadata
out of database
Repository metadata
- identifier- title- url- project description- ...
Repository LOM metadata
- lom.general.identifier- lom.general.title- lom.general.description- lom.technical.location- ...
2. Mapping Process
4. Serve results
3. Copy results in
OAI-PMH service
OAI-PMH Target Software
Repository
13
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Setting up a target
14
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
• Install Java, Ant & apache-tomcat
• Set environment varsset JAVA_HOME=C:\j2sdk1.5.0set ANT_HOME=C:\apache-ant-1.6.5set PATH=%JAVA_HOME%\bin;%ANT_HOME%\bin;%PATH%
• Download oaicat source and buildcd <your build-dir>type “ant”
• Deploy codeput “oaicat_5.0/dist/oaicat.war” under “apache-tomcat/webapps/”
Setting up a target
14
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Setting up a target
15
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
• Testing the target
- http://localhost:8080/oaitarget/OAIHandler?verb=Identify
- http://localhost:8080/oaitarget/OAIHandler?verb=ListMetadataFormats
Setting up a target
15
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Setting up a target
16
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Setting up a target
• Binding to example lucene index
• Download the index :- http://ariadne.cs.kuleuven.be/MeltSqiOai/OAI/lucene.zip
• Extract
• open properties file with text editor :- oaicat_5.0/WEB-INF/oaicat.properties
• Enter the full path of index under the "LuceneLomCatalog.lucenePath" property
• reload the target
16
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Setting up a target
17
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Setting up a target
• Testing the target
- http://localhost:8080/oaitarget/OAIHandler?verb=ListRecords&metadataPrefix=oai_lom
- http://localhost:8080/oaitarget/OAIHandler?verb=ListRecords&resumptionToken=
- http://localhost:8080/oaitarget/OAIHandler?verb=GetRecord&identifier= oai:oaicat.ariadne.org:CS_LKP_v_3.0_nr_1162 &metadataPrefix=oai_lom
17
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb18
Setting up a target
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
• Validate the result metadata in the validationService online :
• http://ariadne.cs.kuleuven.be/validationService
18
Setting up a target
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
19
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
✓ Harvesting metadata : the simple way
• Binding your database to the OAI-PMH target✓ Setting up an OAI-PMH target
• Harvesting metadata - take 2
✓ How OAI works
19
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
✓ Harvesting metadata : the simple way
• Binding your database to the OAI-PMH target✓ Setting up an OAI-PMH target
• Harvesting metadata - take 2
✓ How OAI works
19
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Binding it to your DB
20
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Binding it to your DB
• Implement /adapt 3 classes :
• org.ariadne.oai.server.X.catalog.XCatalog.java
➡ Implements the OAI-PMH verbs
• org.ariadne.oai.server.X.catalog.XRecordFactory.java
➡ Creates OAI-PMH headers
• org.ariadne.oai.server.X.crosswalk.XCrosswalk.java
➡ Maps arbitrary object to LOM XML object
20
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Binding it to your DB
21
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Binding it to your DB
• XCatalog.java
• For each “verb” :
- get parameters
- query database to get matching metadata
- parse metadata into java objects (Vector, HashMap, ...)
- call RecordFactory and Crosswalk to create records
21
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Binding it to your DB
22
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Binding it to your DB
• XRecordFactory.java
• Two important methods :
- getLocalIdentifier(Object nativeItem);- getDatestamp(Object nativeItem);
22
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Binding it to your DB
23
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Binding it to your DB
• XCrosswalk.java
• Map fields of your object to LOM XML object :- createMetadata(Object nativeItem);
23
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
24
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
✓ Harvesting metadata : the simple way
✓ Binding your database to the OAI-PMH target✓ Setting up an OAI-PMH target
• Harvesting metadata - take 2
✓ How OAI works
24
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Overview
✓ Harvesting metadata : the simple way
✓ Binding your database to the OAI-PMH target✓ Setting up an OAI-PMH target
• Harvesting metadata - take 2
✓ How OAI works
24
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Harvesting metadata - take 2
25
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb
Harvesting metadata - take 2
Download ARIADNE harvester :http://www.cs.kuleuven.be/~bramv/oaiharvester.war
Wiki page with detailed instructions :http://ariadne.cs.kuleuven.be/lomi/index.php/Harvesting_Metadata
25
www.ariadne-eu.org www.cs.kuleuven.be/~hmdb26
References
• http://ariadne.cs.kuleuven.be/lomi/index.php/Setting_Up_OAI-PMH
• http://ariadne.cs.kuleuven.be/lomi/index.php/Harvesting_Metadata
• http://oai.grainger.uiuc.edu/FinalReport/JCDL_2003_OAI_Intro.ppt
• http://www.cs.kuleuven.ac.be/~hmdb/SqiOaiAspect/
• http://www.oclc.org/research/software/oai/cat.htm
• http://ariadne.cs.kuleuven.be/validationService
top related