adding oai-ore support to repository platforms alexey maslov, adam mikeal, scott phillips, john...
TRANSCRIPT
![Page 1: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/1.jpg)
Adding OAI-ORE Support to Repository Platforms
Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland
Texas Digital LibraryTCDL’09
![Page 2: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/2.jpg)
Overview
• Texas Digital Library Use Case for OAI-ORE
• Mapping ORE model to DSpace architecture
• Implementation
• Results and Implications
![Page 3: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/3.jpg)
Texas Digital Library
• State-wide initiative
• Eighteen members
• Public/Private
• Small/Medium/Large
![Page 4: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/4.jpg)
Electronic Theses and Dissertations
• Federated Collection
• Built on top of DSpace/Manakin
![Page 5: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/5.jpg)
Current Federation Method
• Performed via scripted ingest process
• New batch every semester
• Manual corrections to existing content
![Page 6: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/6.jpg)
Replacement Requirements
• Perform maintenance automatically
• Detect changes in existing content
• Support interchange of metadata and content
![Page 7: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/7.jpg)
Harvesting Solution
• Use the Open Archives Initiative Protocol for Metadata Harvesting
• Member institutions as data providers
• TDL Federated Repository as a service provider
* Open Archives Initiative Protocol for Metadata Harvestinghttp://www.openarchives.org/pmh/
![Page 8: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/8.jpg)
OAI-PMH, advantages
• Ubiquitous
• Supports selective harvesting
• Tracks changes
• Can be automated
![Page 9: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/9.jpg)
OAI-PMH, obstacles
• No existing harvesting solution for DSpace
• Supports harvesting of metadata specifically
![Page 10: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/10.jpg)
Disseminating content
• How do you disseminate content through a metadata harvesting protocol?
– Wrap it in a packaging format
– Include the metadata
– Encode the references to the files
– Harvest the package
![Page 11: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/11.jpg)
METS, advantages
• Metadata Encoding and Transmission Standard
• Maintained by the Library of Congress
• Mature standard
• Widely adopted
* Metadata Encoding and Transmission Standard, Library of Congresshttp://www.loc.gov/standards/mets/
![Page 12: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/12.jpg)
Packaging, disadvantages
• Complete packaging format
• Open to interpretation
• Ambiguities at the OAI-PMH layer
![Page 13: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/13.jpg)
OAI-ORE
“Open Archives Initiative Object Reuse and Exchange defines standards for the description
and exchange of aggregations of Web resources.”
• Specialized
• Simple
* Open Archives Initiative Object Reuse and Exchangehttp://www.openarchives.org/ore/
![Page 14: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/14.jpg)
Mapping DSpace to OAI-ORE
• ORE Abstract Data Model
• DSpace architecture
• The Mapping
![Page 15: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/15.jpg)
ORE Data Model
• Aggregations
• Aggregated Resources
• Resource Maps
![Page 16: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/16.jpg)
Aggregation (A)
• Describes a set of resources
• Conceptual construct
![Page 17: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/17.jpg)
Aggregated Resource (AR)
• Object of interest
• Part of an aggregation
• Can itself be an aggregation
![Page 18: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/18.jpg)
Aggregated Resource (AR)
• Object of interest
• Part of an aggregation
• Can itself be an aggregation
![Page 19: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/19.jpg)
Resource Map (ReM)
• Describes an aggregation
• Enumerates its aggregated resources
• Can be serialized in RDF or Atom XML
![Page 20: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/20.jpg)
DSpace Model v1.x
• Communities
• Collections
• Items
• Bundles
• Bitstreams
![Page 21: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/21.jpg)
ORE DSpace
![Page 22: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/22.jpg)
Mapping
![Page 23: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/23.jpg)
Mapping
![Page 24: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/24.jpg)
Mapping
![Page 25: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/25.jpg)
Bundles?
![Page 26: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/26.jpg)
Bundles, Potential Options
• Bundles as Aggregations of Bitstreams
• Bundles as filters for Aggregated Resources
• Bundles as DSpace-specific metadata
![Page 27: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/27.jpg)
Bundles, Observations
• By default, specialized for internal tasks
• Extendible for any use
• Obscured from the end user
![Page 28: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/28.jpg)
DSpace Bundles
![Page 29: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/29.jpg)
Serialization in Atom
![Page 30: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/30.jpg)
Implementation• ORE Dissemination
• ORE Harvesting
• Automation
![Page 31: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/31.jpg)
Interfacing with DSpace
• Web UI
• LNI and SWORD
• Ingest and export scripts
• Crosswalks
– Ingestion
– Dissemination
![Page 32: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/32.jpg)
ORE Dissemination Crosswalk
• Requires:
– A DSpace Item
• Produces:
– Atom-serialized ORE ReM
![Page 33: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/33.jpg)
ORE Dissemination via OAI-PMH
• Dissemination crosswalk produces ORE ReMs from DSpace Items
• OAI-PMH data provider disseminates them
![Page 34: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/34.jpg)
ORE Harvesting
• Item-level ORE ReM interpreter
• Collection-level OAI-PMH harvester
• Repository level harvest scheduler
![Page 35: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/35.jpg)
ORE Ingestion Crosswalk
• Requires:
– A DSpace Item
– Atom-serialized ORE ReM
• Produces:
– A DSpace Item with Bitstreams created from AR’s
![Page 36: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/36.jpg)
OAI-PMH Harvester
• Queries remote OAI-PMH providers
• Processes responses as individual records
• Implemented at Collection level
![Page 37: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/37.jpg)
Collection Settings
• Source of collection’s content
• OAI-PMH provider information
• Harvesting Level
![Page 38: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/38.jpg)
Collection Source
![Page 39: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/39.jpg)
OAI-PMH Settings
• OAI-PMH Provider
• OAI Set Id
• DMD Format
![Page 40: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/40.jpg)
Harvest Level
![Page 41: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/41.jpg)
Harvesting a CollectionLocal collection(OAI-PMH harvester)
Remote collection(OAI-PMH provider)
![Page 42: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/42.jpg)
Harvest MetadataLocal collection(OAI-PMH harvester)
Remote collection(OAI-PMH provider)
![Page 43: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/43.jpg)
Metadata ReplicatedLocal collection(OAI-PMH harvester)
Remote collection(OAI-PMH provider)
![Page 44: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/44.jpg)
Case 1: Metadata OnlyLocal collection(OAI-PMH harvester)
Remote collection(OAI-PMH provider)
![Page 45: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/45.jpg)
Harvest ORE ReMsLocal collection(OAI-PMH harvester)
Remote collection(OAI-PMH provider)
![Page 46: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/46.jpg)
Case 2: Metadata + Content Ref’s
Local collection(OAI-PMH harvester)
Remote collection(OAI-PMH provider)
![Page 47: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/47.jpg)
Case 2: Metadata + Content Ref’s
Local collection(OAI-PMH harvester)
Remote collection(OAI-PMH provider)
![Page 48: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/48.jpg)
Case 3: Metadata + Content
Local collection(OAI-PMH harvester)
Remote collection(OAI-PMH provider)
![Page 49: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/49.jpg)
Case 3: Metadata + Content
Local collection(OAI-PMH harvester)
Remote collection(OAI-PMH provider)
![Page 50: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/50.jpg)
Harvest Scheduling System
• Monitors harvested collections
• Starts harvests at regular intervals
• Alerts administrators of errors
![Page 51: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/51.jpg)
Results• The Primary Use Case
• TDL in General
• The Greater Web Community
![Page 52: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/52.jpg)
Harvesting using PMH+ORE
• Federated ETD collection currently in pre-production at TDL
• Addresses primary requirements
– Performs maintenance automatically
– Detects changes in existing content
– Supports interchange of metadata and content
![Page 53: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/53.jpg)
Other Possibilities
• Specialized DSpace instances
• Flexible repository architecture
• Interoperability with other repository systems
“Large-scale ETD repositories: A case study of a digital library application”, Adam Mikeal, James Creel, Alexey Maslov, Scott Phillips, John Leggett, Mark McFarland.
JCDL 2009
![Page 54: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/54.jpg)
Current Priorities
• Live deployment at TDL
• Release to the open source community
• Integration into DSpace 1.6
![Page 55: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/55.jpg)
National Leadership Grant #LG-05-07-0095-07
![Page 56: Adding OAI-ORE Support to Repository Platforms Alexey Maslov, Adam Mikeal, Scott Phillips, John Leggett, Mark McFarland Texas Digital Library TCDL09](https://reader035.vdocuments.site/reader035/viewer/2022070305/5513da165503466f748b5119/html5/thumbnails/56.jpg)
Questions?