Download - DSpace & DuraCloud Integrations
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
DSpace + DuraCloud IntegrationsTim Donohue
DuraSpace
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
1. DSpace AIP Backup & Restore (1.7 +)(Initial DuraCloud use case: Backup & Restore)
2. DSpace Curation Task System (1.7 +)
3. DSpace Replication Task Suite (1.8)
Basis for DSpace Integration
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• Primary Use Cases– Backup & Restore of DSpace Content
• All content or just partial (Community/Collection/Item)
– Migration/Export of DSpace Content• All content or just partial (Community/Collection/Item)
– DuraCloud Integration
Intro to Archival Info Pkgs (1.7+)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
How to Backup DSpace (pre-1.7)
Full Database Backup
FolderBackup
Database Assetstore Folder
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
How to Restore All (pre-1.7)
Full Database Backup
FolderBackup
Database Assetstore Folder
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
How to Restore a Collection (pre-1.7)
Full Database Backup
FolderBackup
Database Assetstore Folder
Temporary Database
Temporary Folder?
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
How to Restore a Collection (pre-1.7)
Full Database Backup
FolderBackup
Database Assetstore Folder
Temporary Database
Temporary Folder?
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Backup via Archival Info Pkgs
Package for each Community,
Collection & Item
AIP backup
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Restore All via Archival Info Pkgs
AIP backup
Package for each Community,
Collection & Item
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Restore a Collection via AIPs
AIP backup
Collection AIP
Items in Collection
1 2
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
What’s in an AIP?
METS(DIM / MODS / PREMIS /
METSRights)
LicenseContent Files
or Logos
*Also a BagIt version in works
Other Filesin Bundles
(optional)
Archival Information Package (AIP)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
What’s in an AIP?
Descriptive Metadata: DIM & MODS Tech/Preservation Metadata: PREMIS
Related Object AIPs
METS(DIM / MODS / PREMIS /
METSRights)
LicenseContent Files
or Logos Other Filesin Bundles
(optional)
Rights Metadata: METSRights
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
The “Site” AIP
Top-Level Community
AIPsMETS
(DIM / MODS / PREMIS / METSRights)
Special AIP for site-wide info/metadata:(e.g. Group Memberships, EPeople)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Restore All In-Archive Content (Files + Metadata)Restore All People & GroupsRestore All Permissions / Access RightsRestore Community / Collection Logos, Metadata,
Rights & Item TemplatesRestore Community / Collection / Item HierarchyRestore In-Process / Incomplete ItemsRestore Collection OAI-PMH/ORE Harvest SettingsRestore all configuration files (dspace.cfg, etc.)
What can AIPs restore?
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Migrate a Collection
Items in Collection
One DSpace Install Another DSpace Install
Collection AIP
12
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Migrate Content
12
(Future work)One DSpace Install
Collection AIP
Items in Collection
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Package for each Community,
Collection & Item
DuraCloud Integration (1.7.x)
Local “Watch” Folder
1
./dspace packager -d
java -jar synctool.jar
[1]
[2]
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
DuraCloud Integration (1.7.x)
Package for each Community,
Collection & Item
Local Folder
2
java -jar retrievaltool.jar
./dspace packager -r
[1]
[2]
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
New: DSpace Replication Suite in 1.8
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• Enables a basic ‘microservices’ approach to curating DSpace objects
• Anyone can build a task & share it.• Currently tasks must be written in Java
– Working on JRuby & Jython integration (1.8?)
• “Frees” admin tasks from Command Line– Can now run from Admin UI or CLI
DSpace Curation System (1.7+)
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• A set of curation tasks geared towards ‘replicating’ (backup/restore/audit) content
• “Wraps” 1.7 DSpace AIP Backup & Restore
DSpace Replication Suite (1.8)
Backup content to AIP (filesystem or DuraCloud)Restore/Replace from AIPAudit AIP (compare to DSpace content)Basic IO Tracking of AIP Upload/DownloadsAll replication tasks can be run via Admin UI
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Package for each Community,
Collection & Item
DuraCloud Integration (1.8.x)
Local Temp Folder(Cache)
ORCommand line Curation Tools
1
“Replication Task Suite”:• Suite of Curation Tasks• One step Sync process• Via UI or CLI
1
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Package for each Community,
Collection & Item
DuraCloud Integration (1.8.x)
Local Temp Folder(Cache)
ORCommand line Curation Tools
“Replication Task Suite”:• One step Retrieve process• Via UI or CLI• Also ‘auditing’ tools
1
1
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
DSpace Replication Suite Demo
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
Cannot yet take advantage of DuraCloud streaming capabilities (AIPs are zip files)
Cannot yet take advantage of DuraCloud transformation services (AIPs are zip files)
Known Limitations
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• Working towards “unzipped” AIPs (1.8?)– METS file & Content files stored in an AIP ‘folder’
but NOT zipped up– Support for DuraCloud streaming, etc.
• DSpace UI Streaming Integration (@mire)• ‘Auto-Sync’ options
– Updates in DSpace -> DuraCloud (queued?)– Updates via DuraCloud services -> DSpace?
Next Steps
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• MIT : Richard Rodgers & Wendy Bossons – Developed Curation Task Framework– Developed initial Replication Suite tasks
• @mire : Mark Diggory– Look for @mire’s “Integrating DuraCloud Services
in DSpace” talk on Friday at 3:30pm
In Large Thanks to…
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• Replication Task Suite:– https://wiki.duraspace.org/display/DSPACE/Replic
ationTaskSuite
• AIP Backup & Restore:– https://wiki.duraspace.org/display/DSDOC/AIP+Ba
ckup+and+Restore
• Curation Task System:– https://wiki.duraspace.org/display/DSDOC/Curati
on+System
For More Information
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]
• Package : http://www.flickr.com/photos/halfbisqued/2353845688/• Harddrive & Terminal icons: http://tango.freedesktop.org/Tango_Desktop_Project• Folder icon: http://www.openclipart.org/detail/13740• Database icon: http://www.openclipart.org/detail/68413• Zip Pkg icon: http://veryicon.com/icons/system/capital-icon-suite-mac/zip-10.html• File icons: http://veryicon.com/icons/system/rhor-v2-part-3/• Checkmark & Delete icons: http://veryicon.com/icons/system/on-stage/• Tools Icon: http://veryicon.com/icons/system/azullustre/
Photo/Icon Acknowledgments
