hydra for cni spring 2014 meeting
DESCRIPTION
Hydra and Blacklight are both successful, vibrant open source projects among research libraries and higher education. They have achieved a critical mass of adopters and rich set of functions, as much because of the methodology and community as their strong technical capabilities. This presentation gives an overview of both projects, and describes the "Hydra-Blacklight" Way, and how this "way" is pulling institutions into the future of digital libraries.TRANSCRIPT
![Page 1: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/1.jpg)
Developing the Hydra - Blacklight Way
Tom CramerChief Technology StrategistStanford University Libraries
@tcramer
CNI Membership MeetingSt. Louis * April 1, 2014
![Page 2: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/2.jpg)
Agenda
• What is Hydra?• What is Blacklight?• Where are they going?• What does it all mean?
![Page 3: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/3.jpg)
What Is Hydra?
• A robust repository fronted by feature-rich, tailored applications and workflows (“heads”) ➭ One body, many heads
• Collaboratively built “solution bundles” that can be adapted and modified to suit local needs.
• A community of developers and adopters extending and enhancing the core➭ If you want to go fast, go alone. If
you want to go far, go together.
![Page 4: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/4.jpg)
Fundamental Assumption #1
No single system can provide the full range of repository-based solutions for a given institution’s needs,
…yet sustainable solutions require a common repository infrastructure.
![Page 5: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/5.jpg)
BooksArticlesThesesImagesMapsData (Raster)Data (Comp.)Data (Observ.)AudioVideoDocuments
![Page 6: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/6.jpg)
Point Solution Approach …Welcome to SilovilleETDs
(Theses)
Books, Article
s
Images
Audio-Visual
Research Data
Maps & GIS
Docu-ments
ETD IRImage
DBDAM ?
Geospa-tial Inf.
Records
Mgmt.
Management
Access Preservation(?)
Effective? Sustainable?
![Page 7: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/7.jpg)
Repository-Powered ApproachETDs
(Theses)
Books, Article
s
Images
Audio-Visual
Research Data
Maps & GIS
Docu-ments
Digital Repository
Scalable, Robust, Shared Management
and Preservation Services
![Page 8: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/8.jpg)
One Body, Many Heads…ETDs
(Theses)
Books, Article
s
Images
Audio-Visual
Research Data
Maps & GIS
Docu-ments
hydraScalable, Robust,
Shared Management and Preservation
Services
![Page 9: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/9.jpg)
Hydra Technical Framework
![Page 10: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/10.jpg)
CRUD in Repositories
![Page 11: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/11.jpg)
CRUD in Repositories
![Page 12: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/12.jpg)
Major Hydra Components
![Page 13: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/13.jpg)
A Note on Ruby on Rails• Rapid application development for
web applications: “Convention over configuration” – 10x productivity
• Supportable: MVC (Model-View-Controller) and Rails framework make code well-structured, predictable
• Testable: Rspec and Cucumber give powerful, automatable, testing tools
• Learnable: Stanford went from 1 to 8 Ruby savvy developers in one year (no new hires)– 1 week learning curve to basic
proficiency
![Page 14: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/14.jpg)
A Note on Fedora• Flexible, Extensible, Durable Object
Repository Architecture– Flexible: model and store any content types– Extensible: easy to augment with apps and
services– Durable: foundation of preservation repository
• Proven, sustained and successful digital repository– 100’s of adopters; 12 years of development, 3
releases– Vibrant community & funding under DuraSpace
• Fedora 4 in development; co-evolving with Hydra
![Page 15: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/15.jpg)
Hydra Community Framework
![Page 16: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/16.jpg)
Fundamental Assumption #2
No single institution can resource the development of a full range of solutions on its own,
…yet each needs the flexibility to tailor
solutions to local demands and workflows.
![Page 17: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/17.jpg)
Hydra Philosophy -- Community• An open architecture, with many
contributors to a common core• Collaboratively built “solution
bundles” that can be adapted and modified to suit local needs
• A community of developers and adopters extending and enhancing the core
• “If you want to go fast, go alone. If you want to go far, go together.”
One body, many heads
![Page 18: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/18.jpg)
Community• Conceived & executed as a distributed,
collaborative, open source effort from the start• Initially a joint development project between
Stanford, Univ of Virginia, and Univ of Hull
• Hydra Partners are the backbone of the project• Coalition of the willing• No fees or dues• Apache-style consensus and governance
• Steering Group provides administration, continuity, and serves as backstop when needed• But no central planning, no Project Director, no
“one” architect
![Page 19: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/19.jpg)
Hydra Partners…
…are individuals, institutions, corporations or other groups that have committed to contributing to the Hydra community; they not only use the Hydra technical framework, but also add to it in at least one of many ways: code, analysis, design, support, funding, or other resources.
Hydra Partners collectively advance the project and the community for the benefit of all participants.
https://wiki.duraspace.org/display/hydra/Hydra+Community+Framework
![Page 20: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/20.jpg)
![Page 21: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/21.jpg)
Code Licensing
• All Hydra code is available under Apache License, Version 2.0
• All code commitments are being managed through Contributor License Agreements• Individual – so each developer is clear
about what they are contributing
• Corporate – so each institution is clear about what it is contributing
• Code contributors maintain ownership of their IP• …and grant a non-exclusive license to
the project and its users
![Page 22: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/22.jpg)
Hydra Current State
![Page 23: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/23.jpg)
Hydra Partners
OR09 OR10 OR11 OR12 OR130
5
10
15
20
25
OR = Open Repositories Conference
![Page 24: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/24.jpg)
Hydra Partners and Known Users
OR09 OR10 OR11 OR12 OR13 Now0
5
10
15
20
25
30
35
40
OR = Open Repositories Conference
![Page 25: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/25.jpg)
Hydra Partners @ UCSD, 2012
University of Virginia, 2008
LibDevConX^3, Stanford, 2012: (Hydranauts and Fellow Travelers)
![Page 26: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/26.jpg)
First Worldwide Hydra Connect @ UCSD, 2013
![Page 27: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/27.jpg)
Trending
![Page 28: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/28.jpg)
Trending
![Page 29: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/29.jpg)
Hydra Heads: Institutional Repositories
University of Hull
![Page 30: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/30.jpg)
Hydra Heads: Institutional Repositories
ScholarSphere / Sufia @ Penn State
![Page 31: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/31.jpg)
Hydra Heads: Institutional Repositories
Hydramata• 6-institution collaboration• Build a shared Hydra
head to serve as an IR• Distributed, agile team
University of Notre DameNorthwestern UniversityIndiana UniversityUniversity of CincinnatiUniversity of Virginia
![Page 32: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/32.jpg)
Hydra Heads of Note
Avalon & HydraDAM for Media
Sufia
BPL Digital Commonwealth
UCSD DAMS
Northwestern Digital Image Lib.
![Page 33: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/33.jpg)
Hydra-based Applications at Stanford
ETD’s – Electronic Theses & Dissertations
SALT – Self-Archiving Legacy Toolkit
EEMs – Everyday Electronic Materials
Argo – Repository Reporting and Management
Hypatia – Archives & Special Collections
SDR – Web UI (aka Hydrus)
![Page 34: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/34.jpg)
Sustainability
No animals were harmed in the making of this film.
grants abused
project / community
![Page 35: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/35.jpg)
Current State of Hydra
• Doubled in size, thrice, since its founding
• Achieved all its original goals• Entering a new phase
![Page 36: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/36.jpg)
Three critical priorities
1. Technical framework continue to advance to enable easy code sharing and reuse
2. Community framework evolves to accommodate growth in partners.
3. Create a full suite of solution bundles
![Page 37: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/37.jpg)
Eight strategic priorities
1. Develop solution bundles2. Develop turnkey applications3. Grow the Hydra vendor ecosystem4. Codify a scalable training framework to fuel
community growth5. Develop a documentation framework6. Ensure the technical framework allows code
sharing7. Refresh and intensify the community ties8. Grow the community of users and adopters
![Page 38: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/38.jpg)
Blacklight
![Page 39: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/39.jpg)
Blacklight for Repositories• Repository-agnostic, feature-rich,
content-aware, turnkey access interface
• Aggregate content from multiple repositories, with links back to source systems
• Vibrant, multi-institutional, open source community on its own
• Can be used independently, or as the first component of, Hydra
![Page 40: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/40.jpg)
Blacklight
• Who’s Using It?– Scores (probably hundreds) of institutions– Notables: UVa, Stanford, Columbia, Cornell, Princeton,
Yale, Johns Hopkins, WGBH, BPL, NYPL, Danish Technical University, Tufts, Indiana University, Univ. of Wisconsin, et al.
– Every Hydra institution
• Remarkably strong community– Regular workshops and tutorials at Code4Lib, DLF, et
al.– 328 members on blacklight-development@
googlegroups.com
![Page 41: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/41.jpg)
Blacklight Development
http://www.ohloh.net/p/blacklight
![Page 42: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/42.jpg)
Library Catalog @ Stanford
![Page 43: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/43.jpg)
CLIO at Columbia University – Bento Box results w/ article search
![Page 44: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/44.jpg)
Virgo at University of Virginia – Music Search View
![Page 45: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/45.jpg)
Forward at University of Wisconsin
![Page 46: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/46.jpg)
Revs Digital Library at Stanford: Blacklight for Image Collection
![Page 47: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/47.jpg)
FRDA at Stanford: Blacklight for text & image collection
![Page 48: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/48.jpg)
Digital Manuscript Index at Stanford: Blacklight for MS Catalog
![Page 49: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/49.jpg)
Bassi-Veratti at Stanford: Blacklight for EAD and digital archive
![Page 50: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/50.jpg)
Rock & Roll Hall of Fame: Blacklight for Catalog, EAD and Media
![Page 51: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/51.jpg)
OpenVault: Blacklight for Video at WGBH
![Page 52: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/52.jpg)
Digital Commonwealth at BPL: Blacklight for statewide repository
![Page 53: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/53.jpg)
GeoBlacklight: Blacklight for geospatial resources
![Page 54: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/54.jpg)
Spotlight: Blacklight for exhibits
![Page 55: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/55.jpg)
The Hydra-Blacklight Way
![Page 56: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/56.jpg)
Why Hydra?
• Need a flexible, rich, robust repository solution
• Be part of a community– Shared resources, shared risk, shared
advances
• “We” is smarter than “me”– Especially in this crowd
• Control your own destiny• Build institutional capacity• Practice best practices• The way work is getting done now
![Page 57: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/57.jpg)
Breadth & Depth: Access, Mgmt & PresETDs
(Theses)
Books, Article
s
Images
Audio-Visual
Research Data
Maps & GIS
Docu-ments
Digital Repository
Scalable, Robust, Shared Management
and Preservation Services
![Page 58: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/58.jpg)
If You Want To Go Fast…go alone.…use Hydra
1) Notre Dame deployed a video cataloging head in 6 weeks, from scratch
https://github.com/projecthydra/hypatia/graphs/impact
2) Hypatia development at Stanford• First 20% of project took 80% of time
(modelling, data acquisition and preparation)
• 80% of deliverables (coding) happened in 8 week spurt, by leveraging Hydra
![Page 59: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/59.jpg)
Becoming a Hub
• Working Groups forming– Repository Managers– Archivsists– Hydra RDF– Fedora…
• Jobs
![Page 60: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/60.jpg)
Sustainability…
• For institutionsPartners help shoulder the load
• For other projectsPost-project maintenance & governance
• For leveraging funders’ investmentsAdoption by growing user community
![Page 61: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/61.jpg)
The Hydra Way
• Not a directed project• Investment in a framework, not an
application– Contributions back to core code base
• Investment in a community, not a vendor– Contributions back to community: Training,
documentation, modeling, evangelism, support
• Travel / face-time
![Page 62: Hydra for CNI Spring 2014 Meeting](https://reader033.vdocuments.site/reader033/viewer/2022060118/558a615bd8b42a626e8b46b2/html5/thumbnails/62.jpg)
http://projecthydra.org