1

DSpace Progress and ChallengesNavigating the Standard Seas

Robert TansleyGoogle Inc.21 September 2006

2

Overview

• State of the DSpace Nation

– New Developments

• Standards

• The China Digital Museum Project

• Future needs

3

State of the DSpace Nation

• Well over 150 institutional users

• 9 committers, very active developer and technical lists

• New features, bug reports, fixes pour in

• Large technical community with deep understanding of system

• Architecture largely unchanged for 4 years

• “Chaotic” OSS community great for incremental feature/UI development; architectural developments more difficult

• Committer group (and everyone else) have day jobs

• Perception that some centralised team somewhere is taking care of it (committers? MIT? HP?)

4

New Developments

• DSpace governance advisory board formed, first met March 2006

• Decision made to form a 501(c)(3) non-profit

• DSpace architecture review group formed, charged with advancing the platform’s architecture (“community-ratified” decision making)

5

DSpace Architecture Review

• Technical scope of the project: Application or middleware?

• Data model, versioning

• Modularisation, network vs local APIs

• “Scalability”

• Managing complexity

• Interoperability

• Key aspect is trying to make DSpace less "monolithic”

– while retaining "out of the box" usage

6

Which standards does DSpace use?

• Dublin Core

• OAI-PMH

• Handle System

• METS

• PREMIS

7

Which standards does DSpace use?

• CSS

• DC

• DTD

• HTTP

• HTTPS

• Handle System

• Harmony/ABC

• LDAP

• METS

• MIME

• MODS

• MPEG-21 DIDL

• OAI-PMH

• PDF

• PREMIS

• RDF

• RDFS

• SMTP

• SQL

• SRW

• SSL

• TLS

• URI + URL

• UTF-8

• WebDAV

• X.509

• XHTML

• XML

• XML Schema

• XSLT

8

So why aren’t all those enough?

• Need a blend of standards

• Standards don’t answer the hard questions

– “Implementation issue”

• Many, radically different data models

• Standards can become an obstacle rather than a help

9

The China Digital Museum Project

Many universities in China have a museum

• Many objects of historic, cultural, educational and research value

Due to space requirements, and size (geographic and population) of the country, museum contents are not optimally utilised

Additionally, some physical objects are deteriorating

Therefore objects are being digitised

• Improve access

• Preserve

HP, China Ministry of Education, Beihang university project

Results will be added to OSS DSpace

10

Architecture

11

Possibilities

12

Data Model Challenges

• Modelling object types/classes

• Relationships (containership, other)

• Representation information

– File formats

– Complex objects

– “Semantics” (e.g. how to interpret columns in dataset)

– Derived, converted, “equivalent” or lossy inter-representation relationships

• Serialisations of all the above

13

China DM Project Data Model

14

How to address

• More aggressive standards

– Data models

– Representation information (beyond file formats)

– Make tradeoffs

– Don’t just “leave as implementation detail”; massively reduces standard’s utility, potential for impact

15

Summary

• Standards are too “visible”

• We need to make standards “disappear”

• And be willing to accept tradeoffs

• Flexibility isn’t always a good thing!!