cwspace archiving mit opencourseware in dspace dspace federation 2nd user group meeting university...

CWSpaceArchiving MIT OpenCourseWare in DSpace

DSpace Federation 2nd User Group Meeting

University of CambridgeJuly 7 - 8, 2005

v. 20050704_2215

July 8, 2005 CWSpace, an MIT iCampus project 2

DSpace: wide adoption as Institutional Repository

• CONTENT– Scholarly

materials– Research or

Education oriented

• FEATURES– Safe

• professionally archived

• persistent, citable URL

• preserved over archival timeframes

• policies re: removal, etc.

– Findable• search (metadata; full-text)

• browse

• notification e-mails

• disseminated metadata (OAI-PMH)

• Google and DSpace ("Scholar")

AUDIENCEHumans; SysAdmins;

Spiders; Harvesters


DSpace: new territory as LOR?Learning Objects Repository

• CONTENT– Educational content:

• Teaching, learning, instruction, assessment

– Courseware materials• Learning Objects

– Compound Digital Objects, Websites…

• FEATURES– Safe

– Findable

– Emphasis on:• Sharing, Re-Use• Aggregation• Evaluation

AUDIENCEHumans; SysAdmins; Spiders ; Harvesters

System-to-System: CLEs; Image gallery tools


Project Goal: InterOperability

"To harvest and digitally archive OCW learning objects, and make them available to learning management systems (LMSs) by using Web Services interfaces on top of DSpace."

• re: "harvest" - Yes. OCW has built "Content Exporter”

• re: "archive" - Yes. DSpace has new IMS-CP ingest module; course rendering

• re: "learning objects" - No, but courseware, Yes. Discovered that from OCW will be Courses not LOs

• re: "Web Services"...to

"LMSs" - Yes. Initial version WS simple clients; also to LMS (SloanSpace) prototypehttp://icampus.mit.edu/projects/DSpace.shtml


InterOperability: the main goal


CWSpace: Standards forPackages & Protocols

• Packages– IMS Content

Package– METS– MPEG21-DIDL– XFDU

• Protocols, APIs– SOAP & WSDL– WebDAV

• JSR 170 JCR

– RESTful (XML, HTTP)

– XML-RPC


Work Activity To DateSchematic showing various areas of development: metadata

specifications, Web Services, export and import programming.


Metadata for Content Packaging

• METS– Libraries…

• IMS-CP– Education…

• MPEG-21 DIDL– Commercial…

• XFDU– Aerospace…


Package Interchange File (PIF)• IMS-CP uses a .ZIP file with a Manifest XML file

and all content files:

http://www.imsglobal.org/content/packaging/cpv1p1p4/imscp_bestv1p1p4.html


IMS-CP and OCW Object Model

IMS-CP imsmanifest.xml


METS & IMS-CP

Manifests Diagrammed

http://cwspace.mit.edu/docs/ProjectMgt/Reports/SPARC-IR-Workshop/sparc-poster.html

IMS-CP imsmanifest.xml

METS mets.xml

Red arrow: Logical organization to Physical (href)

Blue arrow (METS only): Descriptive metadata is separated from Logical or Physical

Red arrow: Logical organization to Physical (href)


XML Elements: METS & IMS-CP

For those who like the pointy angle brackets…


METS SIP Profile for DSpace• Work-in-progress

– MIT CSAIL Publications Archive

• Anticipates DSpace 2.0 possible METS as AIP• Preservation and Technical Metadata• Licenses as METS metadata (Deposit; Creative

Commons)• StructMap raises questions for flat DSpace file

storage (Bundles & fileGrps) (Export re-creation)• Option for new PluginManager to manage• DIP work ahead as well; may differ


D E M O (SCREENSHOTS)


OCW Course Rendered in Three Systems

OCW………….. DSpace ………….. SloanSpace …………………………….


Year One Demo: OCW to DSpace


OCW Course Website• Static website• HTML, PDF,

JPG, XLS• Akamai:

Multimedia• LOM XML• Search;

Feedback; Tracking

• Copyright cleared

http://ocw.mit.edu/OcwWeb/Sloan-School-of-Management/15-040Spring2004/CourseHome/index.htm


OCW Content Exporter (CE)

• CE generates entire course website (rewriting links)

• CE writes imsmanifest.xml

• CE publishes .ZIP to web page

• CE can publish whole dept.

• CE also used: translation; professors; etc.


Client: IMS-CP DSpace Import Driver

• OCW CE not yet WS client

• Simple CGI driver to DSpace session(), auth/auth(), upload(), and ingest() WS

• 4 params (w-i-p)

RESULTS: Success - Created DSpace item 123456789/75

Using collections='123456789/2’

Started session with token='2-1039ad464ee-7665e5a950ff7ff2'

upload response :: <?xml version="1.0" encoding="UTF-8"?>

<uploadservice><URI>http://rotarran.mit.edu:8080/dspace-ws/upload/package-21802.zip</URI><size>8309770</size> </uploadservice>

Ingesting IMSCP package in package-21802.zip

ingest response :: 123456789/75

Session ended"


DSpace Item Record for OCW Course

• DSpace info model maps “Item” to OCW “Course”

• Files (all types) are “Bitstreams”

• Metadata: basic LOM-2-DC

• Year 2: Further DSpace dev re: websites, LOs


DSpace Serving Full OCW Course• Static

website• HTML, PDF,

JPG, XLS• Akamai:

Multimedia• DSpace

search (Lucene)

• LOM XML to Dublin Core (w-i-p)

• OAI metadata distribution

• Copyright cleared, DSpace, CC licenses


Year Two Demo: Other CLE(early preview!)


SloanSpace Searching, Uploading OCW CoursePROTOTYPE

• SloanSpace (dotLRN, OpenACS) portal functionality

• Search DSpace (SRW)

• Retrieve via (provisional) DSpace WS

• Directly “Add Course” to dotLRN module for “Learning Object Repository System” (LORS)


SloanSpace LORS Navigation for OCW Course

PROTOTYPE

• dotLRN module, for IMS-CP standard

• LOM XML• (Use of

SCORM permits tracking, etc.)


SloanSpace LORS Serving Full OCW Course

PROTOTYPE

• HTML, PDF, JPG, XLS

• Akamai: Multimedia

• Lifecycle issues in Year Two work


Year Two Deliverables

• OCW Production– Operationalize to scale for OCW archiving (100

courses Fall 2005…)

• InterOperability with other CLE/LMSs– Package: “CWSpace IMS-CP Profile” for SloanSpace,

for Stellar, for Sakai

– Protocol: Web Services use cases, clients for same

• Archived websites– Improved contextual presentation

• Terminology Extraction Tool (CSAIL)– Explore integration into DSpace, OCW


CWSpace: Planned Work

• DSpace Platform– Plugin Manager

• Packagers (I/O)• Crosswalks

– Structured Metadata– Stackable

Authentication

• Protocols– Lightweight

Network Interface (LNI)

• SOAP & WSDL• WebDAV


Planned Work on CWSpace


Existing Interfaces to DSpace

• DSpace Web U/I• SRU/SRW Search• OAI-PMH• Command-line

– DSpace Batch Importer– Media Filters…

http://wiki.dspace.org/NetworkInterfaces

http://wiki.dspace.org/NetworkInterfaces


Lightweight Network Interface

• New Proposed Interface(s)– DSpace “Web Services”, for CWSpace…

• SOAP & WSDL• WebDAV (Extension to HTTP protocol)

http://wiki.dspace.org/LightweightNetworkInterface


A Few (Quick) Thoughts on

Web Services Design

• Standards-based vs. Custom model

• Abstract vs. Strongly modeled specification

• Technology Approaches (SOAP; REST…)

The next few slides provide a (very) brief tour of the topics we are investigating with our Lightweight Network Interface (LNI)


Approaches To Expose Your Object Model, 1

• ‘X’ axis: Degree of match to Standards– Custom

• PRO More control for service• CON Harder to interoperate for consumer

– Standards-based• CON Compromises for service• PRO Easier interoperation for consumer

In selecting an API or protocol or approach to exposing your object model to a network interface there are two axes to consider.


Approaches To Expose Your Object Model, 2

• ‘Y’ axis: Degree of precision in model abstraction– Abstractly Modeled

• PRO Open to extension, interpretation, wide application• CON You have a specification, but not implementation

guide

– Strongly Modeled• CON Compromises for service; line is drawn in the sand (!)• PRO Implementable directions; line is drawn in the sand (!)


Matrix: Expose Object Model


Technology Approaches

• SOAP, WSDL– Enterprise developers

– Contains hints re: objects, methods

• RESTful (“Representational State Transfer”)– Developers comfortable with XML markup, HTTP

– Straightforward XML messages over HTTP

• XML-RPC– Early, simplified spin-off from SOAP (ca. 1999)

• WebDAV (Protocol: extension to HTTP)– “Resources” and “Properties” works well with Repository

In then opting for a style of developing the services your Object Model will provide, there are some technology choices to consider.


Resources, Properties: WebDAV

DSpace (root) /mylib/dspace-ni/ Community /mylib/dspace-ni/dso_1721.1%2F46 Collection /mylib/dspace-ni/dso_1721.1%2F3549 Item /mylib/dspace-ni/dso_1721.1%2F5543 Bitstream

/mylib/dspace-ni/dso_1721.1%2F5543/bitstream_13 Workflow /mylib/dspace-ni/workflow/wf_23

The server only pays attention to the ''last'' element in a path of DSO's, e.g.

http://myserver/DAV/dso_123456789%2F1/dso_123456789%2F4/dso_123456789%2F13

is the same as

http://myserver/DAV/dso_123456789%2F13

http://wiki.dspace.org/LightweightNetworkInterface


Resources, Properties: WebDAV

Collection objectslogo; short_description ; introductory_text ; sidebar; copyright; default_license; provenance

Item objectssubmitter; owning_collection; license; cc_license; cc_license_rdf; DAV:getlastmodified

Bitstream objectsDAV:getcontentlength; DAV:getcontenttype; source; description; format-id; format-description; checksum; checksum-algorithm; sequence-id


Example GET URIs

/mylib/dspace-ni/dso_1721.1%2F5543?session=123xyzzy456&package=org.dspace.METS

/mylib/dspace-ni/dso_1721.1%2F5543/bitstream/13?session=123xyzzy456


Example PUT URITo add a new item to the collection at handle 1721.1/3549:

PUT /mylib/dspace-ni/dso_1721.1%2F3549?session=123xyzzy456&package=OCW-IMSCP ....package contents...

HTTP/1.1 201 OK Location: /mylib/dspace-ni/dso_1721.1%2F5549 ....other headers....


Features Matrix

• Comparing the Interfaces to DSpace with the Features they provide


Three Things to Take Awayfrom this talk on the iCampus Project: CWSpace

• We’re working on Packaging Metadata

• We’re working on Web Services

• We’re also working on Archiving Websites

... help us refine the thinking …



Thank You. Questions, Comments…

William Reilly, Larry Stone, MacKenzie Smith—MIT Libraries’ Digital Library Research Group (DLRG)

Rob Wolfe—MIT Libraries’ Metadata Services UnitCec d’Oliveira—MIT OpenCourseWare, Technology

http://cwspace.mit.edu

cwspace archiving mit opencourseware in dspace dspace federation 2nd user group meeting university...

Documents

mit icampus projectdspace

mit icampus projectcwspace

mit icampus projectmetadata

mit icampus projectimscp

ocw learning objects

new imscp

possible mets

xmlmets mets