the planets testbed dpe, planets and caspar 2nd annual conference lisbon, 5–6 september, 2007 max...

The PLANETS Testbed

DPE, PLANETS and CASPAR 2nd Annual ConferenceLisbon, 5–6 September, 2007

Max Kaiser, Austrian National [email protected], http://www.onb.ac.at

mailto:[email protected]

http://www.onb.ac.at/

http://www.onb.ac.at/

Max Kaiser (Austrian National Library, Vienna): The PLANETS TestbedDPE, PLANETS and CASPAR 2nd Annual Conference, Lisbon, 5–6 September, 2007

ToC

Why do we need Testbeds in Digital Preservation? What will the PLANETS Testbed deliver? Role of Testbed within PLANETS PLANETS Testbed services for Digital Preservation

community Testbed Methodology, Experiment 6-step Process Testbed Software Application Testbed Components User Interface Mock-Up Demonstration Next Steps


Why do we need Testbedsin Digital Preservation? Concern about digital preservation rapidly increased

in recent years Several approaches to digital preservation

implemented by various programs and projects A number of digital preservation tools and services

(e.g. for migration and characterization) developed

Need for systematization, benchmarking and evaluation urgent in many fields:

Systematic analysis of current and potential preservation strategies

Evaluation and benchmarking of characterisation and migration tools

Assess the role of emulation tools …


Why do we need Testbedsin Digital Preservation? If we want to

perform scientific research in digital preservation evaluate preservation approaches in diverse “real life” settings avoid duplication of work maximise use of invested resources

we need a dedicated research environment, that allows the systematic execution of experiments by different

institutions with experiments that

• follow a formalized methodology• are reproducible• are fully documented and accessible for further analysis and

comparison


A Testbed for Digital Preservation

Definition:

“A controlled environment for experimentation and evaluation, with metrics and benchmark content that allow comparison of preservation tools and strategies”


PLANETS Testbed

Platform for objective evaluation of preservation tools and strategies: Load content Perform digital preservation experiments:

• Design the experiment, select data and tools, specify resources and evaluation criteria, run the experiment, evaluate results against criteria, compare outcomes

Find support for validation of preservation strategies Benchmark tools and services

Will consist of Hardware, data storage, PLANETS software, Testbed application Benchmark and other content

Will provide services for The PLANETS project partners The preservation community Tool developers and vendors


Role of Testbed in PLANETS | 1

Test and validate the technical solutions and approaches developed in PLANETS: provide a controlled hardware and software environment for

testing and evaluating preservation action (migration, emulation) and characterisation tools and services

record experiments data in registries for further analysis and comparison

assist the validation of the effectiveness of different digital preservation plans


Role of Testbed in PLANETS | 2

Assess the suitability of the approaches across „real life“ scenarios in various organisations Analyse applicability of the outcomes of PLANETS in

existing workflows and organisational contexts Evaluate their efficiency in providing practicable solutions for

organisations engaged in digital preservation


Role of PLANETS Testbed in theDigital Preservation Community In a second phase, the PLANETS Testbed will

Offer services to organisations outside PLANETS:• Support institutions to test preservation tools and

services against benchmark content• Assist institutions to validate their preservation plans

against their policies & content profiles Enable developers and third party vendors to submit tools for

benchmarking and certification:• Validate the suitability of their tools in preservation

workflows


PLANETS Testbed: Foundations

Build on work undertaken by the Dutch Digital Preservation Testbed and DELOS Testbed Framework

Integration in PLANETS Interoperability Framework Focus on formalisation of testing process design Strong focus on comparability and traceability of

results


Main Participants

Austrian National Library Humanities Advanced Technology and Information

Institute at the University of Glasgow (HATII) Austrian Research Centers (ARC) National Archives of the Netherlands British Library Vienna University of Technology University of Cologne


Testbed Methodology

Experiment Process: Refined to a focused 6-step process that each formal

experiment must follow

PLANETS Testbed experiment: consist of a consistent methodology for Defining its metadata (exploration area, scope, criteria,

references etc.) Selecting or creating a workflow of (orchestrated)

preservation services Evaluating the results against the experiment’s initial goals


Basic Experiment Properties Definition

Basic Experiment Properties Definition

Tools and Data Specification

Tools and Data Specification

Declaration ofGo / No go

Declaration ofGo / No go

2. Design Experiment

2. Design Experiment

3. Specify Resources &

Outcomes

3. Specify Resources &

Outcomes

4. Go /

No go

4. Go /

No go

StartStart 1. Define Basic Experiment Properties

1. Define Basic Experiment Properties

Evaluation Checklist

Evaluation Checklist

Resource Specification

Resource Specification

• Name• Reference to other

experiments• Summary• Purpose• Participants• Scope• Type of Tools• Type of Objects• Approach• …

• Name• Reference to other

experiments• Summary• Purpose• Participants• Scope• Type of Tools• Type of Objects• Approach• …

• Specify Experiment Type• Migration• Characterisation• Workflow

• Specify Tools• Specify Data

• Specify Experiment Type• Migration• Characterisation• Workflow

• Specify Tools• Specify Data

• Estimate number of output files

• Estimate computer resource requirements

• Specify benchmark goals for experiment

• Estimate number of output files

• Estimate computer resource requirements

• Specify benchmark goals for experiment

• Automatic approval by system• Based on resource

specification and system monitoring

• Manual approval by administrator

• Automatic approval by system• Based on resource

specification and system monitoring

• Manual approval by administrator

Testbed Experiment Process | 1


Experiment Workflow

Experiment Workflow

Experiment Results

Experiment Results

Experiment Evaluation

Experiment Evaluation

6. Evaluate Experiment

6. Evaluate Experiment

StopStop

4. Go /

No go

4. Go /

No go

5. Run Experiment

5. Run Experiment

• Experiments executed as BPEL workflows

• Processed by workflow execution engine of the Interoperability Framework

• Presentation of status report

• Experiments executed as BPEL workflows

• Processed by workflow execution engine of the Interoperability Framework

• Presentation of status report

• Presentation of input and output data

• Evaluation of benchmark goals

• Presentation of input and output data

• Evaluation of benchmark goals

Testbed Experiment Process | 2


Testbed System Design

Initial understanding of the required characteristics of PLANETS Testbed derived from the Dutch Preservation Testbed and the DELOS Digital Preservation Cluster

Additional requirements gathered following a clearly structured software design process Interviews User Scenarios ~ 60 Use Cases Software Requirements

• IF Requirements (Authentication Interfaces, Logging, Versioning…)

• Perform Experiment, Define Basic Properties, Select A Tool, View Available Services, Handling Stalled Experiments…

Non-functional requirements• Usability, Reliability, Performance, Security, Supportability…

Design and Implementation Plan• High-level design of the Testbed application • Initial breakdown of the system into components


Testbed Software

Platform-independent system based on Java Enterprise Edition (EE)

Three-tiered system:

Handles presentation of user interfaces; logic for dynamically creating content. Based on Java

Server Faces and Facelets

Provides business logic (core functionality). Based

on JavaBeans and JAX web service endpoints

Provides data persistency. Uses data registries provided by PLANETS Interoperability

Framework


Interoperability Framework Service Bus

Security/Authentication/ Authorization

Monitoring/Logging/Auditing

Workflow Execution Engine

Error/Exception Handling

Transaction Manager

Database Layer

WorkSpace

Service Registry

RegistryServicesRegistry

Services

IF Applications

Wo

rkflow

De

sign

er

Ad

min

istra

tion

T

oo

l

Integrate

System Monitoring Client

Persistence

Scheduling Approval

TB Service RegistryClient

TB Data RegistryClient

Testbed Backend Components

Use

Configure

Testbed Application

Experiment

De

sign

er

Eva

lua

tor

Im/E

xpo

rter

Data

Data

Manager

Services

Service

Manager

GeneralAdm

inistration

Bro

wse

r


Experiment Designer component | 1

Testbed Application

Experiment

De

sign

er

Eva

lua

tor

Im/E

xpo

rter

Data

Data

Manager

Services

Service

Manager

General

Adm

inistration

Bro

wse

r



Guides the user trough design of an experiment User Interface for steps 1–3 of the experiment

process Define basic properties Design experiment Specify resources

Select whether experiment is formal or informal Select type of experiment

Migration Characterisation Emulation Preservation plan evaluation Execution of a predefined workflow



Interaction with several other Testbed components, most notably the service manager and data manager

Interaction with Service Manager Browse PLANETS Service Registry Selection and configuration of (wrapped) tools for

Preservation Action (PA) and/or Preservation Characterisation (PC)

„Service“: PA/PC tool exhibiting well-defined functionality through a PLANETS IF compliant web service wrapper

Interaction with Data Manager Interface for handling data input and output via API for

accessing the IF Data Registry



Specification of Resources E.g. number of output files

Specification of Outcomes of an experiment Will appear at the evaluation stage of experiment

Experiment stored as BPEL file and handed over to Approval component

Additional features: Loading existing experiments Viewing of experiments in “passive” mode (read-only) Commenting on experiments Re-use and Repeating experiments

• Re-use certain aspects of existing experiments, or• Repeat entire experiments with original data



Rendering of BPEL workflows All experiments are executed as BPEL workflows “Simple” migration and characterisation experiments:

• BPEL files are created within Testbed based on templates

• Up to three orchestrated services (e.g. characterisation followed by migration followed by characterisation)

Complex workflows are created by using the IF Workflow Designer (or any other BPEL tool) and imported into the Testbed

Component does not process the actual execution of an experiment! Handled within the PLANETS Interoperability Framework by

the Workflow Execution Engine


Experiment Approval & Scheduling | 1

System Monitoring Client

Persistence

Scheduling Approval

TB Service RegistryClient

TB Data RegistryClient

Testbed Backend Components

Testbed Application

Experiment

De

sign

er

Eva

lua

tor

Im/E

xpo

rter

Data

Data

Manager

Services

Service

Manager

General

Ad

min

istratio

n

Bro

wse

r


Experiment Approval & Scheduling | 2

Approval component handles technical approval of experiments by the system and/or the administrator Makes use of information provided by the experimenter and

by the system monitoring component – e.g.:• Number of input files• Estimated number of output files• Current CPU / Memory usage• Threads / Processes• Available Disk Space

Scheduling component allows administrator to Select a date/time for an experiment to start Assign a maximum run time Check progress of experiments Stop experiments


Experiment Evaluation | 1

Testbed Application

Experiment

De

sign

er

Eva

lua

tor

Im/E

xpo

rter

Data

Data

Manager

Services

Service

Manager

General

Adm

inistration

Bro

wse

r


Experiment Evaluation | 2

View files and results that have been produced during the experiment

Capture experiment conclusions in a structured way Evaluate results based on the criteria defined during

experiment design process


Experiment Importer / Exporter | 1

Testbed Application

Experiment

De

sign

er

Eva

lua

tor

Im/E

xpo

rter

Data

Data

Manager

Services

Service

Manager

General

Adm

inistration

Bro

wse

r


Experiment Importer / Exporter | 2

Create BPEL files from existing experiments and make them available for download

Upload BPEL files to the Testbed Import whole experiments Import predefined workflows as part of the experiments

design process

Check validity of the BPEL file structure Check data contained within BPEL files Download experiment results

Information on experiments in PDF format created on-the-fly• Basic properties, tools used, data used, information

about approval process, information about execution of experiment, experiments results, validation of experiment

local demo WWW demo

http://127.0.0.1:4001/testbed/

http://www.planets-project.eu/private/testbed-ui/

http://www.planets-project.eu/private/testbed-ui/

Browse Experiments

Browse Experiments

Browse ServicesBrowse Services

My Experiments

My Experiments

New Experiments

New Experiments Browse DataBrowse Data Import

ExperimentsImport

Experiments

Login / LogoutLogin / Logout HomepageHomepage

View Experiment

Details

View Experiment

Details

Annotate ExperimentAnnotate

Experiment

Export Experiment

Setting

Export Experiment

Setting

Experiment Designer

Experiment Designer

Upload DataUpload DataRun

ExperimentRun

Experiment

Evaluate Experiment

Results

Evaluate Experiment

Results

Edit Conclusions

Edit Conclusions

View Metadata

View Metadata

ExperimenterPages


Conclusions / Next Steps

PLANETS Testbed: controlled environment for experimentation that allows benchmarking of preservation tools, services and strategies

First release to be expected in February / March 2008 Will be used to experiment with the preservation tools and services

developed in PLANETS In a 2nd Phase (by end 2008) Testbed will offer services to

organisations outside PLANETS: Support institutions to test preservation tools and services against

benchmark content Assist institutions to validate their preservation plans against their

policies & content profiles Main Testbed Instance hosted by University of Glasgow (HATII)

• Dedicated hardware with sufficient processing power and storage space

The PLANETS Testbed

Max Kaiser, Austrian National [email protected], http://www.onb.ac.at

Thank you!

Questions?

the planets testbed dpe, planets and caspar 2nd annual conference lisbon, 5–6 september, 2007 max...

Documents

planets testbed dpe

role of planets testbed

planets testbed platform

planets software

outcomes of planets

digital preservation

role of testbed

preservation approaches