1. 'interoperability. a quick chat, a few war stories'. carl wilson, open planets...

Interoperability

A quick chat, a few war stories...

The OPF teamCommunity Manager● Events (face-to-face/virtual)● Training (staff development)● Comms (web/email/social)

Executive Director● Membership (engagement/value)● Open preservation advocacy● Operational management

Technical Lead● Infrastructure (host/test)● Software stewardship (roadmap/maturity/packaging)● Data corpora

About OPF

Trying to help institutions create robust digital preservation workflows by:

● sharing knowledge and best practise;

● improving the quality of digital preservation software.

Introduction

Here to set the scene by:

● Defining interoperability

● Providing a personal perspective

● Covering a few engineering fundamentals

● Looking for an easier way

Interoperability

“is the ability of making systems and

organisations work together (inter-operate).”

straight from Wikipedia

Origins?

Coined in IT and system services to allow for

information exchange.

But it’s not just an IT thing

A broader definition also considers:

● social issues● political & organisational factors

that “impact system to system performance”

Another “definition”

“Task of building coherent services for users

when the individual components are technically

difficult and managed by different

organisations.”

Wikipedia again

Who’s in the interoperability game?

● Information professionals● Software engineers● Staff who work on distributed projects● People dealing with complexity

Sound like anyone we know?

So those of us lucky enough to work in IT for Information professionals on collaborative projects it’s welcome to interoperability X 3…..

It’s an older problem than IT

=?

Old and international...

But IT does it better...

The UK NHS National Program for IT

● biggest civilian IT project of its kind● “mashups” for health records ● in theory 2003 - 2007 costing £2.4 billion● in reality 2003 - 2011 costing £13 billion● incidentally it didn’t work…….

But it’s possible in real life

Think of a system that is:

● decentralised

● fault tolerant

● built on simple building blocks

And it’s possible in IT…..

What would that look like?

Things to think about

● simplicity

● standards

● clarity

● test early

● test often

Small is beautiful

● small parts can build beautiful robust wholes

● it’s one way of tackling complexity

● BUT the parts need testable, tested and reliable

Standards are your friends

It’s always worth checking if there’s a standard

available and in use but:

● there’s often more than one

● they’re not always easy to understand

● they’re hostage to commercial interests

Be clear in your intentions

Again simplicity / small size helps here

● Document clearly and publicly● Keep it up to date● If you’re not willing to explain how it works….

Test early and often

And test the parts together rather than as well as separately

Technologies to automate testing:

● virtualisation● online public continuous integration

Interoperability is the future

The Internet of Things is gathering pace

Information exchange happening between more and more devices, more regularly.

REST, XML, Json

But it’s NEVER finished

Two aspects of interoperability:

● Syntactically interoperable● Semantically interoperable

That’s to say speaking the same language is one thing, understanding what the other is saying is another…..

Why data is a special problem

In the cultural heritage sector

● shared data == shared problems● legal issues (IPR) means we don’t share as we

should

Moving the mountain

Moving the large volume takes way too long….

● If I want to move > 5 TB for processing● THEN move the results back…..

A better way…..

It’s now MUCH easier to move the application to the data:

● It’s more secure● virtual machines are approx. 300 MB● Docker - application containers

So the oldest interoperability story?

● It might not be the oldest but it’s close,● about old school integration issues.● Something of a cautionary tale,● it didn’t end well…..

The Tower of Babel

My Details

● Carl Wilson ● Technical Lead● Open Planets Foundation● Email : carl@openplanetsfoundation.org● Skype : carl.f.wilson● GitHub : carlwilson● Twitter : @openplanets● Google+ : carl@openplanetsfoundation.org