[en] trends in records management, archiving and digital preservation | abbot | dr. ulrich...

U n t e r n e h m e n s b e r a t u n gD r . U l r i c h K a m p f f m e y e r G m b H

PROJECT CONSULT

PROJECT CONSULT Unternehmensberatung Dr. Ulrich Kampffmeyer GmbH

www.PROJECT-CONSULT.com© PROJECT CONSULT 2011

Breitenfelder Straße 1720251 Hamburg 1

Trends in records management, archiving and digital preservation

Wiesbaden, 13.09.2011

Dr. Ulrich Kampffmeyer

ABBOTT GmbH

Trends in records management, archiving and digital preservation ABBOTT GmbHDr. Ulrich Kampffmeyer 2

Agenda

1. Introduction

2. Records management is not archiving but needs archives

3. Archiving, long-term storage and digital preservation

4. Trends with impact on archiving, digital preservation and records management

5. Outlook

3Trends in records management, archiving and digital preservation ABBOTT GmbHDr. Ulrich Kampffmeyer

1 INTRODUCTION


Information overflow


2 RECORDS MANAGEMENT IS

NOT ARCHIVING BUT NEEDS ARCHIVES


Definition: Records Management

Field of management responsible for the efficient and systematic control of the creation, receipt, maintenance, use and disposition of records, including processes for capturing and maintaining evidence of and information about business activities and transactions in the form of records.


Records Management: ISO 15489

• Relevant for document management in the private and public sectors, and for archiving

• Definition of the importance, function and elements of document management systems

• The standard does not go substantially beyond a definition of the subject matter

• Lowest common denominator of the participating countries

• Problematic terminology in German translation, for example “Records Management” = “(paper) document management”


Not every record moves into the archive

Many documents

Some records

Few Archive objects


ARMA: GARP

Generally Accepted Recordkeeping Principles• Principle of Accountability

An organization shall assign a senior executive who will oversee a recordkeeping program and delegate program responsibility to appropriate individuals, adopt policies and procedures to guide personnel, and ensure program auditability.

• Principle of IntegrityA recordkeeping program shall be constructed so the records and information generated or managed by or for the organization have a reasonable and suitable guarantee of authenticity and reliability.

• Principle of ProtectionA recordkeeping program shall be constructed to ensure a reasonable level of protection to records and information that are private, confidential, privileged, secret, or essential to business continuity.

• Principle of ComplianceThe recordkeeping program shall be constructed to comply with applicable laws and other binding authorities, as well as the organization’s policies.


ARMA: GARP

Generally Accepted Recordkeeping Principles• Principle of Availability

An organization shall maintain records in a manner that ensures timely, efficient, and accurate retrieval of needed information.

• Principle of RetentionAn organization shall maintain its records and information for an appropriate time, taking into account legal, regulatory, fiscal, operational, and historical requirements.

• Principle of DispositionAn organization shall provide secure and appropriate disposition for records that are no longer required to be maintained by applicable laws and the organization’s policies.

• Principle of TransparencyThe processes and activities of an organization’s recordkeeping program shall be documented in an understandable manner and be available to all personnel and appropriate interested parties.


Standards

Records Management is governed by dozens of international and national standards.

A lot of these standards cover or influence as well archiving and preservation.


ICA – International Council on Archives

ICA-Req• Objective: to define global uniform principles and

functional requirements for software used to generate and manage electronic records

• Organized into three modules:• Module 1: Overview and Statement of Principles• Module 2: Guidelines and Functional Requirements

for Electronic Records Management Systems• Module 3: Guidelines and Functional Requirements

for Records in Business Systems• Based on various information elements

Relevant for archiving


MoReq2010

• Project to refine MoReq (Model Requirements for the Management of Electronic Records) by the DLM-Forum

• 8. June 2011: The DLM Forum released MoReq2010® Core Services Publication(MoReq now stands for Modular Requirements forRecords Systems)

• 14.July 2011: Release of the XML scheme

Relevant for archiving:The DLM Forum is a joint initiative of all European National Archives supported by theEuropean Commission


MoReq2010


MoReq Roadmap

Relevant for pharmaceutical:Special pharma MoReq specification planned


New version of the RM standards ISO

„Old“: ISO 15489

Stays, but …

„New“: ISO 3030x standards seriesISO 16175


ISO 30300 – Information and documentation

Content:1) Management standards for records2) Fundamental explanations3) Core principles of RM-Systems4) Terminology and definitions of RM-Systems

Fundamentals and vocabulary:Objective:

• Principals for using Records Management Systems (electronic and physical)

• Specification of process-oriented dispositions• Specification of the role of the “Top Management“• Definitions of the used terminology within the

ISO 30300


ISO 30301 – Requirements

Content:• Functional requirements for a RM-System• Requirements for a certification of the RM-System after ISO

30301

Requirements:Objective:• Specification of the requirements for a Records Management

System• Guidelines for a Records Management Policy for an enterprise• Support for achieving optimal enterprise goals by implemating

a compliant RM-Systems• Refers to all kinds of forms of organisations


New ISO standard ISO 16175-2.2

Information and Documentation

• Part 1: Information and documentation – Principles and functional requirements for records in electronic office environments

• Part 2: Guidelines and functional requirements for records in electronic office environments

Relevant for archiving:Originally developped by the ICA International Council on Archives


New ISO standard ISO 16175-2.2

Objective:• ISO 16175-2.2 references to Good – Practice-

methods and policies, risks and their reduction for:• The improvement of the Records Managements in

companies• The support of company requirements, leading to a

more effective and efficient operation• A Compliance supported, save operation• The development of key principals for a functioning

and automated RM-System• A common „language“ for provider and user


OMG Records Management

• Specification for Records Management Services, Submission to the Object Management Group

• Contains models for Software-Services• Area of application given by the "Platform

Independent Model (PIM)“, fundamental: OMG MDA Framework

• Three special technologic implementations:• PSM-1 – Web Services definition for Records

Management Services in Web Service Description Language (WSDL)

• PSM-2 – A Records Management Service XSD. • PSM-3 – An Attribute Profile XSD.


… and some defacto standards …

• SAP Records Manager

• Microsoft Sharepoint Records Center

Both with a different concept than „traditional” Records Management ...


RECORDS MANAGEMENT IN THEPHARMACEUTICAL INDUSTRIE


APPLICATION REQUIREMENTS


STANDARDISATION AND COMPLIANCE REQUIREMENTS


USA / international

• FDA Food and Drug Administration• CFR Code of Federal Regulations:

Title 21 Part 11 for Electronic Records and Electronic Signatures:• Defines the criteria under which electronic records

and electronic signatures are considered to be trustworthy, reliable and equivalent to paper records.

Different laws for electronic signatures may cause problems … i.E. in Nordrhein-Westfalen


Europe

• GxP quality guidelinesExamples:• Good Clinical Data Management Practice GCDMP• Good Clinical Practice GCP • Good Clinical Laboratory Practice GCLP• Good Hygiene Practices GHPs• Good Manufacturing Practice GMP• Good Microbiological Practice GMiP• Good Pharmacovigilance Practice GPvP • Good Pharmacy Practice GPP

GxP will become the guideline for the future “Moreq2010 for Pharma”


Legal requirements

• „Other“ consumer, healthcare, pharmaceutical and chemical

• Quality• Commercial, taxation• Patents

• Problems:Different regulations and laws in every country:• Research• Development• Production• Delivery• Channel


3 ARCHIVING, LONG-TERM STORAGE

ANDDIGITAL

PRESERVATION


An archive refers to a collection of historical records, as well as the physical place they are located. Archives contain primary source documents that have accumulated over the course of an individual or organization's lifetime, and are kept to show the function of an organization.

Archive

… is this definition correct for “archives” and “archiving” in the pharmaceutical industry?


Synonym or divergence? English vs. German

English GermanPreservation Archivierung?

Langzeitarchivierung?Digital Preservation Elektronische

Langzeitarchivierung?Archive Archiv?Trusted Archive Vertrauenswürdige

Langzeitarchivierung?Archiving Datensicherung!

Archivierung?Electronic Filing Ablage, Zwischenarchiv?


Digital Preservation

• Electronic archives for digital preservation allow database-supported access to longterm, unchangable archived electronic information.• Electronic archives are not data saving systems or

backup systems.

• In general, digital preservation serves to save any data, information, records, documents, content and other digital information objects long term, secure, authentic and undestroyable with direct but controlled access to every object in any context.


Standards Archiving

Archiving is standardized by ISO:

• ISO 17799: Information Security• ISO 14721: OAIS Open Archive Information

System• ISO 18492: Long-term preservation of electronic

document-based information • ISO/CD TR 26102: Information and documentation

Requirements for long-term preservation of electronic records


OAIS ISO 14721

OAIS-Reference Model for an Open Archival Information System

• 2002: Blue Book of NASA/CCSDS (650.0-B-1)• 2003 adopted as standard ISO 14721 • Especially intented for processing digital

information• Logically structured and therefore independent of

products and implementation• Authoritive in view of functionality as well as in

view of information flow


OAIS-Reference model



Problem area 1:Longterm stability and longterm availability of documentation of the system itself



Problem area 2:Interfaces



AIPDIP

SIP

Problem area 3:Container formats

AIP


PDF Standards

• ISO 15930 PDF/X Format for pre-press PDF/X-1a = ISO 15930-1 PDF/X-3 = ISO 15930-3

• ISO 19005 PDF/A PDF Archive Format for archiving documents PDF/A-1 = ISO 19005-1 PDF/A-2 = ISO 19005-2

• PDF/E PDF Engineering Format for technical drawings(in development)

• ISO 32000 PDF Standardized format for standard PDFPDF = ISO 32000-1


NESTOR

• Network of Expertise in long-term Storage and availability of digital Resources in Germany

• Cooperation partner from libraries, museums and archives

• Criteria catalogue to design, plan and build a trustworthy digital long-term archive

• Working Group on Trusted Digital Repository Certification

• Entered into the international standardization process via German ISO/DIN NABD planned standard

Relevant for Pharma?Nestor aims mainly for historic and cultural archives


NESTOR Certification

• Catalogue of Criteria and the Evaluation of Trusted Repositories:“Trustworthiness of a system means that it operates according to its objectives and specifications (it does exactly what it claims to do). From an information technology (IT) security perspective, integrity, authenticity, confidentiality and availability are important building blocks of trustworthy digital preservation repositories.”

Problem for testing and certification: …“operates according to its objectives and specifications” …


International Standards in the ERMS-environment

FEA Federal Enterprise Architecture

DRM Data Reference Model – Final Version 2.0

e-GMS UK e-Government meta data standard

DCMI The Dublin Core meta data initiative

AIIM Integrated EDM/ERM Functional Requirements

The National Archives (UK) Functional Requirements for ERM

Meta data standard

LMER Longterm archiving meta data for electronic ressources. Nestor project, 2007

ISO 15489, Teile 1 + 2 Information and documentation

PREMIS PREservation meta data: Implementation strategies

Examples for Metadata



Examples for MetadataFEA Federal Enterprise Architecture

DRM Data Reference Model – Final Version 2.0

e-GMS UK e-Government meta data standard

DCMI The Dublin Core meta data initiative

AIIM Integrated EDM/ERM Functional Requirements

The National Archives (UK) Functional Requirements for ERM

Meta data standard

LMER Longterm archiving meta data for electronic ressources. Nestor project, 2007

ISO 15489, Teile 1 + 2 Information and documentation

PREMIS PREservation Meta data: Implementation Strategies



MoReq Model Requirements for the Management of Electronic Records

ISO 23081, Part 1 Records management processes, meta data for records

Australian RKMS Recordkeeping meta data sheme

MARC Machine-Readable Cataloging

METS Metadata Encoding & Transmission Standard

DIRKS Designing and Implementing Recordkeeping Systems (Australia)

US DoD Dept. of Defense 5015.2-STD

Standard for Electronic Records Management

Examples for Metadata


Standards for Import, Export & Harvesting

• International:EAD – Encoded Archival DescriptionEAC – Encoded Archival Context

• Germany:SAFT – Standard Austauschformat für Findmittel

• Context plus Objects:METS - Metadata Encoding and Transmission Standard


GENERAL ARCHIVE INFRASTRUCTURE


The Vision of theUniversal Federated Archive

for every Type of Information

Archive Management Middleware


Archive Management Middleware

Federated Archive Repositories

Read only„historic“

Read + write„current“

Backup

Store Store

IndexDB

Archive Connectors

SAP Archive Link Application Interface Connectors CMIS

Workflow-Engine

OCR/ICR

LDAPAuthorities

Viewer

Manage-ment

Con-verter

Adminis-tration

UIDGenerator

Timestamp

SignatureGenerator

Audit trail

Recovery Load-balancer

SignatureVerifier

Configu-ration

DB

Termi-nology


Appliacation Interface Connectors

One Archive Management Middleware…

SAPE-Mail REOXML PDF FaxOfficeDigitalSignature

Archive Management

Archive Connectors

Lotus WebHTML

…to fit all kinds of information objects.


Application Interface Connectors


HGB AO GDPdU GAUFZ REACH Basel II GxP

§ § § § § § §SOX

§FDA

§

Archive Connectors

Archive Management

LSF

§ §…

…to comply with all legal requirements.



…for all applications.

SAP SAP Lotus Share-point

Federated Archive Repositories

Intranet

Archive ManagementMeta data

IndexDatabase

Spec

ial f

orm

ats

Portal …

no in

dex

data

base

m

etad

ata


CHALLENGES OF THE LAST DECADE


Scanning and archiving

• Indexing and classification are done mainly manual: • Bottleneck with lots of mistakes• Automatic classification helps but still has potential for

further improvement• Checking against existing master data supports good

quality and automated indexing• Quality assurance (readability, integrity) and

indexing are build into special capture workflows• Conversion into different renditions supports long

term accessibility of the documents• Certain applications and sectors need electronic

signatures in order that the scanned paper can be destroyed

• In the past mainly special archives for scanned documents (Imaging) only


Email-Archiving

• Lotus, Exchange/Outlook & Co.• Special formats• Attachments• Redundancy• and other problems

• Archiving strategies• Journaling – everything including spam• Individual – each user has the responsibility to archive

important mails in the business context• Automatic and semi-automatic support for both

strategies and hybrid alternatives• Mainly inhouse archiving solutions• Unclear legal value of an archived email• In the past mainly special archives for email only


SAP and archiving

• Control size and performance of the database• At the center of interest: data objects • Synchrone vs. asynchrone archiving

• Direct archiving or background processes• Database based vs. Database integrated

• Functionality drafted or integratedSAP: asynchrone, database based archiving

• Archive link: Communication interface among the applications, contents:• Interfaces• Repository• Research• Report

• In the past mainly special archives for SAP data and externally referenced documents only


Office documents and file systems

• Different formats: images, graphics, charts, texts, tables, media-objects etc.• Formats depend on program version• ODF is a standardized, open format, but there are

only a few ODF supporting providers• Office-Documents can be linked, embedded,

clustered and inside references can be lost• Outdated Office formats are not reproducible• Office Documents with own authorisation and rights

cause access and viewing problems in the archive• Users often do not add meta data and thus prevent

a controlled index for the archive• In the past mainly special archives for file system

replacements with office document formats only


E-Discovery only in the US?

• Enterprises who work together with an american enterprise or who trade directly or indirectly in the USA have to follow the rules of e-Discovery of the FRCP

• Contains evidence in legal disputs in the USA, informationen administrated in the abroad (e.g. Germany) can be relevant

• US administrations keep the right to search as well repository service in foreign locations (based on the patriot act which was prolongued by President Obama)

• Entrepreneurial risks as well for international companies with daughters and subsidiaries in the US as well for companies owned by US companies or having shareholders in the US


CURRENT CHALLENGES


Database archiving

• COLD-method only saves tables, lists or reports but not the logical connection of the database

• Saving the context is important for preserving a researchable and processable archived database

• Exporting the database or database application logic in extracts or archiving the complete database for (restricted) further use is typical for database archiving • Mainly used for systems which will be replaced but

have to be kept accessible (system retirement)• Tendency as well towards saving databases in

universal archives


Universal archives

• Principle: „One archive for all kinds and types of information managed together following the same criteria and principles“• In the past different types of documents and the

connection to different application systems made it necessary to use isolated solutions or silo solutions

• ECM Enterprise Content Management supports the idea of universal archives with a “federated repository“ concept

• Archiving in the World Wide Web with its diversity of formats and utilization models requires as well an „Universal archive“ concept


Archiving for web content and transactions

• Archiving websites and portals with business relevance content • Archiving the transactions of users of such websites• Often only saving the data and the screens seen by the

user• More complex but more complete the documentation of

the complete situation, related data and journal of the process

• Personalization, individualization and automatic updates make archiving of websites difficult

• Archiving information from social communities, special formats, Enterprise 2.0 portals, social and short messaging systems, videos, multimedia objects, 3 dimensional objects, tweets, … cause growing problems without solutions today


Email-Archiving in a Social World

• Google Mail• Google Mail-Archiving / Postini• Classic „Mailbox problem“

• Facebook „Mail“ and Messages in Social Communities• No more sending mails but messages as views on

database entries• All messages kept in a central database

• How to archive important business correspondence from these channels and databases?

• And … can I trust the provider of the systems in regard to confidentiality and privacy?


Archiving for mobile appliances

• Mobile devices have no individual archive memory• Special service providers (Google, Apple) cache

communication and documents whith direct access on their servers, but not in their archives; special archive offerings like iCloud, Amazon S3 to come

• To control the information flow gains more and more importance and may proscribe to use mobile devices for certain application areas

• Client-/Server-oriented applications are not sufficient to support modern mobile communication in regard to managing and archiving information

• Archiving mobile communication mainly takes place in the CloudProblem: Governance and legal requirements


Archiving in the Cloud

• A question of trust• Legal requirements• Archiving with SaaS Software as a Service needs

special functional requirements:• Special invoicing modules, secure authentification,

response systems, logging, interfaces to integrate local software, backup and softstandby for a large amount of data, special prefetch-strategies, preview- converter, legal hold, HSM, a.s.o.

• Need for an univeral archive since there is no control of what information objects are uploaded and used


Archiving of „special“ Emails

There are two „very special“ email offerings in Germany:• DE-Mail

• Medium of communication to exchange electronic documents in a reliable and confidential way via internet

• Project of the German Federal Goverment with the collaboration of several service providers

• Goal: Reduction of replacement costs for administration in enterprises

• Based on the German De-Mail law• E-Postbrief

• Offering by Deutsche Post AG• Hybrid mail service with integrated website to exchange messages

via internet• Goal: higher authentication, superior data protection and greater

integrety in comparaison with a “normal“ E-Mail• Today not compliant with the De-Mail law

• General question: keep the De-Mails/E-Postbriefe with the provider (only) or transfer them (additionally) into an inhouse archive to combine them with the other documents of the business case?


4 TRENDS WITH IMPACT ON

ARCHIVING, DIGITAL

PRESERVATION AND RECORDS

MANAGEMENT


… AND A DIFFERENT PERCEPTION OF ARCHIVING …


COLLABORATION GOES MOBILE AND SOCIAL

• A new direction: Collaboration develops under the pressure of social software, mobile apps and Web 2.0• „2.0“-like tools replace traditional groupware-

functionality• Present inhouse applications are now offered as

SaaS: Archiving, E-Mail-Management, Email, CRM, Project Management …

• Major suppliers like Facebook, Salesforce.com, Microsoft, Google etc. use collaboration & communications as a first step into SaaS enterprise software• The goal: no longer inhouse implementations of

enterprise software


THE MOBILE REVOLUTION

• Every future system has to ensure that not only professionals can use it but also the average consumer .

• In the future, the way to operate a TV, a mobile phone or a stereo equipment at home will be standard as well for business applications.

• Also ECM services like email archives, electronic records and workflow management can be used trough Apps.

• Many ECM provider have already Apps for smart phones, tablets, netbooks i.E.

Still the question remains wether mobile devices are fit to serve professional working with documents, folders, and complex processes.


Cloud Computing Chances & Risks

Chances Risks &• Provider

• Confidence• Control• Dependance

• Reliability of the Cloud• Shortages in the datatransfer• Software-Licenses• Property rights

• Faster availability of innovations• Theoretically unlimited scalability• Cost-saving concerning hardware, employees and licenses• Reducing efforts regarding program updates• Faster adaption to changing requirements• Cost-control with Pay-for-use

Redundant, 24/7/365 archiving with access from any where at any time is an issue ...


Systems of Record vs. Systems of Engagement

Systems of Engagement – like collaboration - are gaining ground while systems of record - like archives – are loosing attention


Social Business Conclusion according to AIIM

• Social business technologies offer significant benefits to organizations that can effectively implement and use them.

• This will frequently require reexamination of existing processes to determine how best to take advantage of social technologies.

• Organizations should review these technologies with an eye towards implementing them in a way that supports their business goals and objectives and that complies with their unique governance requirements.

Social Software and Enterprise 2.0 Applications are not designed for Records Management and Archiving requirements …


Ubiquitous Computing

• Vision of small, inexpensive, robust networked processing devices, distributed at all scales throughout everyday life and generally turned to distinctly common-place ends• Human-computer interaction model• The use is intransparent

• “Using" ubiquitous computing engages many computational devices and systems simultaneously, not necessarily being aware that they are doing so• Only a vision: “Machines that fit the human

environment instead of forcing humans to enter theirs"

Archiving becomes a backbone and invisible service


Everything moves under the surface

• “Library services such as version control and checkout/checkin are happening behind the scenes.“

• “Cloud-based storage and automatic synchronization services enables seamless working across any device.”

• “ Nothing is deleted. Ever.“

Oscar Berg, 2011


GRC Governance, Risk Management & Compliance

Compliance• Compliance describes the entire measures to justify

the rule-consistent behavior of a company, their organisational member and their employees concerning every legal command and prohibition.

• Compliance with and fulfillment of legal and regulatory requirements.

„Compliance alone is not enough, it has to be integrated in a framework with Governance on the high level, Risk Management and Compliance on the operational level.“


GRC

GRC brings together the disciplines of Corporate Governance, Risk Management and Complianceas a consistent procedure model.


Compliance: General criteria

• Authenticity• Completeness• Traceability• Access security• Structure • Integrity• Findability• Reproducibility• Unchangability• Accuracy• Verifiability• Portability• Trustworthiness

These are as well the relevant criteria for

Records Management and Archives


Compliance: General criteria

• Authenticity• Completeness• Traceability• Access security• Structure • Integrity• Findability• Reproducibility• Unchangability• Accuracy• Verifiability• Portability• Trustworthiness

Compliance may be the only reason for having Records Management

and Electronic Archives in the future …


5 OUTLOOK


A CHANGE OF PROFESSION


Digital archivists

• Gartner expects around 15 per cent of companies to add a digital archivist role by 2012 compared with fewer than 1 per cent in 2009

“Organisations typically have vast quantities of records, which require specialist expertise to access, appraise and preserve. This isn’t a job for conscientious users to perform if they have time; it requires training and expertise. […] you need a digital curator.”


THE OTHER VIEW OF ARCHIVING


The Loss of Control

• Information flood: enterprise search versus controlled organization of information:

Why organize information if Google on my Desktop can search them easily for me out of the big data?

• More and more „easy use“ concepts from web 2.0 and mobile devices:

Why use tree views and folder structures to organize records and documents? We need features like Youtube!

• More and more people relying on automation:Why index manually and order manually when automatic categorization can do this? Otherwise social tagging can help!


Responsibility for Archiving

“Electronic Archives are the Memory of the Information society.“

Erkki Liikanen, Commissioner for information technology of the European Commission, 1999

U n t e r n e h m e n s b e r a t u n gD r . U l r i c h K a m p f f m e y e r G m b H

PROJECT CONSULT

PROJECT CONSULT Unternehmensberatung Dr. Ulrich Kampffmeyer GmbH

www.PROJECT-CONSULT.com© PROJECT CONSULT 2011

Breitenfelder Straße 1720251 Hamburg 101

Thank you for listening!

Ulrich KampffmeyerE-Mail: [email protected]

Further information:www.PROJECT-CONSULT.com