©2013 Waters Corporation 1

Long Term Archiving, Data Retention/Legal Hold in NuGenesis SDMS

©2013 Waters Corporation 2

Agenda

Data Retention requirements and how they can be addressed

Long term data archival (LTA) – data formats

NuGenesis SDMS interface with LTAs

Storage Gateway interface solution

©2013 Waters Corporation 4

Typical Labinformatics Scenario

DISCOVERY DEVELOPMENT QC

ERP

LIMS B

LES

SDMS

CDS 2

Instrument

LIMS A

ELN B

SDMS

CDS 1

Instrument

SDMS

MS Software

Instrument

ELN A

©2013 Waters Corporation 6

Sampling Analytical Data Acquisition

Data Processing Results

File Data Printed Results

NuGenesis SDMS Repository

Development & QC

Long Term Archive

LIMS

©2013 Waters Corporation 7

Regulatory Requirements

Examples – GxP demands archival of records for extended time

– Reprocessibility

– What are your retention requirements?

Industry Segment, Regulator and Type of Record

Typical Retention Period

Pharma: Good Laboratory Practices; records related to a new drug application (NDAs)

Date of submission plus 5 years

Health Care: medical records Life of patient plus “n” years

Drug/device study records Marketing application plus 2-3 years

Government records 20-50 years, or permanent

Copyright records (all organizations) Life of copyright = 95 years or as business needs dictate

Patent records and supporting data Application plus 17 years

©2013 Waters Corporation 8

Lifespans

Software lifetime: approx. 9 years*

Hardware depreciation: 3-5 years

Comparison of reliability over three years

*Soft

war

e lif

etim

e an

d its

evo

lution p

roce

ss o

ver

gen

erat

ions,

Tam

ai,

T., To

rim

itsu

, Y.

, Confe

rence

on S

oft

war

e M

ainte

nan

ce,

1992,

Proce

edin

gs

©2013 Waters Corporation 10

History

©2013 Waters Corporation 11

Common Strategies

Keep old hardware – Hardware failures

– Lack of spare parts/maintenance

– Lack of knowledge

Virtualize old software – Lack of knowledge

– No updates

Convert data – Either updated proprietary formats/databases

– Or platform independent standards

– Processing Algorithms...

– Original raw data must still be retained!

©2013 Waters Corporation 12

Current Data Standards

JCAMP 1988

today

Andi/AIA

1992/93 2001

AnIML

2013/14

©2013 Waters Corporation 13

Migrate to new standards

JCAMP ANDI AnIML

JCAMP ANDI

+ + /

today tomorrow

©2013 Waters Corporation 14

PDF/A

ISO standard

100% self-contained

Based on Adobe PDF Reference 1.7

Specific features either mandatory, recommended, restricted, or prohibited

Several parts: – Part-1 (PDF/A-1) approved September 28, 2005

– Part-2 (PDF/A-2) approved June 20, 2011

o PDF/A-1 remains in effect

o PDF/A-1-conformant files also conform to PDF/A-2

– Part-3 (PDF/A-3) approved October 18, 2012

©2013 Waters Corporation 15

Long Term Archive Repository (DB + Managed Store)

Keep data secure

Observe retention periods – Protect against premature deletion

Make available on request

Repository vs. Archive Long Term Archiving

Data collection

Print & file data

Metadata extraction

Share data in community

Simple retrieval

Data re-use

Header Data

Analytical Data,

Binaries

LT Archive Repository Database

Archive

Record Data

Restore

Search

©2013 Waters Corporation 16

Data Retention

Data Retention can be enabled on a project – Cannot be turned off

– No manual deletion or moving possible through VISION or SDK

– Governed by Retention Policies

– DR-enabled projects cannot be deleted!

©2013 Waters Corporation 17

Data Retention Policies

1. Define Candidates – User needs “Create Data Retention Policy” and/or “Edit Data

Retention Policy” privilege to create/edit policies

– Candidateship defined by

o Filter criteria

o Search criteria

– Dynamically takes care of legacy and new records

– May overlap

TIP!

o Avoid cookie-cutter criteria

o If you have multiple record types in a project, make sure you carefully define the selection criteria

©2013 Waters Corporation 18

Data Retention Policies

2. Define Retention Period – Retention defined by

o Fixed future date

o Days/months/years after date-based tag field

• Date format must be yyyy-mm-dd!

o Event (future event + x dd/mm/yyyy)

• Events are triggered manually

NOTE:

o Multiple policies possible on the same project

• Longest period wins

©2013 Waters Corporation 20

Data Retention Policies

©2013 Waters Corporation 21

Data Retention Policies

3. Define Deletion – Automatic deletion

o Configure Data Retention Service

– Manual deletion

o With or without RFC/reauthentication (project setting)

o User needs “Delete Reports” Privilege

– Fixed Date = Last Day of Retention Period

o Deletion happens 1 day after

– Email (per policy)

o Auto Delete: Deleted Records

o Manual Delete: Candidate Count

©2013 Waters Corporation 22

Data Retention Policies

Manual or automatic deletion after expiration – Deletion of metadata, binary data, and record audit trail

– From DB, Managed Storage, and OSM (HD, Raid, HSM)

– The act of deletion is logged in the system audit trail

Creation/Editing of policies is audit trailed

Changes to retention time will pop up a warning

©2013 Waters Corporation 23

Manual Deletion

©2013 Waters Corporation 24

DRS

DR-enabled Projects

(for user)

Selected Projects will be scanned

©2013 Waters Corporation 25

Legal Hold

Will prevent:

– Changes to the record

– Changes to the metadata

– Deletion

– Data Retention Management

Allowed:

– OSM

– Data Management

Privilege controlled

Multiple legal holds possible

Across Projects

©2013 Waters Corporation 26

Legal Hold

©2013 Waters Corporation 27

Storage Gateway for OSM

Waters Storage Gateway – paid option (new for NuGenesis 8)

One interface to multiple storage solutions – Tested and supported for SDMS: EMC² Centera

Can be configured as an OSM Device – For Data Retention enabled projects ONLY

– SDMS Data Retention is supported

Database

OSM

Storage Gateway

©2013 Waters Corporation 28

Waters Storage Gateway Integration

SDMS OSM WSG

send data send data Store + lock data

success

status

check status

status

SDMS OSM WSG

delete data delete data Unlock +

remove data

success

status

check status

status

OSM to Centera:

Data retirement:

EMC² Centera

EMC² Centera

©2013 Waters Corporation 29

Waters Storage Gateway

©2013 Waters Corporation 30

Why a Gateway Application?

Chromatography Data System

Other Analytical Systems

NMR

Office Apps

LTArchive 1

ELN

LIMS

Repository 1

Repository 2 LTArchive 2

©2013 Waters Corporation 31

Why a Gateway Application?

Chromatography Data System

Other Analytical Systems

NMR

Office Apps

LTArchive 1

LTArchive 2

ELN

LIMS

Repository 1

Repository 2

New LIMS

Individual data

©2013 Waters Corporation 32

Origin System

Storage Gateway

LTA Brand 1

Repository 1

LTA Brand 3

Storage Gateway

Ori

gin

Sys

tem

In

terf

ace

(API

) Archive Adapter 1

Archive Adapter 2

Archive Adapter 3 LTA LTA Brand 2

LIMS

Repository 2

NuGenesis 8 SDMS

AWM

©2013 Waters Corporation 33

Architecture

Storage Gateway

SG Server and

Actback

Archiving Backend

SG Database

Web Client Application

Administration & Queries

©2013 Waters Corporation 34

Archive Adapters

Are specific to an LTA Architecture – Are separate add-on modules

– Use an LTA programming interface where available

– Do not necessarily support the full LTA function set

Currently 3 available – File system

– IBM Tivoli

– EMC Centera (supported by SDMS)

Archive Adapter

LTA

SG Server

LTA API

©2013 Waters Corporation 35

Supported Archiving Functions

As of Storage Gateway 2.0 SP3:

Store data – using pre-defined Storage Classes/Pools

– with individual retention times

– with event-based remaining retention time

Restore data

Lock data against deletion – Separate from retention times or Storage Classes

Delete data – If retention period is over

– If not locked

©2013 Waters Corporation 36

Functions

System Administration – All via web client

– Mapping Origin System to Archive Setup

– Mapping OS Archive category names to Archive storage categories

User management

Data audit trail – Audit trail search and review

Storage Gateway metadata configuration – SG stores metadata for each archived set of data

Fallback data search and retrieval

©2013 Waters Corporation 37

Origin System

Storage & Retrieval

LTA 1

Repository 1

Storage Gateway

LTA 2

NuGenesis 8 SDMS

AWM

Database

SDMS -> LTA 1 ID 123abc

LTA def345

SDMS ID 123abc LTA def345 Meta data Audit trail

©2013 Waters Corporation 38

Origin System

Storage & Retrieval

LTA 1

Repository 1

Storage Gateway

LTA 2

Database SDMS ID 123abc LTA def45 Meta data Audit trail

Web Client Application

©2013 Waters Corporation 39