yokogawa electric corporation exaquantum/batch validation planning guide copyright © yokogawa...

74
Yokogawa Electric Corporation Exaquantum/Batch Validation Planning Guide Copyright © Yokogawa Electric Corporation 1 st Edition April 18, 2005 Exaquantum/Batch Validation Planning Guidance (PM36J04B90-01E_001) Industrial Automation Systems Division Exaquantum/Batch Validation Planning Guide

Upload: dorthy-taylor

Post on 30-Dec-2015

255 views

Category:

Documents


12 download

TRANSCRIPT

Yokogawa Electric Corporation

Exaquantum/Batch Validation Planning GuideCopyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Industrial Automation Systems Division

Exaquantum/BatchValidation Planning Guide

Page 2Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Purpose

Provide Yokogawa sales, marketing and project teams with recommendations on how to validate Exaquantum/Batch systems that conform with 21 CFR Part 11

Target audience:– Internal sales, marketing, project teams

• Serves as starting point for project specific documentation

Page 3Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Introduction

Purpose:– Provide Yokogawa sales, marketing and project teams with

recommendations on how to validate Exaquantum/Batch systems that conform with 21 CFR Part 11

There is no one solution that will work for all applications

Recommendations are a starting point and reflect Yokogawa’s opinion for best practices

Recommendations can be used in creating user requirement specifications and project specific implementations

Assumptions used for preparing this presentation– Reader is familiar with:

• Exaquantum/Batch features & functions• Stelex’s Exaquantum/Batch Part 11 Whitepaper

– This document is not meant to supersede our customer’s corporate policies and procedures regarding Part 11

Page 4Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Background Material

U.S. FDA Regulations - http://www.fda.gov/– Predicate Rules that apply for each process– 21 CFR Part 11

Corporate policies & procedures for using Part 11

GAMP 4– Good Automated Manufacturing Practices– Published by ISPE, www.ispe.org

“Achieving 21 CFR Part 11 Compliance using Exaquantum/Batch” whitepaper by Stelex validation consultants

“Use of Exaquantum/Batch in Applications Regulated by FDA 21 CFR Part 11” Exaquantum/Batch help topic

Part 11 Preamble– Provides background information not in the actual rule

FDA guidance document “Part 11, Electronic Records; Electronic Signatures – Scope and Application” published September 3, 2003

Page 5Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Validation

Process used to provide evidence of a high degree of assurance that a facility will consistently operate correctly– Process Automation Systems and Plant Information Systems are parts of the

facility• The facility is validated

• Validating individual systems is part of validating the facility

Traditionally validation consists of:DQ = Design Qualification

• “Documented verification that a facility, system and instrument are designed according to written and pre-approved specification (URS)” [Design Review]

IQ = Installation Qualification• “Documented verification that a system is installed according to written and pre-

approved specifications” [GAMP 4]

OQ = Operational Qualification• “Documented verification that a system operates according to written and pre-

approved specifications throughout all specified operating ranges” [GAMP 4]

PQ = Performance Qualification• “Documented verification that a system is capable of performing or controlling

the activities of the process it is required to perform or control, according to written and pre-approved specifications throughout all specified operating ranges” [GAMP 4]

Page 6Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

21 CFR Part 11

U.S. government regulation– Applied worldwide when products are to be sold in the U.S.

– Other countries and regions have similar regulations

Established criteria under which electronic records and electronic signatures are considered the equivalent of paper records and handwritten signatures– Potential for cost savings by reducing quantity of paper records

• Cost reduction by space saving and record management e.g. Saving of paper consumption (ISO14001) and management cost

• Saving of seaching and looking for time of necessary records• Prevention of deterioration and damage of records compared paper

based one

Part 11 requirements are intended to:– Make it difficult to falsify records

– Ensure strict data integrity so electronic records are trustworthy

– Ensure that electronic signatures cannot be readily repudiated by the signer

Page 7Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

When to apply 21 CFR Part 11

21 CFR Part 11 does not require records to be saved or signed– Predicate rules require records and signatures

When predicate rules require records and signatures

and a company decides to store them electronically,

then Part 11 is applied to determine the requirements for handling the electronic records and electronic signatures

Even when electronic records and signatures are not used, manufacturing plants must still be validated.

Page 8Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Predicate Rules

Rules that require records to be kept – May require signatures

When a predicate rule requires a record be saved or signed and it will be stored electronically then 21 CFR Part 11 is applied to determine the requirements for the storage of the electronic record and rules for the electronic signature

U.S. FDA Regulations - http://www.fda.gov/– “Predicate Rules” must be understood for each process

• Used to identify electronic records requirements• Used to identify electronic signature requirements• Used in risk analysis to determine if Part 11 applies to the

Exaquantum/Batch system• For Exaquantum/Batch typically include Good Manufacturing

Practices 21 CFR 210 and 21 CFR 211

– Sec. 211.182 Equipment cleaning and use log:» “…The persons performing and double-checking the cleaning and maintenance

shall date and sign or initial the log indicating the work was performed.”

Page 9Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Validating Exaquantum/Batch Systems

Exaquantum/Batch was designed for use in applications that require use of the U.S. Food & Drug Administration’s 21 CFR Part 11 regulation

Considerable work is required to validate any computer system used in a FDA regulated environment. – No pre-prepared list of recommendations or actions will be

acceptable for all companies and installations.

– This set of recommendations may be used as a starting point for each Exaquantum/Batch installation.

Each installation must be validated individually– No vendor can provide a pre-packaged validated solution

– Yokogawa will work with customers to validate each Exaquantum/Batch installation

The first step in validating an Exaquantum/Batch system is:

PLAN

Page 10Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Planning Checklist

Risk assessment

Project Plan

Master validation plan

System architecture

Identification of Electronic Records

Identification of Electronic Signature requirements

Access control

Audit trail

Reporting

Standard Operating Procedures (SOPs)

Page 11Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Risk assessment

FDA recommends need for Part 11 controls be based upon a risk assessment. From Final Part 11 Guidance August 2003:

“We suggest that your decision to validate computerized systems, and the extent of the validation, take into account the impact the systems have on your ability to meet predicate rule requirements. You should also consider the impact those systems might have on the accuracy, reliability, integrity, availability, and authenticity of required records and signatures. Even if there is no predicate rule requirement to validate a system, in some instances it may still be important to validate the system.

We recommend that you base your approach on a justified and documented risk assessment and a determination of the potential of the system to affect product quality and safety, and record integrity. For instance, validation would not be important for a word processor used only to generate SOPs.”

Section III, C, 1 Validation, Lines 222-232

Page 12Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Risk assessment

Most processes controlled by PASs will be high risk– This means a deviation can have large negative impact on

product quality

– Will require validation and enforcement of Part 11 based on predicate rules

– This is not a reflection on PAS system quality

End user responsibility– Usually corporate policy dictates who, when, how risk

assessments are performed.

Yokogawa may assist with risk assessments– However risk assessment will go beyond PAS boundary and be

process specific

Page 13Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Planning Checklist

Risk assessment

Project Plan

Master validation plan

System architecture

Identification of Electronic Records

Identification of Electronic Signature requirements

Access control

Audit trail

Reporting

Standard Operating Procedures (SOPs)

Page 14Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Project Plan

Project plan is required:– Identifies tasks

– Used to develop schedule

– Required for validation

GAMP 4– Good Automated Manufacturing Practices

– Published by ISPE• ISPE = International Society of Pharmaceutical Engineers

– Widely recognized by regulators and pharma companies as basis for manufacturing practices

– All project teams must have access to and training in GAMP 4

Page 15Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Project Plan – V Model

All Exaquantum/Batch projects should follow the GAMP 4 “V Model”– Required for cGMP– Recommended as a good practice for non-regulated projects

PlanningURS

FS

DS IQ

OQ

PQ

Verifies

Verifies

Verifies PerformanceQualification

OperationalQualification

User RequirementSpecification

InstallationQualification

FunctionalSpecification

DesignSpecification

System Build

Verifies

This Planning Guide is targeted to support User Requirements Specification

Page 16Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Project Plan – GAMP 4 Software Categories

Exaquantum/Batch – Category 4 Configurable Software Package

Major Exaquantum/Batch sub-systems:– Automatic Batch Data Collection (ABDC)

• Category 3 Standard functions• No configuration

– Custom Batch Data Collection (CBDC)• Category 4 Configurable Software Package• Configuration required

– BatchWeb• Category 3 Standard functions• No configuration

– Reporting• Category 4 Configurable Software Package• Complex reports may be considered Category 5 Custom (Bespoke)

Software

– Custom MES Applications• Category 5 Custom (Bespoke) Software• Not a standard function therefore does not impact product

categorization

Category

Software Type

1 Operating System

2 Firmware

3 Standard Software Packages

4 Configurable Software Packages

5 Custom (Bespoke) Software

GAMP 4 Software Categories

Page 17Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Project Plan – Configurable Software Package

Validation approach– Record version (and configuration of environment) and verify

operation against user requirements• Windows 2000 and Windows 2003 Server are Category 1 Operating System

– Typically qualified for use by customer– “Challenged indirectly by the functional testing of the application” (GAMP 4)

– Vendor audit is welcome• Master records help by QA group in Tokyo

– Manage any custom (bespoke) programming as Category 5• Typically this would be complex reports performing functions outside of

Exaquantum/Batch– Examples:

» File system access» Access to other databases

SQL Server 2000 is considered an embedded component of Exaquantum/Batch– Does not have to be separately validated

DCS, SCADA and MES systems are typically considered Category 4

Page 18Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Planning Checklist

Risk assessment

Project Plan

Master validation plan

System architecture

Identification of Electronic Records

Identification of Electronic Signature requirements

Access control

Audit trail

Reporting

Standard Operating Procedures (SOPs)

Page 19Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Master Validation Plan

Customers will prepare a master validation plan– Depending upon the scope of the project this may focus on an

entire facility or the Exaquantum/Batch system

Master Validation Plan provides overall description of the project’s goals and the method to validate they can be achieved– State overall objectives, philosophies and approaches

– State, or reference, validation and Exaquantum/Batch system terminology and concepts required to understand required tasks

– State validation methodology to be used

– State major assigned tasks and responsibilities

– Establish means for creating and maintaining documentation developed during the validation process

Typically created by customer or their contractor– Yokogawa may assist with the Master Validation Plan

• As a minimum provide description of the Exaquantum/Batch system

Page 20Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Planning Checklist

Risk assessment

Project Plan

Master validation plan

System architecture

Identification of Electronic Records

Identification of Electronic Signature requirements

Access control

Audit trail

Reporting

Standard Operating Procedures (SOPs)

Page 21Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

System Architecture

Scalability

Hardware configuration

System software– Part 11 license

Network architecture– OPC

– Redundancy/Fault Tolerance requirements

– Failure scenarios

– Time considerations

Page 22Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Scalability

Requirements drive system configuration– How valuable is the data?

– Reliability vs. Cost trade-off

Scaling options– Centralized Configuration

• All on 1 computer• Suitable for very small systems & for

development/training systems

– Distributed Configuration• Different options • Driven by requirements

Deployment Components:• Exaquantum/Batch

Server • Database Management

System • Web Server• Administration Tools

BatchWeb may be accessed from any computer

Page 23Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Centralized Exaquantum/Batch System

V-net or Vnet/IP

Control System LAN

FCS

CS Batch 3000

ExaopcStation

ENGMaster Recipe Storage

HIS Batch ServersHIS

Plant Office LAN

. . .

Corporate LAN/WAN

. . .

. . .

Exaquantum/Batch Serverwith Databases, Web Server& Administration Tools

BatchWebUsers

BatchWeb Users

Switch

Internet

Internet

Switch

BatchWeb Users

Corporate Firewall

Page 24Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

FCSCS Batch 3000

Exaopc Stations

Distributed Exaquantum/Batch System

HIS Batch Servers

HIS

Plant Office LAN

. . .

Corporate LAN/WAN

Exaquantum/Batch Server

ENGMaster Recipe

Storage

Administration Tools& BatchWeb Users

. . .

BatchWebUsers

BatchWebUsers

BatchWeb Users

Exaquantum/BatchWeb Server

Switch

Switch

Corporate Firewall

Internet

Internet

. . .

BatchWeb Users

. . .

BatchWeb Users

Exaquantum/Batch Database Server

Control System LAN

V-net orVnet/IP

Page 25Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Standard Deployment Options

Installation program supports:– Exaquantum/Batch Server & Database Management System on 1

computer

– Administration Tools may be installed on Windows 2000/2003 Professional or Server

– Web Server may be installed on its own computer

Project customization is required to install the Database Management System on a separate computer– Not a major effort

Page 26Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Deployment Recommendations (1 of 2)

Exaquantum/Batch Server & Database Management System on 1 computer– Sufficient for most applications

– Rule of thumb when to separate:• When number of concurrent BatchWeb or 3rd party database connections

exceeds 100 and number of concurrent batches exceeds 20

Separate Web Server from data collection & storage– Separate when expecting more than 100 registered BatchWeb users

and 3rd party database connections

– Separation benefits• Separates BatchWeb data requests do not content with data collection

tasks

• Increased cyber-security – Intruders look for web servers first, separation provides additional level of

isolation for data collection functions

• Easier to have separate DBA (Database Administrator) and Web Server Administrator

• Easier to apply Microsoft updates to the web server computer

Note: Rules will vary with computer specifications

Page 27Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Deployment Recommendations (2 of 2)

Administration Tools– Always load on Exaquantum/Batch Server

• For initial configuration and when restricting network access during disruptions

– Primary use of Administration Tools should be on remote computers

• Lessens load on Exaquantum/Batch Server– Less competition with data collection for memory and CPU utilization

• Access Control – Different Administration Tools can be enabled on each computer

» System Configuration Tool

» Equipment Configuration Tool

» Custom Batch Data Collection Configuration Tool

» Report Template Manager

– Not all persons using a Tool should have access to Exaquantum/Batch server and Database Management System Server

» Lessens chance of accidental disruption due to human error

• Administration Tools can be loaded on multiple computers– Warning: Individual tools not designed for concurrent use

» No locks on data

» Most recent “Save” could overwrite other’s work

Page 28Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

System Software

Windows 2000 Server SP 4 or Window 2003– Web server uses Microsoft IIS

– Net Framework Version 1.0 SP 2

Microsoft Internet Explorer 6 SP 1

Microsoft SQL Server 2000 SP 3

Microsoft Office 2000, 2003 or XP (for reporting)– Office 2000 SR-1, SP2 and SP3

– Office XP SP1, SP2 and SP3

Standards BasedOPC– Data Access (DA 2.05a)– Alarm & Events (A&E 1.1)– Historical Data Access (HDA

1.1)– Batch (Batch 1.0)

S88– Part 1 Models & Terminology– Part 2 Data Model

SQL– ANSI standard supported by

Microsoft SQL Server 2000World Wide Web Consortium– HTTP– XML

Page 29Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Exaquantum/Batch Part 11 License

Exaquantum/Batch has an optional license that enables features useful when working with Part 11

Features enabled with the license:– Electronic signature for manual data entry

– Signature manifest data and icon is displayed to indicate manually changed values

– Report output must be approved with an electronic signature

– Report check out/in feature is disabled to increase report data integrity

• Excel copies of reports may not be accessed by normal means • The PDF format report is retained as the master copy

Other features such as audit trails and access controls are part of the base product offering

See the Configuration Guide Presentation for details on the Part 11 license

Page 30Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Network Architecture

Exaquantum/Batch must be implemented as a closed system– Should be treated as an Intra-net web site

• DO NOT treat as an Internet web site

– Utilize strict and strong access control (more later)

Grant access using Windows 2000/2003 Domains– Provides consistent method for access control

– Single login from any computer can grant access

Consider using a firewall to isolate Exaquantum/Batch and the control system from the plant and/or corporate network– Isolation of data collection and storage from general plant and

corporate networks increases data integrity and lessens chances of disruption

Page 31Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Firewall Usage

Firewall may be used to isolate the control system LAN from the Plant Office & Corporate LANs

Place Firewall between the web server and intranet web clients– Firewall Ports Opened:

• HTTP (Port 80)• Windows Authentication

(Port 445)

If a Firewall is placed between the Exaquantum/Batch Server and web server DCOM ports must be left open– This is not recommended

ExaopcStation

Plant Office LAN

Exaquantum/Batch Server

BatchWebUsers

Firewall

Switch

SwitchCorporateFirewall

Internet

Internet

BatchWeb User

Control

System

LAN

Exaquantum/BatchWeb Server

Page 32Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Redundancy / Fault Tolerance

Exaopc Servers– Redundant Data Access, Alarm & Event and Historical Data

Access Servers

– Dual Batch Servers

Multiple Ethernet paths

Fault tolerant computers– Stratus fault tolerance gives highest server availability

RAID 5 data storage– Provides highest data availability

Level of Redundancy & Fault Tolerance

Is a

Cost – Benefit Trade-off

Examine failure scenarios and their impacts

Page 33Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Exaopc Configuration

Exaopc Stations are part of the CS Batch 3000 system

Exaopc supports OPC Foundation Specifications

– Data Access 2.05a (DA)

– Alarm & Events 1.1 (A&E)

– HDA 1.1 (includes Alarms & Events)

– Batch 1.0

High data availability options– Data buffering for DA and A&E servers

– Historical catch-up for DA server

– Redundancy for DA and A&E servers

– Dual Batch servers

Control System LAN

V-net orVnet/IP

FCS

CS Batch 3000

ExaopcStation

ENG

Exaquantum/Batch

Switch

HIS

Page 34Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

V-net or Vnet/IP

Ethernet

FCS

Unit Supervision

Exaquantum/Batch

Exaopc Servers

A B

Exaquantum/Batch

Exaopc Server

A

Exaopc Client

Exaquantum/Batch OPC Client

Exaopc Server

Data AccessAlarm & Event

Redundant OPC DA and A&E Servers

Exaopc Server

B

CENTUM CS 1000 or CS 3000 Batch

HIS & BatchServers

Page 35Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Exaquantum/Batch

Exaquantum/Batch OPC Client

Dual OPC Batch Servers

Manual switchov

er required

Exaopc Server

A

Exaopc Server

B

CENTUM CS 1000 or CS 3000 Batch

Ethernet

FCS

Unit Supervision

Exaquantum/Batch

Exaopc Servers

A BHIS & BatchServers

V-net or Vnet/IP

Page 36Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Exaopc Recommendations

Driven by importance of data

Option 1 Standard Availability:– Single Exaopc station providing Data Access, Alarms & Events

and Batch interfaces

Option 2 – Highest Availability:– Two Exaopc stations

• Redundant DA and A&E servers• Dual Batch servers

To recover from disruptions in connections between Exaopc station and Exaquantum/Batch Server:– Use history catch-up and data buffering for trend data (Data

Access)

– When redundant servers used provides highest availability • Note: maximum of 1 minute failure

– Use persistence of CS Batch 3000 batch data

Page 37Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Failure Scenarios

Partial list of failure scenarios to consider– Controlled shutdown of each computer for maintenance

– Unexpected shutdown of each computer – causes include• Power failure• Physical damage (building collapse, fire, sprinkler system discharges,…)

– Network disruption• Cables cut or unplugged• Competing programs use network bandwidth denying sufficient

bandwidth for Exaquantum/Batch

– Disk failure

– Programs manually stopped

– Unauthorized programs run on server conflict with Exaquantum/Batch

– Computer virus/hacker attack• File deletion• Web Server attacked• Passwords stolen• Unauthorized intrusion

Page 38Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Failure Scenarios – Data Collection Recovery

ImpactFailure

Batch Data Recovery Trend Data Recovery Alarm & Event Data Recovery

Exaopc Station Failure(No Exaopc Redundancy)

Upon recovery persistent batch data in CS Batch 3000

automatically recovered

Recovery not possibleMinimize risk using

redundancy

Recovery not possibleMinimize risk using

redundancy

Exaopc Station Failure- Single server of redundant pair fails(Exaopc DA and A&E RedundancyDual OPC Batch servers)

Data collection resumes after manual configuration change in

Exaquantum/BatchUpon recovery persistent batch

data in CS Batch 3000 automatically recovered

No interruption provided 1 server of redundant pair is

operational

No interruption provided 1 server of redundant pair is

operational

Exaopc station controlled shutdown

Upon recovery persistent batch data in CS Batch 3000 can be

recovered

Recovery not possibleMinimize risk using

redundancy

Recovery not possibleMinimize risk using

redundancy

Exaopc station controlled shutdown - Single server of redundant pair shutdown(Exaopc DA and A&E Redundancy)

Data collection resumes after manual configuration change in

Exaquantum/BatchUpon recovery persistent batch

data in CS Batch 3000 automatically recovered

No interruption provided 1 server of redundant pair is

operational

No interruption provided 1 server of redundant pair is

operational

Exaquantum/Batch Server Failure (not a disk failure)

Upon recovery persistent batch data in CS Batch 3000 can be

recovered

Data can be recovered using History Catch-up

(manual procedure)

Data can be recovered using History Catch-up

(manual procedure)

Database Disk Failure Recovery not possibleMinimize risk using RAID 5 Disks

Recovery not possibleMinimize risk using RAID 5

Disks

Recovery not possibleMinimize risk using RAID 5

Disks

Web Server Failure No Impact No Impact No Impact

Loss of Communication between Exaquantum/Batch Server & Exaopc

Upon recovery persistent batch data in CS Batch 3000 can be

recovered

Recovery possible for DA data only

(AE data, Server Calcs and Aggregations not recovered)

Recovery not possibleNotes:• Slide refers to use of Automatic Batch Data Collection. Custom Batch Data Collection

scenarios differ and are dependant upon the OPC server and control/information system used.

Page 39Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

High Availability Recommendations

Exaquantum/Batch Server– Stratus fault tolerant computer

– RAID 5 disks

– Dual power feeds

Exaopc Stations– Redundant DA and A&E servers

• Use buffering and catch-up features

– Dual Batch server• Do not delete batch from CS Batch 3000 until data collected

Isolate Control System LAN from Plant Office & Corporate LANS with a firewall

Web Server on separate computer from the Exaquantum/Batch Server

Administration Tools on multiple remote computers within data collection firewall

Page 40Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Recommended Small System Configuration

V-net and Vnet/IP

Control System LAN

FCS

CS Batch 3000

ExaopcStation

ENGMaster Recipe Storage

HIS Batch ServersHIS

Plant Office LAN

. . .

Corporate LAN/WAN

. . .

. . .

Exaquantum/Batch Serverwith Databases, Web Server& Administration Tools

BatchWebUsers

BatchWeb Users

Switch

Internet

Internet

Switch

BatchWeb Users

Corporate Firewall

Page 41Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

V-netor Vnet/IP

FCSCS Batch 3000

Exaopc Stations

Recommended Large System Configuration

HIS Batch Servers

HIS

Plant Office LAN

. . .

Corporate LAN/WAN

ENGMaster Recipe

Storage

. . .

BatchWebUsers

BatchWebUsers

BatchWeb Users

SwitchCorporateFirewall

Internet

Internet

. . .

BatchWeb Users

. . .

BatchWeb Users

Control System LAN

Firewall

Switch

Exaquantum/Batch Server & Database Server

Administration Tools& BatchWeb Users

Exaquantum/BatchWeb Server

Page 42Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Time Considerations

Records must be clear and consistent with regard to timestamps– Time zone– Standard Time vs. Daylight Savings Time (Summer Time)

Exaquantum/Batch stores all timestamps in UTC in the database– UTC = Universal Coordinated Time (~GMT)– When collected times are converted from local time to UTC and stored– When viewed on BatchWeb times are converted from UTC to the Web

Server’s local time

No changes required when switching between Standard & Daylight (Summer) Time

All computer’s must have synchronized time to avoid discrepancies

More detail in Configuration Guidance Presentation

Page 43Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Planning Checklist

Risk assessment

Project Plan

Master validation plan

System architecture

Identification of Electronic Records

Identification of Electronic Signature requirements

Access control

Audit trail

Reporting

Standard Operating Procedures (SOPs)

Page 44Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Electronic Records/Signatures

User Requirements Document is key for planning– Without clear requirements project is in jeopardy

Identify electronic records– All Exaquantum/Batch data are potentially electronic records

– What is required by predicate rules?

– What will be handled electronically vs. with paper?

Identify electronic records requiring signatures– Will electronic signatures be used?

– If so, what electronic records must be signed?• Only human actions require signatures

– Done by

– Reviewed by

– Checked by

Page 45Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Identification of Electronic Records

Identification of electronic records is specific to each application

Exaquantum/Batch is designed so all data in the databases can be considered electronic records

Possible Exaquantum/Batch electronic records– Batch data

– Master recipe data

– Equipment data

– Trend data

– Alarms & Event data

– Reports

Page 46Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Electronic Records – Data Integrity

Stored securely– Stored in SQL Server 2000

– Long-term on-line storage

Access control– Windows 2000/2003 security used to limit access

– Administrators set access rights

Data Collection– Changed formula values not overwritten

• Save each changed value with timestamp

– Property value changes recorded in audit trail

– Automatic Batch Data Collection• Product automatically collects data, no configuration – reduces

validation effort

– Custom Batch Data Collection• System specific configuration increases validation effort

Page 47Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Electronic Records – Configuration Data

Configuration/Engineering data not covered by an audit trail– System Configuration Tool

• Small amount of simple data

– Equipment Configuration Tool• Definition of tag aliases for use with batch trends and reports

• Entry of custom equipment hierarchy for use with Custom Batch Data Collection

– Report Template Manager• Report definitions

• May range from simple to complex, an number of report templates possible

– Custom Batch Collection Tool• Optional package

• Definition of collection lists and their triggers

• May range from none to large amounts of data

– Administration Tools• Defines OPC gateway, tag template and tag definitions

• Typically simple data, may be expanded to more complex applications

Page 48Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Electronic Records – Configuration Data

Requires SOPs to document changes– Normal change control procedures acceptable

Reports can include configuration data– Use to document configuration

Page 49Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Electronic Signature

Applied using account name & password– Uses Windows 2000/2003 domain accounts & passwords

Required for manual entry of operational data:– Formula value changes

– Adding new formula items

– Approving report templates

– Approving report output

– Entry of manual performance rating

All operational data manual entries recorded in an audit trail

Exaquantum/Batch only supports 1 signature per entry– SOPs required to document checked by or approved by

Signature manifest pop-up window– Provides information about electronic signatures

– Satisfies § 11.50 Signature manifestations

– Manifest data may be included in reports

Page 50Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Planning Checklist

Risk assessment

Project Plan

Master validation plan

System architecture

Identification of Electronic Records

Identification of Electronic Signature requirements

Access control

Audit trail

Reporting

Standard Operating Procedures (SOPs)

Page 51Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Access Control

Different types of access control

Physical– Physical access to Exaquantum/Batch and Web servers should be

limited

Networked– Network access to servers should be limited

• Require Intra-net access– Corporate VPN can be used to allow remote access

• Restrict access to Exaquantum/Batch Server using a firewall

Privileges– Grant individuals access to specific functions using Windows

2000/2003 Groups– Create roles

• Each role is represented by a Windows 2000/2003 Group• Add individual accounts to the role’s group• Individuals can still be given specific privileges outside of the group

– Windows 2000/2003 domain makes managing access control simpler

Page 52Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Exaquantum/Batch Privileges

Each Windows 2000/2003 Group corresponds to an Exaquantum/Batch privilege

Group Privilege

QUserGroup Access BatchWeb

QDataWriteGroup Write data to the databases

QExplorerDesignGroup

Access to Exaquantum/Explorer, not required for Exaquantum/Batch

QAdministratorGroup Access to configuration tools

QBReportApprove Approve reports in BatchWeb

QBReportCheckout Check-out and Check-in report spreadsheet files (Note: This function is disabled when the Part 11 license is used)

QBReportRun Run reports from BatchWeb

QBReportTmplApprove

Approve report templates in the Report Template Manager

QBTrendTmplAdmin Create, modify and delete batch trend templates in BatchWeb

QBPerfRatingModify Modify performance rating values in BatchWeb.

QBFormulaAllWritable Override individual lockouts to change formula item values in BatchWeb

QBFormulaCreate Create new formula items in BatchWeb

QBFormulaParamMod Modify existing parameter formula item values in BatchWeb

QBFormulaResultMod Modify existing result formula item values in BatchWeb

Page 53Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Use Roles to Assign Privileges

Each Windows 2000/2003 Group corresponds to an Exaquantum/Batch privilege

GroupDB

AdminEnginee

rOperat

or

Operations

Supervisor

Office Staff

QUserGroup X X X X X

QDataWriteGroup

QExplorerDesignGroup

QAdministratorGroup

X

QBReportApprove X

QBReportCheckout

QBReportRun X X X X X

QBReportTmplApprove

X

QBTrendTmplAdmin X X

QBPerfRatingModify X

QBFormulaAllWritable

X X

QBFormulaCreate

QBFormulaParamMod

QBFormulaResultMod

Page 54Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Roles

DB Admin– Responsible for database

maintenance (backups, disk space mgt,…)

– No operational responsibilities or privileges

Engineer – Responsible for configuration,

but not for operations

– Approve report templates

GroupDB

AdminEnginee

r

QUserGroup X X

QDataWriteGroup

QExplorerDesignGroup

QAdministratorGroup

X

QBReportApprove

QBReportCheckout

QBReportRun X X

QBReportTmplApprove

X

QBTrendTmplAdmin X

QBPerfRatingModify X

QBFormulaAllWritable

QBFormulaCreate

QBFormulaParamMod

QBFormulaResultMod

Easy to change privileges and individuals assigned to roles

Page 55Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Use Roles to Assign Privileges

GroupOperat

or

Operations

Supervisor

Office Staff

QUserGroup X X X

QDataWriteGroup

QExplorerDesignGroup

QAdministratorGroup

QBReportApprove X

QBReportCheckout

QBReportRun X X X

QBReportTmplApprove

QBTrendTmplAdmin X

QBPerfRatingModify

QBFormulaAllWritable

X X

QBFormulaCreate

QBFormulaParamMod

QBFormulaResultMod

Operator– Use BatchWeb to view

historical data

– Run reports

– Enter specific manual entries

Operations Supervisor – Operator’s privileges,

plus:

– Approve reports and batch trend templates

Office Staff– Only access to BatchWeb

and running reports

Page 56Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Using Local Groups & Domains

Fred’s Windows Domain account is made a member of the Local Group “Engineer”

Fred now may log into the Exaquantum/Batch server or web server from any location

Local Group:QBEngineer Privileges:

• QUserGroup• QAdministratorGrou

p• QBReportRun• QBReportTmplAppro

ve• QBTrendTmplAdmin• QBPerfRatingModify

Individual Domain Account:Fred Smith

Page 57Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Using Local Groups & Domains

Fred’s Windows Domain account is made a member of the Domain Group “Engineer”– The Domain Group has no privileges by itself

The Domain Group is made a member of the Local Group– This gives all members of the Domain Group “Engineers” the

privileges set in the Local Group “Engineer”

Fred now may log into the Exaquantum/Batch server or web server from any location

Local Group:QBEngineer Privileges:

• QUserGroup• QAdministratorGrou

p• QBReportRun• QBReportTmplAppro

ve• QBTrendTmplAdmin• QBPerfRatingModify

Individual Domain Account:Fred Smith

Domain Group:QBEngineer

Page 58Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Access Control – System Administrators

Multiple System Administrators– Limits capability of 1 person to circumvent controls

– Enables different organizational reporting structures to reduce motivation to circumvent controls

System Administrators:– Domain Admin

– Operating System Admin

– Database Admin

– Exaquantum/Batch Admin

Administrators are trusted individuals– No system is fool-proof, ultimately we depend upon trust

Page 59Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Domain Administrator

Responsibilities– Maintain integrity of the Windows Domain

• Implement policies

– setting up and maintaining• User accounts • User groups • Administering Windows 2000/2003 group policy

– Assign individuals to domain groups

– Monitor Windows 2000/2003 log files• System log• Application log• Security log

– Ensuring there is no unauthorized access to Exaquantum/Batch from beyond the company's domain  

– License management

Page 60Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Operating System Administrator

Responsibilities– Maintain integrity of the Local Computer

• Should be 1 person responsible for all servers• Implement policies

– Assign Domain Groups to Local Groups

– Assign privileges to Local Groups

– Monitor Windows 2000/2003 log files• System log• Application log• Security log

– License management

Page 61Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Database Administrator

Responsibilities– Ensure smooth operation of the Exaquantum/Batch relational

database  • Typically requires a low level of on-going activity    • Maintain integrity of the Database

– Database backup• Establish schedule• Follow schedule to backup database • Securely store backups on-site and off-site

– Preparation for disaster recovery• Include practicing

– Restoration of backup data as necessary

– Control access to database• Assign Read/Write data access rights

– Monitor database logs

– Monitor disk space• Ensure free space is always available

Page 62Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Exaquantum/Batch Administrator

Responsible for – Installing, configuring, maintaining Exaquantum/Batch.  

Uses Exaquantum/Batch configuration tools – System Configuration Tool

– Report Template Manager

– Equipment Configuration Tool

– Custom Batch Data Collection Tool

– Administration Tools

– Other Exaquantum configuration tools may optionally be used:• Exaquantum Explorer, Database Creation Tool, Graphics Editor,…

Monitor system operation– Regularly review Windows application and system log files for

Exaquantum/Batch errors and warnings

– Create reports to record configuration data

– Disk space usage

Page 63Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Planning Checklist

Risk assessment

Project Plan

Master validation plan

System architecture

Identification of Electronic Records

Identification of Electronic Signature requirements

Access control

Audit trail

Reporting

Standard Operating Procedures (SOPs)

Page 64Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Audit Trail

Different audit trails must be checked– Domain level

• Check for unauthorized access attempts

– Server Operating Systems – Windows 2000/2003• System Event Log• Security Event Log (must enable security auditing in Windows

2000/2003)• Application Event Log

– Exaquantum/Batch messages are found here

– Operational entries recorded in QBatch database• Property History table records changes to object properties• Electronic Signature Manifest records manual entries

– Report template manager preserves old report formats• Useful to reference what queries were used to produce a report

FDA information requests– Typically will be satisfied with data in the databases

– Run reports to produce reports to satisfy FDA requests

Page 65Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Planning Checklist

Risk assessment

Project Plan

Master validation plan

System architecture

Identification of Electronic Records

Identification of Electronic Signature requirements

Access control

Audit trail

Reporting

Standard Operating Procedures (SOPs)

Page 66Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Reporting Package

Approvals part of reporting lifecycle

All versions of report templates saved in the database

All report output (created when reports are run) are saved in the database

Report files can be saved to the file system to be made available to FDA inspectors

3rd party report packages may be used with Exaquantum/Batch– To comply with corporate standards

Page 67Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Reporting Lifecycle

Report template created– Must be approved before reports are run

Reports run– Report property specifies if individual

reports must be approved

Report template modified– Must be approved before reports are run

Reports run– Must be approved before reports are run

Report template placed out of service– Template can not longer be used

– May be placed back in service to be used again Report

Template Out of Service

Create Report Template

Run Report

Modify Report

Template

Run Report

Approval

Approval

Approval

Approval

Access Control On all Lifecycle

Steps

Page 68Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Report Types

Many uses of reports– Regulatory

– Business

– Production efficiency

– Time based summaries

– Batch End Reports

Regulatory reports must be validated– Other reports may or may not be validated depending upon

company policy

– Report master copies stored in database• Validating that Exaquantum/Batch can consistently display and user

can save to file system an exact copy of the report should be performed

Page 69Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Providing Data to FDA Inspectors

Part 11 requires the “ability to generate accurate and complete copies of records in both human readable and electronic form suitable for inspection, review and copying by the agency

Reports can be designed to:– Document all aspects of batch execution, including master

recipe, trend and event data

– Produce PDF format output

– Macros may be used to write Excel, CSV or other format files to disk

Method used to provide data to FDA inspectors will be heavily influenced by company policy

Page 70Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Planning Checklist

Risk assessment

Project Plan

Master validation plan

System architecture

Identification of Electronic Records

Identification of Electronic Signature requirements

Access control

Audit trail

Standard Operating Procedures (SOPs)

Page 71Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Standard Operating Procedures (SOPs)

SOPs define manual tasks are to be performed and documented

SOPs will be required to augment Exaquantum/Batch automated actions – Provide consistent manual actions

– Administratively enforced

List of required SOPs derived from– User requirements

– Functional specification

– Design specification

Page 72Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Typical Exaquantum/Batch SOPs

SOPs recommended for Exaquantum/Batch installations– User account management

• New accounts, maintaining existing accounts• Recommended account settings• Assigning Exaquantum/Batch privileges

– Keeping passwords and accounts in Windows & CS Batch 3000 in synchronized

– Data backup

– Data restoration

– Disaster recovery

– System monitoring• Windows log files• SQL Server logs• System state of health

– Report creation & maintenance

– Change Control of Software and Computer Systems

Page 73Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Planning Checklist

Risk assessment

Project Plan

Master validation plan

System architecture

Identification of Electronic Records

Identification of Electronic Signature requirements

Access control

Audit trail

Standard Operating Procedures (SOPs)

Page 74Copyright © Yokogawa Electric Corporation1st Edition April 18, 2005

Exaquantum/Batch Validation Planning Guidance(PM36J04B90-01E_001)

Summary

Planning for validation is critical

Exaquantum/Batch is designed to operate under Part 11 regulations

Yokogawa will work with customers to validate Exaquantum/Batch systems

More detailed information is available:

– Exaquantum/Batch Configuration Guide Presentation

– Exaquantum/Batch Help System