developing and implementing calibration programs...the calibration range must be, at a minimum,...

Developing and

Implementing Calibration Programs

March 19, 2013 Pennsylvania Convention Center Philadelphia, PA USA

Andy Ferrell PCI CEO & President

Wm. Andy Ferrell - Bio

• Bachelors and Masters degrees from East Carolina University • Engineer with big pharma companies from 1989 to 1996 • Speaking on topics such as:

– Smart Asset Management – Emerging Trends in Calibration Compliance – Predictive Maintenance and Calibration Programs: Improving Operations

• Founded PCI in 1996 • PCI is 110 person firm. Headquartered in RTP, NC with 5 field offices and

national consulting & technical service coverage • PCI’s Technical Services: delivering calibration, maintenance, and repair

services for both manufacturing and laboratory equipment • PCI’s Consulting Services: building calibration & maintenance programs

including developing business workflows and policies, implementing software systems, FDA remediation plan development & execution. We development world class maintenance & calibration programs

Introductions: Who are you?

Name Your role Company Industry Any Regulatory Concerns What are you seeking to get

from this workshop

What we will cover

Background Info …. Calibrations Basic Calibration Program Requirements What is a Calibration Program? Calibration Program Requirements Responsibilities of a Calibration Department Critical Elements of a Calibration Program Paperless: Changing Existing Cultures Benefits of going to a Paperless Program

Calibration: Definition

“A comparison of a measurement standard (Calibration Standard) of known accuracy with another instrument of unknown accuracy to detect, correlate, report, or eliminate by adjustment any variation in the accuracy of the item being compared.”

Every day, industries throughout the world perform a huge number of measurements.

The results of these measurements are used to make decisions that could affect people’s lives both personally and in the workplace.

Why Calibrate?

Calibration is warranted based on good engineering and business practices, and must adhere to standards and regulations in the following ways:

Good engineering and business sense. Instrumentation wears and settings drift, either mechanically or because of environmental effects. These conditions can have a negative effect on instrumentation and can ultimately affect the quality of a product.

Why Calibration?

Without Calibrations

All of us would pay more at the Gas station.

Food would be weighed incorrectly.

No more watching your favorite television shows or listening to your favorite songs on the radio, because due to calibration errors, the frequency would be off enough that TV and radios would be useless.

Without Calibrations

Ingredients in your prescription drugs could cost more or even more important cause illness or death.

Because of incorrect calibrations, criminals could be either not convicted or released on Bad evidence.

In pharmaceutical manufacturing and other life science businesses, companies take measurements throughout the various stages of the product life cycle to ensure product integrity.

The certification and calibration of instrumentation, combined with a robust calibration management system, helps achieve product integrity throughout all production processes.

With Calibrations

The Calibration Train wreck

In January 2005, during the annual Measurement Science Conference (MSC) held in in Anaheim, California, one of the participants from Germany explained to the group that he had been a guest professor at an University in China. He explain that they had an input of 8,000 students every year studying measurement techniques. He went on to explain that the Chinese government has increased the attendance to almost 12,000 students.

Then the question came up, why doesn’t the United States or

Germany train in the field of metrology and calibration the way the Chinese are doing?

His answer was “ Because we haven’t had the train wreck yet.” We haven’t had the train wreck where someone says, “ Calibration was the Problem.” Where there was a great loss of life or limb, or many businesses were effected.”

Regulations & Guidances


Regulations are Law.

Example: the FDA has GMPs (Good Manufacturing Practices) or CFR 21.

CFR 21 is Code of Federal Regulations for Food and Drugs.

Regulations are usually vague.


Guidances are not Law. Usually published by industry organizations. Much more specific than Regulations. Following Guidances does not provide 100%

certainty of zero observations or violations.

Regulation & Guidance

Guidance Documents - Benchmarking

ISPE GAMP Calibration Management (second edition)

ISPE GAMP Validation of Lab Computerized Systems

NCSLI RP-6 Calibration Control Systems

ISO 17025 General Requirements for Calibration Labs

ISO 98 Guide for Measurement Uncertainties

Regulation & Guidance

IF we follow all the Guidelines, we are guaranteed to be in compliance with the Regulations.

True? No, False All Guidelines are fallible.

Industry Guidelines

Industry Guidances

NCSL International Recommended Practice RP–6 ISO/IEC 17025:2005 specifies the general

requirements for the competence to carry out tests and/or calibrations, including sampling (proficiency testing)

ANSI/NCSL Z540.3-2006 Requirements for the Calibration of Measuring and Test Equipment

The ISPE GAMP® Good Practice Guide: Calibration Management

Implementing a Calibration Program

Calibration Program Basics

Calibration Policy and support Procedures (SOP) Calibration Records Qualified Personnel Calibration Training Calibration Labeling Calibration Intervals Calibration Logistics / Scheduling Define Out of Tolerance conditions or Non Conformance Process for Investigation of Non Conformances Adequacy of Calibration Standards Traceability (and Reverse Traceability) Analysis of Calibration Data Continuous Improvements (Audits)

Calibration Software Requirements

Implementing a Calibration Software

1. Need to put together a team

2. Obtain Site/Management “Buy In”

3. Obtain appropriate funding

4. Put together timeline and resource allocation

Calibration Software Considerations Implementation budget Platform supported Networked/ web based Number of assets to be managed Number of transactions annually Number of years required to be maintained

User Requirements (URS)

Ability to capture unique instrument identification

Ability to assign classification for each instrument in the program

Ability to capture the calibration test points for each instrument

Ability to schedule calibration services for assigned specified intervals

Ability to handle multiple test ranges Ability to record calibration data


Ability to run a matrix report showing, for a specific time period and by various selection criteria: • Total # of scheduled calibrations • Total # of active and total # of inactive

instruments at the beginning of time period • Total # of scheduled calibrations [redundant, 1st

above] • Total # of active and total # of inactive units as

they were at the end of time period • Net change in total # of scheduled calibrations


Both internal corporate compliance and third-party regulatory bodies conduct Audits throughout the pharmaceutical industry to ensure that manufacturers are following their own procedures and to see how they react to issues and deviations to their processes.

A robust Calibration Program is vital to ensure that

when similar audit points are challenged, it can be demonstrated that systems, processes, and procedures are in place to address potential noncompliance issues.

Calibration Program Checks and Balances

Performance Metrics

Make strategic objectives clear, in order to focus and bring together the total organization

Tie the core business processes to the objectives

Focus on critical success factors for each of the processes, recognizing there will be variables

Track performance trends and highlight progress and potential problems

Identify possible solutions to the problems

Performance Metrics: Purpose

What Gets Measured Gets Done.

What Gets Measured, Improves!!!

Performance Metrics

80.0%82.0%84.0%86.0%88.0%90.0%92.0%94.0%96.0%98.0%100.0%

050

100150200250300350400450500

Jan-04 Feb-04 Mar-04 Apr-04 May-04 Jun-04 Jul-04 Aug-04 Sep-04 Oct-04

% Calibrations performed: Laboratory Instruments

Calibrations Scheduled % calibrations performed

Performance Metrics: Example

Auditing

Developing & Implementing Calibration Programs:

The 4 Most Critical Elements

The 4 Most Critical Elements of Calibration Programs

• Classification • Intervals • Ranges • Limits

Classification, Intervals, Ranges, and Limits

An instrument/device is classified based on its potential to affect a product or products.

This classification is defined and documented in a paperless system using such terms as “critical,” “non-critical,” or “reference only.” Characteristically, where process parameters are monitored, controlled, or recorded, and are used to determine product quality, the associated instruments would warrant a critical classification.

Most instruments associated with noncritical and utility classifications are used to provide indications of general equipment conditions and for troubleshooting.


Interval or service frequencies or schedule.

For regulated environments, it is typical to assign Intervals of weekly, monthly, quarterly, semi annual, and annual.

Bi-annual are used but are uncommon.

Daily are typically not used. Calibration Before Use is typically assigned instead of daily intervals.

• Does a measured and controlled parameter on product have a direct effect on product safety or quality (e.g., sterility)?

• What is the instrument’s (OEM) stated

reliability and accuracy? • What is the time from product manufacture to use?

If this time is relatively short, adulterated product could be distributed & used if improper intervals are set.

Interval Assignment Considerations

o Instrument Design Range The design range or maximum range of operability of the

instrument, as specified by the manufacturer. This must be wide enough to ensure that the instrument is capable of reliably operating across the entire qualified range for the equipment.

o Process Operating Range Specified range within which the process must operate. o Instrument Calibration Range The calibration range must be, at a minimum, equal to the

qualification range. A wider calibration range, however, can be useful for non-linear instruments or whenever the qualified range is too small to prove linearity of the instrument.


Considerations • Given to the criticality (or classification) of

the instrument • Given to the environment in which the

instrument is used • Given to the users • Will the instrument be used for the same

test consistently


An Example: Instrument’s Design Range 0 deg C to 200 deg C

Operating Range 20 deg C to 80 deg C

Instrument Calibration Range 10 deg C to 100 deg C

Classification, Intervals, Ranges, and Limits (tolerance)

The calibration tolerance is the maximum allowable deviation of an instrument from true value before there is an effect on product and is utilized for critical instruments/devices. This predetermined tolerance is used as an out-of-tolerance investigation trigger.

The Manufacturer’s Accuracy Specifications should not

be used as the Calibration Tolerance Limit


An Example: Instrument’s (OEM) Design Accuracy +/-1.0 deg C

Your Adjustment Limit +/- 1.5 deg C

Your Calibration Tolerance +/- 2.0 deg C


Instrument Tolerance Relationship .

Calibration Tolerances


General Rule

If an investigation of an Out of Tolerances does not result in a product rework or recall, the Process Tolerance should be set to avoid doing the same investigation if the instrument fails calibration by a similar or less value

The Rule of Thumb for Setting Calibration Tolerances

All instruments drift. No exceptions.

Therefore, Never use the Manufacturer’s Specifications for Accuracy as the Calibration Tolerance!

Rule of Thumb: double the OEM’s stated accuracy for your calibration tolerance. If that doesn’t meet your needs, purchase another instrument.

Your Bonus Jean Piaget was a scientist who in the early 1900's studied the tendency to focus attention on only one characteristic of shape. In particular, he studied the human inability to distinguish identical volumes in different sized containers. Piaget studied the inability of small children to understand that cylinders with different aspect ratios could have the same volume. Small children have difficulty conceiving the shift in volume from one dimension (say the width of a glass) to another dimension (say the height of a glass). They have difficulty visualizing that there is no more liquid in a tall, slender glass than in the shorter, wider glass. He showed that for certain individuals, this tendency could extend into adulthood.

This is illustrated in the following Figure:

The Visual Measure The tendency is to believe that even though the two glasses have the same volume, the tall, slender glass has more volume because the height of the liquid is greater. This fact explains why studies show that mixed drinks at bars that use short, wide glasses are typically stronger then the same drink at bars that use tall, slender glasses. The height of a 1.5 oz. shot of alcohol in the short, wide glass is not as high as the same 1.5 oz. shot in the tall, slender glass. When bartenders pour a shot in a short, wide glass they have the tendency to think that since the liquid level is not very high they have mis-poured, and so add a little more alcohol to the drink. The moral of this study is always get your mixed drink from bars that use short, wide glasses.

The Beer Gauge

This misconception of relative volume is what leads us to think that the relatively small top portion of a pint glass could not contain such a large percentage of the total volume. In fact, if beer is poured into a pint glass to about 1/2 inch from the top, 13% of the beer is GONE.

A pour to about 1 inch from the top of the glass leaves out 25% of the beer. This is illustrated below, where the "Beer Gauge" is used to indicate the amount of beer missing from the pint glass.

Your Bonus

The PCI Logo’d BEER GAUGE!

Use Responsibly

Paperless Calibration Programs

Why Paperless Calibration?

Why Paperless Calibration?

Digitizing and automating calibration has decreased downtime and improved efficiency

Through better calibration testing and setting of tolerances, firms will find a great cost reductions in operations and fewer Out of Tolerance investigations.

Understand and accept that in the world of Calibration, everything around us is changing, it doesn’t matter if it’s the calibration procedure, calibration records, our calibration standards, instrument owners, or instrument use.

Investigation Activity

Features of a Paperless Calibration Program

Features of a Paperless Calibration System

Investment in a paperless system is easy to justify; a few of the basic advantages include: • Fewer errors occur because planners, managers, and

technicians are reliably notified of pending calibration events.

• Calibration obligations cannot be forgotten or overlooked. • Technicians can no longer perform a calibration without

properly documenting. • The use of unsuitable or out-of-calibration test standards is

no longer an option. • A paperless system ensures that everyone who needs to

know of a specific event is notified in a timely fashion Example: if a device is found out of tolerance or if a calibration failure occurs, notifications are automated.


• Worker training and qualifications • Facility documentation • Work flow diagrams • Sample details • Work requested • Work performed (procedures, calibrations, etc.) • Results, reports, graphs, etc.

Records are auditable Records are accessible (by others, and from

year to year) Provides a framework to implement/prove

work controls Automated report generation Records tend to be more complete than

notebook entries. That is, all supporting materials (spreadsheets, plots, models, figures, references, protocol, etc.) are attached, too


Overall Program Compliance Improvement

Can be set up to ensure Technicians do not leave

instrumentation out of compliance or hard document GMP issues via the business rules of the application and real time warnings, alerts, and work order application rules.

All Measure Data Test Points can be loaded and

Technicians must follow decision data of Calibration Tolerance limits,

Adjustment Limits and Process Limits. Each of these can

have a business rule that correspond to an application of the system electronically notifying the designated party via email, and shows pending days, hours, etc of work pending.

Culture Challenges

• Anything new is initially resisted: the historical notebook and notebook pages are gone.

• Scientists may hate paper, but experience separation-anxiety in its absence.

• Your notebook is not your private space anymore: viewed by all, pre-signature.

• E-signature compliance is easier to monitor, but resented.

Culture Challenges

•Data and results may be stored on separate computers

•Handwritten notes on paper

•Generated data files are stored on a network server (Linux based, dedicated to and physically located in my lab, backed up corporately) that has terabytes of storage available, and room to expand

•Experimental results are communicated to customers via memos, printouts, email, etc.

Break Anyone?

Paperless Program Considerations

This is a PROJECT.

Paperless Benefits: Business Areas/End Users Engineering Department QC Laboratories Manufacturing Departments R&D Laboratories Identification and specification of business area

and end user needs Identification and specification of the intended

uses of the system Review and approval of the User Requirements

specification document

Paperless Program Considerations Realistic Expectations

Few Companies Achieve 100% Paperless We see a Continuum from 0 to 100%

“Paperless” It doesn’t have to be all or nothing

Corporate Attitudes to support it vary greatly

Investment rates vary

Facilitating regulatory compliance Completeness of records Ease of searching records Managing who has access to information Review and approvals

- Time savings when generating reports - When recreating the past - When archiving data

Cost savings (by-product of the previous benefits listed)

Benefits of a Paperless Calibration System


Saves time Reduce human error Full Standards traceability Automatic tolerance calculations Automatic flags and notifications for

calibration failure


Productivity Improvement Print Calibration certificates with pre-

populated data Measurement Data Template reuse Reduce redundant data entry Reduces head count or maximizes resource

allocation


Improved Compliance – Automated reviews – Less data errors – Faster inspections and audits – Lowers costs related to Paper, filing,

storage


Management Of Work Work Scheduling Work Assignment Calibration Planning SOP Measurement Data Template Standards

Benefits of a Paperless Calibration System (Recap)

History Record Permanent record Reviews & Approvals Consistent Service Delivery Auto Notifications Audit Trail

Making It Happen

Assess Infrastructure Wireless Mobile Computing Laptops, tablets, netbooks PDA’s, cell phones, etc.

Network Drops Cellular cards

The ‘Take Away’

A solid Calibration Program is a must in business and science today

A paperless Calibration Program is a commitment from the top

The benefits of a paperless Calibration Program far outweigh the cost IF this approach is right for your organization

Developing & Implementing Calibration Programs

Q & A

THANK YOU

Wm. Andy Ferrell PCI, LLC

'Providing Instrument & Equipment Compliance Solutions Since 1996'

8100 Brownleigh Ste 100-A

Raleigh, NC 27617 [email protected] (919)781-7787 Office