Produced by: FOI Services, Inc. 704 Quince Orchard Road • Suite 275 Gaithersburg MD 20878-1770 USA

Phone: 1-800-654-1147 or +1-301-975-9400Fax: 301-975-0702Email: infofoi@foiservices.comwww.foiservices.com

TC001688A

Spreadsheet Validation: Ensuring Compliance with 21 CFR Part 11

by Daniel Waterman Ofni Systems, Inc.

Date: Thursday, February 7, 2013Time: 1:00pm – 2:30pm Eastern Standard Time (GMT/UT 1800)

12:00pm – 1:30pm Central Time 11:00am – 12:30pm Mountain Time 10:00am – 11:30am Pacific Time

Teleconference Course MaterialsYou may duplicate this for each person attending the conference.

Call-In: Dial 1-888-848-0354 approximately 10-15 minutes prior to the start time. When prompted, enter the PIN code followed by the # key: 2420192# Outside the U.S. & Canada, you will receive special instructions by email. If you become disconnected and cannot reconnect through the number above, you can also call in on +1-801-853-0800.

At the conclusion of the conference, an audio file will be made available for order. Attendees receive a special reduced price of $225. To order go to www.foiservices.com/shelf0192.htm or call 301-975-9400.

Important NoticeThe information provided in this course by the instructor is his/her personal opinion and does not necessarily

represent the opinions of FOI, Inc. or its staff. Companies relying on the information do so at their own risk and assume the risk and any subsequent liability that results from relying on the information.

The information provided does not constitute legal advice.

Computer Systems Compliance

Spreadsheet Validation: Ensuring Compliance with 21 CFR Part 11

Developed & Presented by

Daniel Waterman Ofni Systems

February 7, 2013 An FOI Services Teleconference

Daniel Waterman Ofni Systems Inc.

Ofni Systems 2

About the Speaker

Daniel Waterman is the Validation Manager for Ofni Systems, Inc., a software provider and Part 11 consulting firm. He has a strong background in software compliance and computer system validation, and has managed numerous spreadsheet validation projects for clinical, laboratory and quality groups.

Daniel has over a decade of experience working for and consulting to FDA-regulated companies, including compliance and validation training, managing validation projects and helping companies to develop and maintain the procedures required for regulatory compliance.

If you have any questions about this presentation, please contact Daniel directly at Daniel.Waterman@OfniSystems.com.

Ofni Systems 3

Presentation Outline

• Introduction • Tips for designing compliant spreadsheets • Spreadsheet validation methodology • Defining user/functional requirements • Documenting design specifications • Best practices for testing inputs, calculations,

macros and security

Ofni Systems 4

Intended Audience • Managers and staff who are responsible for providing valid GxP data

for spreadsheets. • Anyone who designs spreadsheets for use in GxP environments. • Regulatory professionals who must defend the compliance

of spreadsheets. • Quality professionals who may audit spreadsheets or are

responsible for the administration of procedural requirements for spreadsheet compliance.

• Information technology professionals responsible for administrating files or network locations storing spreadsheets containing GxP data.

• Computer validation professionals working with spreadsheets for use in GxP environments.

• Executives planning a company’s future.

Ofni Systems 5

Terminology

• Calculation: Any mathematical/logical operation performed by spreadsheet. Synonymous with formula.

• Macro: User defined function performed by the spreadsheet.

• Workbook: The entire spreadsheet. In MS Excel, this is the entire MS Excel file.

• Worksheet: An individual page within the spreadsheet. In MS Excel, this is each tab within the Workbook.

• Validation: A documented process demonstrating that software meets a defined set of requirements.

Ofni Systems 6

FDA Expectations for Compliance & Validation

The Code of Federal Regulations (CFR) makes it clear that all spreadsheets containing GxP or quality data must be compliant with 21 CFR 11 and the data within must be controlled:

21 CFR 11.10: Persons who use closed systems to create, modify, maintain, or transmit electronic records shall employ procedures and controls designed to ensure the authenticity, integrity, and, when appropriate, the confidentiality of electronic records, and to ensure that the signer cannot readily repudiate the signed record as not genuine. Such procedures and controls shall include the following: 21 CFR 11.10(a): Validation of systems to ensure accuracy, reliability, consistent intended performance, and the ability to discern invalid or altered records.

FDA is emphasizing careful enforcement of Part 11, as expressed in this quote from an FDA official:

“Industry has had since August of 2003 to understand how we intend to enforce Part 11 as per the guidance and be in compliance. Therefore, the intention of CDER to take appropriate enforcement action should not be a surprise.”

Ofni Systems 7

Spreadsheet Warning Letters A Few Examples -

April 2010: "Your firm's laboratory analysts have the ability to access and modify the formulas in the Excel spreadsheets used to calculate assay results for Guaifenesin and [redacted] drug products. Due to this unrestricted access, there is no assurance that the formulas in the Excel spreadsheets are accurate and valid.“

June 2010: "Your firm has no procedures for controlling laboratory worksheets including changes to worksheets, issuance of laboratory worksheets, and reconciliation of laboratory worksheets. ... This sheet is used to communicate batch release information to warehouse operations. Use of this sheet is not documented in any Medefil procedure nor is control over this sheet documented."

October 2010: "We observed 8 of 9 worksheets where one or more tabs with formula cells were not locked. These worksheets were used for analyzing raw data from drug component and product samples, including [redacted]. Your firm's SOP 100-G-0110, 'Creation and Use of Templates,' stated that cells, in which data is entered, must be locked within their electronic template."

Ofni Systems 8

Why Validate? (FDA tells us we have to…and it’s good business practice.)

21 CFR 11 (and Annex 11) ensures control over data from creation to final use. Compliance with 21 CFR 11 means that you (and your customers) have full confidence in the data that your organization stores electronically. Validation ensures that spreadsheets are appropriate to their function and calculations/macros performed as expected. Validation provides a systematic verification that the spreadsheets work as expected.

Ofni Systems 9

Which Spreadsheets Require Validation?

• Spreadsheets which control GxP data – Clinical: Research data, results data, etc. – Laboratory: Lab data, especially QC results

• Spreadsheets used to make quality decisions – Documented evidence of SOPs – Tracking Logs: Internal audits, approved supplier lists,

approved batches, etc.

• Any spreadsheet containing data which is reported to a regulatory agency

Ofni Systems 10

Tips for Designing Compliant Spreadsheets

Best practices include: – Use standardized design/colors to indicate data entry

and calculation cells • Yellow for Data Entry (An asterisk was added as a design element in the

completed spreadsheet to accommodate black and white hard copy) • Green for Calculations (“Sum” cells in example below)

* * * *

Tips for Designing Compliant Spreadsheets

– Restrict data entry to appropriate cells. • Format Cells Protection Tab

– Identify formulas, other cells as locked – Identify data entry cells as unlocked (Locked is unchecked)

• Apply protection to the worksheet • This also prevents formulas from being altered

– Use data verification (Data Menu Data Validation) to

restrict data inputs to appropriate results. • Data lookup tables can be placed in hidden columns or

worksheets • Messages can inform users of correct data inputs

Ofni Systems 11

Ofni Systems 12

Tips for Designing Compliant Spreadsheets

• Use formatting to specify data inputs (Format Cell; in Excel 2007, see the number toolbar ribbon).

• Use standardized coding standards (e.g., comments, parameters, returns, history) for macros.

• Apply protection to workbook to prevent sheets being added, renamed or deleted.

• Where practical, create spreadsheets as templates.

• Remove any blank worksheets.

• Procedurally require that compliant spreadsheets are stored in defined network locations. Use IT controls to limit access to spreadsheets to appropriate personnel.

Ofni Systems 13

Sample Macro Template

Ofni Systems 14

Spreadsheet Validation

• Validation Methodology

• Required Documentation

• Spreadsheet Risk Assessment

• Defining Requirements

• Testing Spreadsheets

Ofni Systems 15

Spreadsheet Validation Methodology • Define basic documentation practices and methodology in

a single document, the Spreadsheet Validation Master Plan.

– Ensure all key parties (e.g., end users, validation, quality) agree to the Master Plan in advance

– The Master Plan should be referred to for all individual spreadsheet validation projects

– Consistent use of a Master Plan offers the advantage of not requiring review/approval for each validation effort

Ofni Systems 16

• Define the requirements for each sheet, toolbar, add-in and chart in the workbook, then test to verify requirements. Heavy emphasis is placed on: − Defining and testing calculations and macros − Security for each sheet by limiting the parts of each sheet that

the users are allowed to edit

• If possible/practical, validate spreadsheets as templates

• The Requirements Trace Matrix ensures all requirements are tested

• A Risk Assessment focuses validation as required

Spreadsheet Validation Methodology

Ofni Systems 17 From: GAMP 5, Appendix S3, Figure S3.1, Page 295

Ofni Systems 18

Spreadsheet Validation Documentation

• Category 3 (Simplest mathematical operations, using Excel to format printed logs)

– User/Functional Specification – Documented verification of calculation accuracy – Documented testing demonstrating that spreadsheet is protected against

unauthorized changes – Documented testing demonstrating users can only access approved version

• Category 4 and 5

– Documentation as required for Category 3, plus: – Validation Plan – Design Specification (Can be combined with User/Functional Specification) – Test Protocol – Requirement Traceability – Validation Summary Report

Ofni Systems 19

Spreadsheet Risk Assessment

• High Risk – Calculations – Macros – Spreadsheet protection

• Medium Risk – Restricted Data Entry – Conditional Formatting, particularly for Alert displays – Charts – Labels which provide GxP directions

• Low Risk – Unrestricted data entry – Other Label text – Other spreadsheet functionality: sorting, formatting, etc.

Ofni Systems 20

Spreadsheet User/Functional Requirements

• Follow existing procedures for writing FRS/URS

• Best Practices: Use Templates – Standardize Section Headings

• Introduction: Table of Contents, Document Change History, Objectives, Scope, Assumptions, Exclusions,

• References: External Document References, Acronyms, Abbreviations

• Spreadsheet Description • Spreadsheet Interface Requirements • Regulatory Requirements • Business Requirements

– Provides standardized, re-usable text for commonly repeated sections of the document, including Assumptions, Regulatory Requirements, etc.

Ofni Systems 21

• List all Worksheets, custom toolbars, add-ins and charts in the workbook

• For each Worksheet, define: – Expected data entry, including Field Names, allowable values,

conditional formatting, etc. – Calculations, including description of the calculation (Example: Acceleration = Force/Mass) – Worksheet output or final calculations

• For each custom function (Macros, Add-Ins, Toolbars) define function names and purpose

• For each chart, define requirements for the title; axis labels and units; the datasets to use to create the chart

Spreadsheet User/Functional Requirements

Ofni Systems 22

• Worksheet A: – Allows users to enter Date of Test. Date of Test cannot

be a future date. Date of Test has the default value of the current date.

– Allows users to enter 20 values of Cell Count. Cell count must be a positive integer.

– Calculates Volume in Cell B20, using the formula Volume = 4 * Pi * Radius3 / 3

– Records all data additions, edits and deletions to the audit trail.

Spreadsheet User/Functional Requirements

Ofni Systems 23

Sample User/Functional Requirements

• Toolbar A: – Allows users to initiate the Collect Data operation. – Exports Collected Data to a CSV file.

• Chart Y: – Displays calculated Pressure, from entered Volume. – Displays data from Cell Range B5:F5 – Displays Alert Level and Action Level

Ofni Systems 24

Log Example

Ofni Systems 25

Log Example – Functional Requirements

• Allows users to enter Log ID (alphanumeric, 5 characters or less) for each entry.

• Allows users to enter description for each entry.

• Allows users to enter Pass/Fail for each entry. No other data entry is allowed.

• Properly formatted for printing.

Ofni Systems 26

Calculation Example

Ofni Systems 27

Calculation Example – Functional Requirements

• Allows users to enter 30 values of Sample ID. Sample ID is an alphanumeric string of exactly 6 characters.

• Allows users to enter 30 values of pH. Entered data must be a rational number between 0.0 and 14.0.

• Allows users to enter 30 values of Volume. Entered data must be rational numbers greater than 0.0.

• Calculates Volume2. Users cannot edit or alter the formula.

• Calculates the Maximum, Minimum and Average of Volume. Users cannot edit or alter the formulas.

• Produces chart of pH vs. Volume2.

• Properly formatted for printing.

Ofni Systems 28

Formatting Example

Ofni Systems 29

Formatting Example – Functional Requirements

• Allows users to enter Sample ID and Date of Test. Date of Test cannot be a date in the past.

• Allows users to enter 3 values. Values must be rational numbers greater than 0.

• Calculates the standard deviation of the three entered values. Users cannot alter or edit the formula.

• If the standard deviation is greater than the defined alert level (1.0), the standard deviation is displayed in bold orange text. If the standard deviation is greater than the defined action level (2.0), the standard deviation is displayed in bold red text.

• Displays the status of the sample. Users cannot alter or edit the status. – If three values have not been entered, the Status displays "Enter 3 values" – If the Standard Deviation is less than the Action level, the status displays

"Sample Passes" – If the Standard Deviation is greater than the Action level, the status displays

"Sample Fails. Do not use sample."

Ofni Systems 30

Requirements for Spreadsheet Compliance

(a) Validation

(b) Accurate and complete copies of records

(c) Protection of records

(d) Limited system access

(e) Audit trails

(f) Operational system checks

(g) Authority checks

(h) Device checks

(i) Training

(j) Written policies for Electronic Signatures

(k) Control over system documentation

Ofni Systems 31

Regulatory Requirements – Procedural Controls

Document SOPs used to meet requirements.

• Validation [21 CFR 11.10(a)] Spreadsheet AA will be validated according to SOP VAL-003 “Spreadsheet Validation”. This Functional Specification is part of the validation documentation.

• Accurate Record Generation [21 CFR 11.10(b)] Spreadsheet AA is stored as an Excel spreadsheet template (.xlt file) which is both human readable and in an electronic form suitable for Agency inspection, review, and copying. This is tested implicitly throughout the validation and does not require explicit verification.

Ofni Systems 32

Regulatory Requirements – Procedural Controls

• Protection of Records [21 CFR 11.10(c)] Data in Spreadsheet BBB will be stored on network file location QC-Labs. All files stored in QC-Labs are backed up according to SOP IT-002 “IT Backup/Recovery Procedures”

• Training [21 CFR 11.10(i)] Training on Spreadsheet CCC will be performed according to SOP-002 “Training Policies, Procedures and Documentation”

Regulatory Requirements – Procedural Controls

• Electronic Signatures [21 CFR 11.10(j)] Discuss if spreadsheets use electronic signatures. If so, describe the appropriate policies. If the spreadsheet does not use electronic signatures, acknowledge the requirement, justify that the requirement does not apply to the spreadsheet.

• Controls over System Documentation [21 CFR 11.10(k)] Use and administration of Spreadsheet GGG is described in SOP-101 Performing Sample Analysis. This SOP will be maintained according to SOP-001 “Document Creation and Maintenance”.

Ofni Systems 33

Ofni Systems 34

Regulatory Requirements – Limited System Access

• Limited System Access [21 CFR 11.10(d)] − Spreadsheet AA will be stored in Network File Location QQ,

which is only accessible to members of the QC Group. − Spreadsheet BB can only be opened in “Read-Only” mode.

Once opened, the user edits the spreadsheet data and prints the spreadsheet to hard copy.

− Access to Spreadsheet CC will be through System EE, which requires users to provide a password before opening the spreadsheet.

Ofni Systems 35

Regulatory Requirements – Audit Trails

Audit Trails (21 CFR 11.10(e)) – If the electronic data is not retained: Spreadsheet AA is

only used as a template and electronic data entered into the spreadsheet is immediately printed to paper. The paper document is signed and stored according to SOP-12345.

– Alternatively: Spreadsheet will produce an audit trail. Note: Excel’s Track Changes feature is not a 21 CFR 11 compliant audit trail.

Ofni Systems 36

Regulatory Requirements – Other Technological Controls

• Authority Checks [21 CFR 11.10(g)] Describe any checks that users can only perform desired operations.

• Device Checks [21 CFR 11.10(h)] Describe any external data input tools. If a requirement is not appropriate to the spreadsheet, acknowledge the requirement, document and continue.

Example of Device Checks [21 CFR 11.10(h)]: Beyond the keyboard, which will be implicitly tested throughout the validation, the AAA Spreadsheet does not use any external data entry tool. Therefore, there are no explicit device check requirements.

• Operational System Checks [21 CFR 11.10(f)] Describe any technological controls which enforce that particular data entry or operations are performed in a specific order.

Ofni Systems 37

Business Requirements

• System Interactions (Computer Systems, Equipment)

• Performance Requirements

• System Availability

• Physical Environments

Note: For many spreadsheets, these will not be applicable. Acknowledge the lack of requirement, document and continue. Example: The AAA Spreadsheet does not interact with any computer system not described in this specification.

Ofni Systems 38

Spreadsheet Design Specifications

• Create Design Specifications according to existing procedures

• Design Specifications should focus on customized sections of spreadsheet (as opposed to spreadsheet functionality); e.g., your calculations, not the program’s Copy/Paste function.

• Spreadsheet configuration will be minimal – In most cases, limited to security

Note: For MS Excel, Protect Sheet screen: Review Toolbar Protect Sheet

Ofni Systems 39

Spreadsheet Design Specifications

• Include Sections based on – Worksheets and Toolbars/Add-Ins:

o Inputs (Data Entry) o Processing (Calculations, Macros, etc.) o Outputs (Exported files, imbedded charts, etc.) o Security

– Chart Worksheets: o Inputs (Data ranges, including defining axis) o Formatting (Type of Chart, Chart Keys/Labels, etc.)

Ofni Systems 40

Documenting Spreadsheet Inputs

• Document Cells: Users are expected to enter/update data. − Include Cell/Range Name, title, format, other information − Document defined data validation rules used to enforce

proper data entry − In an automated system, define the source or the input

data or instruction. − If data entry comes from source other than the keyboard,

define the data input source and procedure for data entry − Suggestion: Use charts to display groups of cells

Ofni Systems 41

Documenting Input Cells Log Example

Title Cell/ Range

Format Other

Log ID Column A (A5 and below)

General Alphanumeric string <= 5 characters

Description Column B (B5 and below)

General N/A

Pass/Fail Column C (C5 and below)

General Values restricted to Pass/Fail

Ofni Systems 42

Documenting Input Cells Calculation Example

Title Cell/Range Format Other

Sample ID Range B6:B35 General Sample ID must be 6 Character Alphanumeric String

pH: Range C6:C35 Numeric (0.0) Value between 0.0 and 14.0

Volume Range D6:D35 Numeric (0.00) Value greater than 0.0

Signature G36 General N/A

Ofni Systems 43

Documenting Input Cells Formatting Example

Title Cell/Range Format Other

Sample ID Cell C9 General N/A

Date of Test Cell C10 Date dd/mm/yyyy

Value <= Today()

Values C12, C14, C16 Numeric (0.0) Value greater than 0.0

Ofni Systems 44

Documenting Spreadsheet Processing Calculations & Formulas

Document calculations performed by the spreadsheet: - Include cell name, title, formula, format, other information (conditional formatting, etc.)

- Best Practice: Document all formulas, as opposed to ranges of formulas

- Best Practice: As the Design Specification is documented, compare with the Functional Specification

Note: The majority of errors I have seen when validating spreadsheets is in the formulas. The best way I’ve found to catch these errors is to define each formula using the actual names of the variables represented as input cells.

Ofni Systems 45

Documenting Calculation Cells Calculation Examples

Calculates the Maximum, Minimum and Average of Volume Title Cell Formula Format Other Max(Volume)= H6 =Max(D6:D35) Numeric (0.0) N/A Min(Volume)=

H8 =Min(D6:D35) Numeric (0.0) N/A

Average(Volume)= H10 =Average(D6:D35) Numeric (0.0) N/A

Calculates Volume2 Title Cell Formula Format Other Volume2 (Entry 1) E6 =D6*D6 Numeric (0.00) N/A

Volume2 (Entry 2) E7 =D7*D7 Numeric (0.00) N/A

Volume2 (Entry 3) E8 =D8*D8 Numeric (0.00) N/A

… … … … …

Ofni Systems 46

Documenting Calculation Cells Formatting Example

Title Cell Formula Format Other Displays the status of the sample. Status G14

=IF(AND(NOT(C12=0),

NOT(C14=0), NOT(C16=0)), IF(G12>=2, "Sample Fails. Do not use sample.", "Sample Passes"), "Enter three values")

WHERE …

General Conditional Format: Bold, Red if Value = “Sample Fails. Do not use sample."

Title Cell Formula Format Other Calculates the standard deviation of the three entered values. Standard Deviation

G12 =STDEV(C12,C14,C16) Numeric (0.0) Conditional Format: Orange, Bold if Value > 1.0, Red, Bold if Value > 2.0

Ofni Systems 47

Documenting Calculations See the requirement for Cell G14 on the formatting example Displays the status of the sample. Users cannot alter or edit the status. – If three values have not been entered, the Status displays "Enter 3

values" – If the Standard Deviation is less than the Action level, the status

displays "Sample Passes" – If the Standard Deviation is greater than the Action level, the status

displays "Sample Fails. Do not use sample.“

Is this formula correct? =IF(AND(NOT(C12=0), NOT(C14=0), NOT(C16=0)), IF(G12>=2,

"Sample Fails. Do not use sample.", "Sample Passes"), "Enter three values“)

It can be very difficult to inspect this formula and determine if it is correct.

Ofni Systems 48

Documenting Calculations

Formula: =IF(AND(NOT(C12=0), NOT(C14=0), NOT(C16=0)), IF(G12>=2, "Sample Fails.

Do not use sample.", "Sample Passes"), "Enter three values")

Suggestion: - Write formulas out explicitly in the Design Specification. - Define each cell and walk through each logical step:

C12, C14, C16 are Entered Values G12 is Standard Deviation IF any of C12, C14, C16 (Entered Values) are 0, THEN G14 displays “Enter 3 Values” OTHERWISE IF G12 (Standard Deviation) is greater than 2, THEN G14 Displays “Sample Fails. Do not use sample.” OTHERWISE G14

Displays “Sample Passes”

Ofni Systems 49

Documenting Calculations

This is also a good strategy with more complicated mathematical calculations. Example: A spreadsheet calculates pressure, using ideal gas law. Cell E5 =(A10*B10*$C$5)/($D$5)

Pressure = (moles of Gas) (Constant) (Temperature) Volume E5 = Pressure A10 = Moles of Gas B10 = Gas Law Constant C5 = Temperature D5 = Volume

Ofni Systems 50

Processing: Macros, Toolbars & Add-Ins

For each Macro/Toolbar/Add-In Function: – Define all required inputs for macro. – Describe operation performed by macro. – Copy the code and annotate as needed to describe the

purpose of each macro or function. – Describe any limitations who may use this function and when

the function may be used.

Note: If you have good coding standards and use proper headers and comments in your code, this step may be already done.

Ofni Systems 51

Documenting Spreadsheet Outputs

Outputs usually fall under one of three main categories: – The cell or range of cells that contain the final result

of all previous calculations – Charts: Many times these are printed and saved with external

reports – Data that is copied into a final result sheet or exported

to a separate file or database

Ofni Systems 52

Security • Describe how the individual spreadsheet is secured to

prevent users from altering calculations/functionality. • Examples of security requirements include:

– All non-input cells are locked to prevent changes. – The spreadsheet is stored in file location A, which prevents

users from deleting the original file. – Spreadsheet is stored within B, which controls access to the

software and ensures that the original spreadsheet file cannot be edited from outside the program.

Note: Be sure to include macros or third-party software. Tools exist which allow users to include additional security settings to implement multiple levels of security to control who can edit certain cells, run a macro or function, etc.

Ofni Systems 53

Combining User, Functional & Design Specifications

• GAMP5 explicitly allows combining of User, Functional and Design Specifications based on system complexity

• With the approval of your quality group, Functional and Design Specifications can be combined into a single document

• Combining specifications reduces the time/resources spent creating, approving and maintaining documents

Note: If you combine specifications, design requirements can be recorded with the functional requirement. Alternatively, design specifications can be put in their own section.

Ofni Systems 54

Testing Spreadsheets: Test Protocols

• The testing of any spreadsheet should prove that the requirements were properly implemented according to the Design Specifications.

– Installation Qualification (IQ) testing is usually limited to documenting the test environment, verifying that the file is in a location where users can access and use the file and confirming the security settings.

– Operational/Performance Qualification (OPQ) testing is primarily about verifying formulas and macros. It also includes the testing of the security of each sheet to verify all non-input cells are locked to prevent changes.

Ofni Systems 55

Installation Qualification • Document Operating System • Document version of spreadsheet program • Confirm/Document spreadsheet file format and file location;

e.g., verify spreadsheet is using correct version of software • Confirm security configuration used to secure spreadsheet

(Tools Protection Protect Sheet) • If third party software is used to provide functionality, confirm

installation/version • If spreadsheet references external files, confirm installation/version

of the external files, including confirmation that the spreadsheet can access the files

• If spreadsheet requires specific file locations, confirm that the testing location is correct

• If spreadsheet requires external data entry source, confirm installation and configuration

Ofni Systems 56

Operational Qualification Test cases should be written for: • Worksheets

– Document if certain worksheets require previously entered data entry on other worksheets

– Toolbar/Macro functionality can be included if limited to a single worksheet

• Toolbars/Macros/Add-Ins (Multiple worksheets) – Low Risk: Test functionality on one generic sheet – High Risk: Do a quick test of functionality on all sheets, write a

generic test case which is a more complete test of functionality

• Specific workflows, if applicable • Compliance features • Security

Ofni Systems 57

OQ – Input Testing

Questions to consider when testing inputs:

– What data can be entered into each input cell?

– Can data be entered into an inappropriate cell?

– Are validation rules being enforced?

– Are formats displayed as expected?

– Is all data input through the keyboard? If not, has the alternative data input been tested?

– Is data being imported from an external source? (If so, verify that data is being imported properly.)

Ofni Systems 58

Input Test Log Example

Requirement: Allows users to enter 30 values of Sample ID. Sample ID is an alphanumeric string of exactly 6 characters.

Procedure Expected Result

Navigate to Cell Range C5:G5, Volume. Enter appropriate data (an alphanumeric string 6 characters long) in each cell.

All cells in Cell Range C5:G5 accept numeric data.

Ofni Systems 59

Input Test Formatting Example

Requirement: Allows users to enter Date of Test. Date of Test cannot be a date in the past. Procedure Expected Result Navigate to Cell C10, Date of Test. Enter the the current date.

Cell C10 accepts the data entry. The entered date has the format mm/dd/yyyy.

Enter a date one day earlier than the current date.

A message box appears, informing the user that the entered date is not valid.

Ofni Systems 60

Testing Calculations Focus on Testing the Calculations: 1. Verify formula accuracy – Is the formula the one you

intended?

2. Verify correct cells used in formula – Are you using the correct cells to model the formula?

3. (Calculation Range) Verify all cells use the correct relative formula – Are all formulas in the range using the correct cells, or have some formulas begun to “drift”?

4. Do the calculations produce the expected results?

5. Can the formulas be modified?

Ofni Systems 61

Calculation Test Example

Procedure Expected Result

Navigate to Cell Range E6:E35. Each cell in Range E6:E35 should be the square of the corresponding cell in Range D6:D35. Select a cell in Cell Range E6:E35 and verify that the cell correctly modifies the intended calculation.

Cell and calculation recorded. The calculation in the selected cell correctly models the intended calculation.

Verify that all cells in the Cell Range E6:E35 use the same calculation in relative terms.

All cells in E6:E35 use the same calculation in relative terms.

Select a cell from the Cell Range. Manually perform the calculation performed by the cell and verify that the manual calculation matches the spreadsheet calculation.

The manual calculation matches the spreadsheet calculation.

Verify that the user cannot modify any of the calculations in E6:E35.

Users cannot modify the calculations in Range E6:E36.

Ofni Systems 62

Calculation Test Example Procedure Expected Result Navigate to Cell G14. Verify that if Value

1, Value 2 or Value 3 is either 0 or NULL, Cell G14 displays “Enter three values.”

If Value 1, Value 2 or Value 3 is either 0 or NULL, that Cell G14 displays “Enter three values.”

Enter appropriate values in Value 1, Value 2 and Value 3 such that the Standard Deviation approaches (but does not exceed) 2. (For example, enter 1, 1, and 4.3 in the three cells.) Verify that the Status displays “Sample Passes”.

The Status displays “Sample Passes”.

Enter appropriate values in Value 1, Value 2 and Value 3 such that the Standard Deviation is greater than 2.0. (For example, enter 1, 1, and 4.5 in the three cells.) Verify that the Status displays “Sample Fails. Do not use sample.”

The Status displays “Sample Fails. Do not use sample.” The text is red and bold.

Verify that the tester cannot modify the calculation in Cell G14.

The tester cannot modify the calculation in Cell G14.

Ofni Systems 63

Testing Processing Macros, Toolbars, Add-Ins

• Testing Macros, Toolbars and Add-Ins depends on data manipulation – Inputs: Do sufficient testing to verify function is

selecting appropriate data – Processing: Verify that calculation is as intended – Outputs: Verify that outputs meet requirements – Security: Verify that only appropriate users can trigger

function

Ofni Systems 64

Testing Charts

• Verify chart input.

• Verify calculations, if appropriate.

Note: The most common mistake with charts is not using the correct or complete set of data (or pointing to the wrong columns of data).

Note: Charts can be tested by a combination of visual inspection and verification of the properties, including the dataset used as the basis for the charts.

Ofni Systems 65

Charts Example

Procedure Expected Result Navigate to the Chart “Plotting pH vs.

Volume2” on the Calculation Example worksheet. Unprotect the worksheet. Right click on the chart and select Source Data. Select the Series tab.

The Source Data screen opens.

Verify that the X Values for the chart are from Range C6:C35.

The X Values for the chart are from Range C6:C35.

Verify that the Y Values for the chart are from Range E6:E35.

The Y Values for the chart are from Range E6:E35.

Close the Source Data screen and protect the worksheet.

The Source Data screen closes. The worksheet is Protected.

Ofni Systems 66

Security Testing The Type/Intensity/Quantity of security testing is largely based on how security has been implemented.

Verify that users: – Are limited to entering data into the defined input cells only – Do not have the ability to alter any other part of the spreadsheet

without detection. – Can open the spreadsheet from all required locations.

Best Practice: Verify that formula cells are locked as part of routine template test steps.

Note: Failure to do secure a spreadsheet can compromise the integrity of the validation effort and of any data or information generated by the workbook.

Ofni Systems 67

Summary: Spreadsheet Validation

• Establish a Spreadsheet Validation Methodology – Use templates for documentation – Combine Specifications, Test Protocols into single documents – Validate Templates when possible

• Focus validation on High Risk sections: Calculations, Macros, File Security – Incorrect calculations – Using incorrect data inputs

• Ensure that the spreadsheet cannot be altered – Read-only file formats – Protected worksheets/workbooks – Third-party solutions which add 21 CFR 11 controls

Ofni Systems 68

Thank You! Questions? All questions are welcome. To protect your privacy, questions will be announced using only the first name reported to the operator at dial-in. If you think of a question later, or would like to arrange a private consultation, you are invited to contact Daniel Waterman directly at +1-919-844-2494 or Daniel.Waterman@OfniSystems.com. More information about Ofni Systems is at www.OfniSystems.com

We value your feedback. Please complete and return the evaluation to be entered into a drawing for a

$100 Amazon.com gift card and/or to receive a Certificate of Attendance verifying your 1.5 contact hours of continuing education.

Additional Information. An online copy of this handout, the full spreadsheet referenced during the

session, and instructions for ordering the audio of this session at the 50% attendee discount are available at www.foiservices.com/shelf0192.htm

Training Evaluation Form Spreadsheet Validation: Ensuring Compliance with 21 CFR Part 11Daniel Waterman, Presenter – February 7, 2013

Win a $100 Amazon.com Gift Certificate from FOI!Just complete this form and fax it to +1-301-569-7506 by February 14, 2013

and you will be entered in a drawing for an Amazon.com Gift Certificate

Please indicate your primary job responsibility (circle one):

Regulatory Affairs QA/QC Statistics Medical Writing Electronic Submissions General Management

Other (please specify):____________________________________

Please rate various aspects of this course (1 = Poor; 2=Fair; 3=Good; 4=Very Good; 5=Excellent – please circle your answers)

What is your overall rating of the course? 1 2 3 4 5

How do you rate the content of the course? 1 2 3 4 5

Were course materials clear and understandable? 1 2 3 4 5

Was the length of the course adequate to cover the content? 1 2 3 4 5

Was the instructor knowledgeable about the subject matter? 1 2 3 4 5

How would you rate the instructor overall? 1 2 3 4 5

Other comments: Please feel free to comment on any aspect of this course, including the instructor, content, and technical arrangements:

TC001688

By providing the information below and faxing this page to +1-301-569-7506 by Thursday, February 14, 2013 you will be entered in a drawing for a $100 Amazon.com Gift Certificate from FOI.

Your Name: _______________________________________________________________________

Company Name: ___________________________________________________________________

Email Address: ____________________________________________________________

Check here to receive a Certification of Attendance verifying your 1.5 training contact hours for this educational session. Please provide your legibly printed name with your signature next to it, email address, and fax this form to the number below. Your certificate will be emailed to you.