audit trails
TRANSCRIPT
TM Subject:
The HollisGroup, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 1
Comments on the Utility vs.Burden of Audit Trails
“Audit trails are the single largest cost component of 21 CFR 11 compliance.”
John Doe, presenting at CHPA / FDA 1999
TM Subject:
The HollisGroup, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 2
A Word From Our Sponsor
Subpart B—Electronic Records§ 11.10 Controls for closed systems.
…Such procedures and controls shall include the following:
(e) Use of secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records. Record changes shall not obscure previously recorded information. Such audit trail documentation shall be retained for a period at least as long as that required for the subject electronic records and shall be available for agency review and copying.
TM Subject:
The HollisGroup, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 3
Part 11’s Literal Meaning
• The only transactions that need audit trails are ones performed by “operators”
• The only data that is required to be in the audit trail itself is the date and time– This means we do not have to replicate data from
the transaction in the audit trail– Technically, we do not we do not even need to
record the operator’s ID• There are some very good reasons to take a
minimalist approach to audit trails
TM Subject:
The HollisGroup, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 4
Audit Trails - Current Pharmaceutical Model
• Audit trails are usually replications of a subset of a transaction record– “Source record” >>> “Audit record”
• Audit records are usually stored in a similar (if not the same) data structure
• Ubiquitously, audit records have the same or lower security level as source records
• Hollis refers to this scheme as“Data-level Audit Records”
TM Subject:
The HollisGroup, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 5
Data-level Audit Records(Creating a New Record)
SOURCE DATABASE AUDIT DATABASE
TM Subject:
The HollisGroup, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 6
Data-Level Audit Records(Correcting a Typographic Error)
SOURCE DATABASE AUDIT DATABASE
TM Subject:
The HollisGroup, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 7
Audit Trails – CurrentFinancial Model
• The term “audit trails” is misleading; these are actually “audited transactions”– System A proposes transaction– System B proposes agreement– System X (the security system) examines
• The data labelling• A’s and B’s privileges• The structure of the transaction
– System X grants permission for the transaction• And keeps a log
– All in real-time
TM Subject:
The HollisGroup, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 8
System-Level Audit Records(Any Type of Transaction)
SOURCE DATABASE JOURNAL FILE
Read:Cust_Rec:tquinn2270;*.*||Writ>:Xact_prop:tquinn2270;Cur_Bal;310.65||Read:ACF_2_Rcpt:Auth_cod:<result>||Writ:tquinn2270:Cur_Bal;310.65:Auth_cod;<result>||
TM Subject:
The HollisGroup, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 9
Comparing the Two
• Data-level audit trails:– Are much easier to program and run– Tend to produce larger record sets– Keep the audit and source data in the format– A MUCH easier to compromise
• System-level audit trails:– Are much more difficult to include in designs– Tend to produce smaller record sets– Keep the audit and source records separate– Are MUCH more difficult to compromise
TM Subject:
The HollisGroup, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 10
Risk Analysis
• Data-level audit records and source data are (about) equally vulnerable to insider threats– Insiders are the most common threat
• Replicating data-level audit records provides outsider adversaries with two attack vectors– It’s more effective to invest in other defenses
• System-level audit records are only useful in prevention if they are used in real-time– In order to assist with detection, they must be
periodically and meticulously reviewed
TM Subject:
The HollisGroup, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 11
Recommendations
• Do NOT change the audit trail wording of 21 CFR § 11.10 (e) to require more information in the audit trail
• Perform a Regulatory Flexibility Analysis to justify the requirement for audit trails, and include details of:– Financial burden of audit trails, particularly upon
small and disadvantaged businesses– Raw and normalized statistics of when audit trails
have been useful in protecting public health
TM Subject:
The HollisGroup, Inc.
Dept. App.
Reg. Aff.
QA
Manuf.
Purch.
R & D
Eng.Infrastructure Assurance
FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn Slide # 12
Questions?
Thomas Quinn, President
The Hollis Group, Inc37 North Valley Rd. #105
Station Square IIPaoli, PA 19301
v: 610.889.7350 f: 610.296.2339