Alexey Khrenov, PhD OTAT/CBER/FDA

CMC Strategy Forum

January 29, 2018 - Washington, DC

Common Issues in Qualification and Validation of Analytical Procedures

2

Disclaimer

• These comments are an informal communication and represent my own best judgment. These comments do not bind or obligate FDA.

www.fda.gov

3

Purpose of Qualification/Validation

• Qualification/validation is the process of demonstrating that the analytical procedure is suitable for its intended use.

• What does “suitable for its intended use” mean for regulatory reviewers?– Can we (and you) trust your past and future

data?

www.fda.gov

4

Validation vs. Qualification• Qualification

– Confirm that the analytical procedure (as performed) is suitable for the specified purpose, and the results are reliable and reproducible.

– Analytical procedures must be qualified, but specifics of qualification are at the company’s discretion based on scientific judgement.

– Qualification reports are not expected, but the company may be requested to provide data to prove that the methods are sufficiently qualified.

• Validation– Can be performed after gaining sufficient knowledge of the analytical

procedure.– Formal exercise to determine the ability of the method to meet the

acceptance criteria pre-defined in the validation protocol for each of the test parameters.

– Validated analytical procedure must be described in sufficient details for it to be performed in FDA laboratory, and produce results comparable to those obtained by the company.

– Validation reports must be included in the BLA/NDA.www.fda.gov

5

Validation vs. Qualification (contd.)– Qualified analytical procedures should always be used in the regulated

environment (IND/BLA stage).– Coordination of the development of analytical methods, manufacturing

process, and product is required • Analytical procedure may need to be re-qualified or re-validated if

the acceptance criteria or matrix changes.• If the analytical procedure is changed, bridging study needs to be

performed. • Do NOT change the manufacturing process and analytical procedure

at the same time!– Commercial process must be supported by fully validated analytical

procedures, i.e., the analytical procedures must be validated BEFORE the commencement of the PPQ campaign.

– Analytical procedures used in clinical trials must be validated.www.fda.gov

6

Risks of Inadequate Validation• Validation failures result in additional development work and delay

regulatory submission.• Data excluded from review of license application• The later the procedure is validated, the greater the risk.

www.fda.gov

Risk mitigation strategy• Define and qualify the performance characteristics of the analytical

procedure that are critical to its intended use from the start.• Develop system suitability procedures and criteria.• Monitor and trend performance of analytical procedure.• Keep gaining knowledge about capabilities of analytical procedures.• Validate the procedure early in product development.

7

Analytical procedure lifecycle

Development

• Initial assessment of the suitability of analytical procedure

• Optimization of analytical procedure details

Qualification

• Assessment of the performance characteristics of analytical procedure

• Assessment of the capability of analytical procedure to support specification

• Gaining knowledge for setting final specification acceptance criteria and design of validation protocol

Validation

• Confirmation that the performance characteristics of analytical procedure are established correctly.

• Confirmation that analytical procedure is capable to support specification acceptance criteria.

www.fda.gov

• Performance characteristics are inadequate to control the product and the manufacturing process.

• Failure to meet validation acceptance criteria

• Changes in specification• Changes in the process

or product• Modification of the

analytical procedure

8

Considerations for Setting Specifications

www.fda.gov

Risk assessment

Clinical safety

/efficacy

Analytical capability

Process capability

• Acceptance criteria for different specification parameters may be limited by different factors.

• Acceptance criteria are frequently adjusted when more data become available.

• Changes in the manufacturing process/ analytical procedures require re-assessment of the acceptance criteria.

• Sound statistical approaches should be used where possible.

• Data to be included in the justification of specification should be carefully selected based on their relevance to the commercial manufacturing process.

9

Full Validation

Validation characteristics• Accuracy • Precision

– Repeatability – Intermediate Precision

• Specificity • Detection Limit • Quantitation Limit • Linearity • Range• Robustness

Additionally, for cross-validation• Comparability

www.fda.gov

10

Validation Acceptance Criteria

• Should be based on intended use, knowledge of procedure performance, and capability to confirm the suitability for its intended use.

• Should be specific for the analytical procedure.

• Should be specific for the validation parameter.

www.fda.gov

11

The Big Problem“We must do validation because FDA wants

validation report”

Leads to • Poor design of the validation study• Lack of scientific assessment of data• Disconnect between validation and intended

use of the method

www.fda.gov

12

Acceptance criteria issues• Acceptance criteria (AC) are too wideRisk: Analytical procedure may be found unsuitable and data excluded

from review.

Example 1

Review of validation is based on the intended use of the analytical procedure. Meeting an irrelevant AC does not confirm suitability.

www.fda.gov

AC for peak position

Actual variation of peak position was 1/200 of AC

Actual RSD% for peak area was 1/25 of AC

Anything will PASS!

13

Acceptance criteria issues (contd.)• Same AC are used for multiple validation parameters and methods

Example 2 – 1/6 of the specification range was used as the range where the

results should fall for Precision validation.– The same value was then used to calculate the AC for standard

deviation (SD) when validating the Accuracy, Repeatability, and Intermediate Precision for multiple methods.

– As a result, an acceptable SD equal to 1/6 of specification range (±3 SD covered the whole specification range) was set as the AC for multiple methods and parameters.

As the approach was defined in the SOP, it resulted in a Form 483 Observation during the facility inspection.

www.fda.gov

14

Typical issues: Linearity • Lack of visual inspection and qualitative assessment of the

linearity data.Risk: Analytical procedure may be found unsuitable and data

excluded from review. Example 3

Linearity should also be evaluated by visual inspection of a plot of signals as a function of analyte concentration or content. A plot of the data should be included. (ICH Q2)

www.fda.gov

No evidence that visual inspection was performed of a plot.

Data show a loss of sensitivity at the lower limit of range.

Procedure was re-validated.

15

Typical issues: Range • Validated range does not cover 80% - 120% of specification

range.Risk: Re-validation of the procedure may be required, part of data may be

excluded from review.

Example 4During validation, the matrix containing the analyte was spiked with analyte solution. The analyte in the matrix was not taken into account, and the low part of the range was not validated as intended.

Supplemental validation was performed.

www.fda.gov

16

Typical issues: Range (contd.) Example 5During validation, the dilution scheme of the standard solution used for validation was incorrect, and an incorrect range was validated.

Supplemental validation was performed.

Example 6After change of specification acceptance criteria, the validated range became insufficient to support specification. The new range was not re-validated before the submission of the license application.

The analytical procedure was re-validated during application review.

www.fda.gov

17

Typical issues: Specificity

• Specificity is validated by showing lack of response to unrelated or irrelevant substances.

Risk: Procedure may be found not specific enough to distinguish between the target analyte and related substances potentially present. Assay may be deemed unsuitable and data may be excluded from review.

www.fda.gov

18

Typical issues: Accuracy • Accuracy was not validated over the entire range of the

analytical procedure. Risk: Results at the edges of the range may not be accurate,

leading to false OOS results.

• Accuracy is validated using the number of repeats different from the routine procedure (for the methods using replicates).

Risk: Accuracy of the assay in general is overestimated.

www.fda.gov

19

Typical issues: Accuracy (Contd.) • Accuracy is calculated incorrectly. Risk: Validation exercise may fail acceptance criteria.

Example 7For the validation of accuracy, different volumes of the standard solution were spiked into the matrix with fixed concentration of analyte to obtain fixed final volume.When the expected amounts of analyte were calculated, the changes in the amount of analyte coming from the variable volumes of the matrix were not taken into account. As a result, accuracy was underestimated.Results were re-calculated and the validation report was revised.

www.fda.gov

20

Typical issues: Precision • For multi-step procedures variability only for final step is

assessed. Risk: Precision of the procedure may be overestimated. Example 8For the validation of a procedure involving complex sample preparation, including chromatographic separation of sampled fractions, followed by separate analysis of each fraction, only the data regarding the precision of final analysis were included.

Data were generated outside of the validation study, and only provided in response to an Information Request.

www.fda.gov

21

Typical issues: Precision (contd.) • Validation study design and AC do not allow for identification and control of

source of variability, and establishment of suitability for intended use. Risk: Unsuitable procedure may be validated.

Example 9 Intermediate Precision validation was designed to have two data sets: (Analyst 1, Day 1, Instrument 1) and (Analyst 2, Day 2, Instrument 2). The single AC for SD was set at 5.8, applicable both to each set and combined set. Specification limit was set at 15. SDs of 0.6 and 0.8 was obtained for each set, and an SD of 3 for the combined set. The difference of means between the sets was ~5. OOS/OOT results were observed in routine testing and attributed to the variability of the test. The procedure was found to be unsuitable for its purpose due to low Intermediate Precision.

Company performed investigation, and found the root cause for the high variability. The procedure was modified and re-validated.

www.fda.gov

22

Typical issues: Robustness • Parameters used to validate robustness are not relevant or set

of parameters is not complete.Risk: Robustness of the procedure is not established and adequate

control are not established to control for potential causes of variability.

Example 10During the validation of robustness, only the effect of the concentration of reagents was assessed. The method is known to be sensitive to temperature and time, etc.

The procedure was re-validated.

www.fda.gov

23

Typical issues: Robustness

Example 11During the validation of robustness for test method using a commercially available kit, only 2 lots of the kit were used, and it was concluded that it is not a factor affecting assay robustness.

The procedure was re-validated.

www.fda.gov

24

Typical issues: Comparability • Acceptance criteria and statistical analysis are inappropriate to

establish comparability.Risk: New procedure may produce significantly different results and may

not be able to support specification.

Example 12The company proposed to use a new method to test an existing specification parameter. The company also proposed to maintain the existing acceptance criteria. Comparability was established based on the mean difference of 5.8% from testing 6 samples by both methods, meeting the acceptance criterion of < 6%. During review, the paired t-test was applied to the data, which produced a p-value of less than 10-6.

The supplement was withdrawn.

www.fda.gov

25

Special Considerations for Reference Material Based Assays

• Manufacturers are recommended to develop in-house reference material(s) early in the development cycle to ensure reliability of test results.

• Assay should be validated using the same reference material as used in routine testing.

• In case of a change in the reference material, calibration of the old material against the new one must be performed.

• Commercially available critical reagents should be qualified using the in-house material.

• Use of reference materials provided with commercially available kits is discouraged because their quality cannot be assured.

www.fda.gov

26

Data Integrity IssuesExample 13• Bridging study was performed for two methods, and the data were

submitted. The company stated that “similar profile” was obtained.

• Upon review, it was found that the data for the new method were not fitted correctly in Excel.

www.fda.gov

27

Data Integrity Issues (contd.)Example 14• A new procedure was submitted for approval along with the validation

summary. The summary stated that validation was successful with one minor deviation. The summary included raw data for all parameters except Intermediate Precision. The original validation report was requested from the company. During review of report, it was found that:– Accuracy validation failed, supposedly because a wrong concentration

of the stock solution was used. The data were then recalculated using the “correct” concentration to obtain acceptable results.

– Intermediate Precision validation failed significantly. Acceptance criteria were “met” by using incorrect datasets in the calculation of statistical parameters.

The analytical procedure was deemed unsuitable for its intended purpose.

www.fda.gov

Thank you!