Download - 1 International Federation for Consulting GmbH IFC Validation by Dr. Michael Scheutwinkel
3
ReferencesReferences
EURACHEM GuideThe Fitness for Purpose of Analytical Methods -
A Laboratory Guide to Method Validationand Related Topics (12/1998)
www.eurachem.com
EURACHEM GuideThe Fitness for Purpose of Analytical Methods -
A Laboratory Guide to Method Validationand Related Topics (12/1998)
www.eurachem.com
CITAC / EURACHEM GuideGuide to Quality in Analytical Chemistry
An Aid to Accreditation (2002)www.eurachem.com
CITAC / EURACHEM GuideGuide to Quality in Analytical Chemistry
An Aid to Accreditation (2002)www.eurachem.com
6
Do not ask me, I am
only a simple guy
from Alemania!
Do not ask me, I am
only a simple guy
from Alemania!
7
Validation is the confirmation by examination
and the provision of objective evidence that
the particular requirements for a specific
intended use are fulfilled.
Validation (ISO/IEC 17025)
Definition in clause 5.4.5.1 of ISO/IEC 17025
….fit for purpose….fit for purpose
8
Validation of an analytical test method
Which analytes can be determined in which matrix in the presence of which interferences?
Within these conditions what levels of precision and accuracy can be achieved?
Which analytes can be determined in which matrix in the presence of which interferences?
Within these conditions what levels of precision and accuracy can be achieved?
Validation is the process of establishing the
• Performance characteristics
• Limitations of a method
• Identification of the influences which may change these characteristics and to what extent.
Validation is the process of establishing the
• Performance characteristics
• Limitations of a method
• Identification of the influences which may change these characteristics and to what extent.
9
Equipment and method valdiation
… different items… different items
Method validationMethod validation
Soft and hardwareSoft and hardware
Equipment validationEquipment validation
IQIQ PQPQ
OQOQDQDQ
10
Validation in order to analytical method
ValidationValidation
Modified standardized andin-house methods
Modified standardized andin-house methods
VerificationVerification
Standardized-methodsStandardized-methods
11
Verification of an analytical test method
The tools are the same as for validation,
the laboratory needs only to verify that
the documented performance characteristics
can be met (e.g. accuracy).
The tools are the same as for validation,
the laboratory needs only to verify that
the documented performance characteristics
can be met (e.g. accuracy).
Verification is the process to demonstrate the
competence in test performance
of an already validated standardized test method.
Verification is the process to demonstrate the
competence in test performance
of an already validated standardized test method.
12
When should be validated?
• Changes in established methods• New method developed for a particular problem• Established method used in different
laboratories, different equipment or different staff
• Out-of-control situations within internal quality assurance
• Non-successful participation in PTs• Demonstration of equivalence between two
methods (e.g. a rapid new test against a standard method)
• Changes in established methods• New method developed for a particular problem• Established method used in different
laboratories, different equipment or different staff
• Out-of-control situations within internal quality assurance
• Non-successful participation in PTs• Demonstration of equivalence between two
methods (e.g. a rapid new test against a standard method)
13
Validation Strategy
Characterizationof the test methodCharacterization
of the test method
Comparison of method
charcacteristicswith
requirements
Comparison of method
charcacteristicswith
requirements
Verification to give
proof that the requirements are
fulfilled
Verification to give
proof that the requirements are
fulfilled
BB CC DD
Specific test requirements defined by customer or marketSpecific test requirements defined by customer or market
AA
14
Description of test method (clause 5.4.4)
Identification / ScopeIdentification / Scope Parameters in which matrixParameters in which matrix
Reagents and materials, CRMsReagents and materials, CRMsEquipment and environmental
conditionsEquipment and environmental
conditions
CalibrationCalibrationSampling and
sample preparationSampling and
sample preparation
Test performanceincluding safety instructions
Test performanceincluding safety instructions
Quality assuranceQuality assurance
Documentation , reportingevaluation criteria
Documentation , reportingevaluation criteria
Uncertainty or procedure forits estimation
Uncertainty or procedure forits estimation
16
Tools
Determination of method characteristics
Characteristics of calibration
Characteristics of calibration
Characteristics of the range
Characteristics of the range
Characteristics of accuracy
Characteristics of accuracy
Selectivity orSpecifity
Selectivity orSpecifity
Robustness /Ruggedness
Robustness /Ruggedness
17
Tools
Determination of method characteristics
Characteristics of calibration Linearity Sensitivity
Characteristics of calibration Linearity Sensitivity
Perform morepoint calibration Perform more
point calibration Correlation coefficient:r² > 0.99 ??
Correlation coefficient:r² > 0.99 ??
Linearity is defined by the correlation coefficient
Sensitivity is defined by the slope of the calibration graph
19
Calibration
• Calibration values should be applied within the working range.
• Lowest point should be the limit of quantification.
• For calibration certified standard solutions shall be used.• In addition, also standard solutions have to be added to blank materials, analysed over all process steps.
• For linear calibration functions at least five concentrations are
necessary.
• Test calibration functions for linearity and highlight failing linearity.
Validation of testing methods
20
Sensitivity (resolution)
What is sensitivity?
It is the difference in an analytical concentration that corresponds to the smallest difference of a signal in a method which is still detectable.
Sensitivity can be extracted from the calibration curve or defined by using samples with different concentrations.
Validation of testing methods
21
Linearity
Definition based on samples with varying concentrations and the calculation of the regression of results.
Signal and concentration do not have to be fully related.
Five standards suffice to produce a calibration curve if linearity is o.k.
More standards are necessary once the linearity is unsatisfactory.
Examine repeat samples and standards over a particular working range to experience whether a reliable line can be drawn between proof and detection limit.
Validation of testing methods
22
Homogeneity of variances
Characteristic values for the comparison
of standard variations for differing
concentration.
In case of great differences between
standard deviations
(in-homogeneity of variances),
the working range needs to be split.
23
Tools
Determination of method characteristics
Characteristics of calibration
Characteristics of calibration
Characteristics of the range
Limit of detection Limit of quantification
Characteristics of the range
Limit of detection Limit of quantification
Characteristics of accuracy
Characteristics of accuracy
Selectivity orSpecifity
Selectivity orSpecifity
Robustness /Ruggedness
Robustness /Ruggedness
24
Working range
Finding out the appropriate working range using
• different matrices
• varying concentrations.
Concentration range within the achievement of
acceptable
• accuracy and
• precision is possible.
In general the working range is broader than the linear range.
Validation of testing methods
25
Prerequisites for calibration
Standards must be faultless.
Precision has to be similar throughout the entire working range.
The model function is applicable: either linear or curved.
Errors only may occur randomly within signals.
Errors have to follow the normal distribution.
Validation of testing methods
26
LODCharacteristics of the range
Limit of detectionLimit of detection
LODLOD xLOD = 3 . sL / b (Blank value method)
xLOD = 4 . sxo (Calibration function method)
xLd = Limit of detectionsL = Standard deviation of the blanks
Sxo = Standard deviation of the calibration function
b = Slope of calibration function
Various conventionsVarious
conventionsXLOD = Mean of blanks + 3 x SL
bycalculation
bycalculation
27
LOQ
Characteristics of the range
Limit of quantificationLimit of quantificationby
calculationby
calculation
xLOQ = 9 . sL/ b (Blank value method)
xLOQ = 11 . sxo (Calibration function method)
xLOQ = Limit of quantificationsL = Standard deviation of the blanks
Sxo = Standard deviation of the calibration function
b = Slope of calibration function
Various conventionsVarious
conventionsXLOQ = 5 or 6 or 10 x SL
LOQ LOQ
29
LOD
Characteristics of the range
Limit of detectionLimit of detection bysignal to
noise ratio
bysignal to
noise ratio
From 5 : 1 to 10 : 1From 5 : 1 to 10 : 1
Limit of quantificationLimit of quantification
From 2 : 1 to 5 : 1From 2 : 1 to 5 : 1
31
Tools
Determination of method characteristics
Characteristics of calibration
Characteristics of calibration
Characteristics of the range
Characteristics of the range
Characteristics of accuracy
Characteristics of accuracy
Selectivity orSpecifity
Selectivity orSpecifity
Robustness or Ruggedness
Robustness or Ruggedness
32
Selectivity / Specifity
SelectivitySpecifity
SelectivitySpecifity
Use ofantibodiesUse of
antibodies
Extent to which particular analytes can be determined in complex mixtures.
A method which is selective for an analyte is said to be specific.
Extent to which particular analytes can be determined in complex mixtures.
A method which is selective for an analyte is said to be specific.
Selective detectors in instrumental
analysis
Selective detectors in instrumental
analysis
Selective media in microbiology
Selective media in microbiologyCross-reactions ?Cross-reactions ?
33
Selectivity
• A selective method gives correct results for all interesting analytes whereas a specific test method gives correct results for the interesting analyte whereas other analytes might interfere each other.
The difference between selectivity and specificity
Specifity
• Outlines the analytical extent to which an analytical substance or substance group can be determined without interference from sample related components.
No problem, usually both words are mixed …No problem, usually both words are mixed …
34
Selectivity
• Outlines the extent to which an analytical substance can be determined without interference from other components.
• Method selective to one analytical substance specific.
• Assess selectivity while application to pure solution up to complex matrices.
• Document disturbances and restrictions of the method.
Validation of testing methods
35
Tools
Determination of method characteristics
Characteristics of calibration
Characteristics of calibration
Characteristics of the range
Characteristics of the range
Characteristics of accuracy
Characteristics of accuracy
Selectivity orSpecifity
Selectivity orSpecifity
Robustness /Ruggedness
Robustness /Ruggedness
36
Robustness
• Part of the routine.
• Learn from other analytical institutes, as the method-establishing laboratory will have proved robustness before publishing a new method.
Degree of proneness of a method to conscious and unconscious alterations of a working instruction.
Validation of testing methods
37
Robustness
Sensitivity degree of the test method against small deviations in experimental conditions.
Examples:
• Times within process steps
• Environmental conditions (e.g. temperatures)
• Minor process changes (e.g. pH, flow rates in
HPLC)
Sensitivity degree of the test method against small deviations in experimental conditions.
Examples:
• Times within process steps
• Environmental conditions (e.g. temperatures)
• Minor process changes (e.g. pH, flow rates in
HPLC)
38
Ruggedness, according to USP
Ruggedness is the degree of reproducibility obtained under a variety of conditions, expressed as relative SD, e.g.:
• different analysts,• different equipment,• different trade marks of reagents,• etc.
Ruggedness is the degree of reproducibility obtained under a variety of conditions, expressed as relative SD, e.g.:
• different analysts,• different equipment,• different trade marks of reagents,• etc.
Internal reproducibility
39
Tools
Determination of method characteristics
Characteristics of calibration
Characteristics of calibration
Characteristics of the range
Characteristics of the range
Characteristics
of accuracy: Correctness Uncertainty
Precision (Repeatability/
Recovery) Reproducibility
Characteristics of accuracy: Correctness Uncertainty
Precision (Repeatability/
Recovery) Reproducibility
Selectivity orSpecifity
Selectivity orSpecifity
Robustness or Ruggedness
Robustness or Ruggedness
40
Accuracy = Precision and correctness I
AccuracyAccuracyExactness of an
analytical methodExactness of an
analytical method
PrecisionPrecision
Degree ofrepeatability of ananalytical method
Degree ofrepeatability of ananalytical method
UncertaintyUncertaintyReproducibilityReproducibility
Recovery /Bias
Recovery /Bias
RepeatabilityRepeatability
Correctness /Trueness
Correctness /Trueness
Systematic error /Correct value
Systematic error /Correct value
41
precise
and
correct
not precise
but correct
precise but wrong
not precise and wrong
Random and systematic deviations
42
Characteristics of accuracy
Correct-ness
Correct-ness
PrecisionPrecision
Measure of correctness, covering systematic and non-systematic mistakes (use of CRM, comparison
to a well characterized method)
Measure of correctness, covering systematic and non-systematic mistakes (use of CRM, comparison
to a well characterized method)
Measure of the degree of repeatability,covering systematic mistakes
Measure of the degree of repeatability,covering systematic mistakes
Repeatability: one laboratory;Better: internal reproducibilityRepeatability: one laboratory;Better: internal reproducibility
Reproducibility: more than one laboratoryReproducibility: more than one laboratory
Accuracy = Precision and correctness II
43
Repeatability
Measurement of reference materials.
Fortification of a blank sample with the analytical substance.
Comparison to a reference procedure.
Participation in a co-operative (interlaboratory) test in which the “true content” is known.
Use at least one of the following procedures for determination:
Validation of testing methods
44
Precision
Information on accuracy of two coinciding and independent analytical results.
Precision depends on the concentration of the analyte.
Repeatability is part of the precision with respect to repeated measurements:
– same material,
– same method,
– same analyst,
– same laboratory and
– short time-span in-between the analyses.
Validation of testing methods
45
Correctness / Trueness
Use the one of the following methods to assure correctness:
• Analysis of certified reference material.
• Participation in interlaboratory comparisons.
• Comparison to a known procedure.
If not available:
• Document any data that prove correctness.
• Make an approximation as a first approach to correctness.• Purchase reference material.• Use a similar method in parallel.
Validation of testing methods
46
Recovery
• Performed by addition of the analytical substance to the matrix.
• A matrix shall be free of the analytical substance.
• If not the sample will be enriched with a weak concentration of the analytical sample.
• Or a simulated matrix will be used.
• Recovery can be determined only when the analytical sample is available in pure form.
• List the recovery rate (in %) and the standard deviation when recovery rates are constant.
• Otherwise issue recovery rate as a function.
Validation of testing methods
47
Frame conditions for the determination of the recovery rate
Frame conditions for the determination of the recovery rate
The analytical substance needs to be added in that form in which it occurs in nature.
The samples requires good homogenisation.
The native content shall be below the determination limit.
Validation of testing methods
48
Reproducibility
Statistically
Comparative standard deviation Replicate standard deviation
Through participation in proficiency testing schemes
or participation in interlaboratory comparisons.
Validation of testing methods
internallyinternally
externallyexternally
Prove reproducibilityProve reproducibility
49
Tools
Determination of method characteristics
Characteristics of calibration Linearity Sensitivity
Characteristics of calibration Linearity Sensitivity
Characteristics of the range
Limit of detection Limit of quantification
Characteristics of the range
Limit of detection Limit of quantification
Characteristics of accuracy
Correctness Uncertainty Precision
(Repeatability/Recovery)
Reproducibility
Characteristics of accuracy
Correctness Uncertainty Precision
(Repeatability/Recovery)
Reproducibility
Selectivity orSpecifity
Selectivity orSpecifity
Robustness orRuggedness
Robustness orRuggedness
50
Classification of the test method
• Analysis of contaminants (e.g. pesticides, heavy metals, food additives)• Analysis of food constituents (salt and minerals)
• Methods which are determining true values
• Test methods which have to be calibrated with instruments
• Physical measurement (pH value, conductivity)
• Qualitative determinations
• Microbiological methods
Distinguish between
Validation of testing methods
51
Validation of testing methods
Parameter Analysis of
trace
contaminants
Analysis of
constituents
Conventional
methods
Methods not
necessary to
calibrate
Physical
methods
Qualitative
methods
Bacteriological
methods
Description of
test method
X X X X X X X
Calibration X X X X
Working range X X X X X X X
Detection limit X X X X
Determination
limit
X X
Recovery X X X
Repeatability X X X X X X
Correctness X X X X
Specificity X X X X X
Reproducibility if possible if possible if possible if possible if possible if possible if possible
Parameters for validation
52
Common validation needs of known methods
Method Accuracy Precision Specificity/Selectivity
Linearity Detection/ Determinationlimits
Robustness
AAS, AES, ICP, RF X X (X) X X X
Polarography/ Voltametry X X X X X
Titration X X X
Water assay X X X X
Chromatography/ CapillaryElectrophoresis
X X X X X
Spectroscopy X X X X X
Release X X X X X
TOC X X X X
Enzyme Immuno Assays X X X X X X
Validation of testing methods
53
The Analysis of Standard Reference Materials
• Generally accepted method for validation.
• Such standards are provided with guarantee on the market.
• It may be necessary to contract the preparation of a unique sample in particular matrix in order to utilise this procedure for method validation.
• The analyst must demonstrate that the method provides accuracy and precision.
Procedures for method validation
54
The participation in PT-schemes or in a laboratory Collaborative Study
The most widely accepted procedure for method validation.
Serious practical draw backs: Costly and time consuming Effort in co-ordination Shipping of samples and data Statistical analysis and interpretation of results.
This method is rarely used for the first description of a method in the literature.
Procedures for method validation
55
Comparison with a currently accepted method
• Usually done by one analyst
• or two using a split-sample.
• using results from the currently accepted method for verification.
Agreement suggests validation.
Disagreement could also suggest that the currently accepted method is invalid.
• In such case, another procedure has to be employed for the method validation.
• The more samples are analysed and the wider the range of concentration the higher the credibility of the validation method.
Procedures for method validation
56
The Zero-Blind Method
• One analyst.
• Using samples at known levels of analyte to demonstrate recovery, accuracy and precision.
• Fast, simple and useful but
it leads to subjective results.
• Suitable for a first approximation requiring minimal time, manpower, samples and cost.
• In general, a good start for the overall validation process.
If this methods fails there is no reason to proceed with further validation of the method.
Procedures for method validation
57
The Single-Blind Method
• One analyst at the start.
• Samples are given to a 2nd analyst to whom levels of analyte are unknown.
• Results are compiled and compared by the 1st analyst.
• Comparison by the first analyst makes objectivity questionable.
• This method still is biased on behalf of the 1st analyst.
• Suitable after the zero-blind method has been successful and before additional analysts or the management will be involved.
Procedures for method validation
58
The Double-Blind Method
• Three analysts.
• 1st analyst prepares samples at known levels.
• 2nd analyst does the actual analysis.
• 3rd analyst (or administrator) compares both data from the first two analysts.
• Only the 3rd analyst has access to these data.
Most effective approach to objectiveness
Procedures for method validation
59
Clearance of validated test procedures
Validation results
Customer requirements
Quality Manager
Validation of testing methods
60
Validation of testing methods
Decision criteria for the extent of a validation
Technical feasibility and staff disposition?
Analytical viewpoints Measuring principle, method complexity?
Occurrence of the sample, wide-spread?
Risk potential for client/ company?
Political decisions Actual aim of the validation?
Consideration of normative
and other requirements?
facts
super-ordinated criteria
61
Extent of validation in analytical sciences
Purpose
• Suitability of the method
• „Marketability“ of the method
Analytical sensibility
Costs
Requirements from the outside
Validation of testing methods
62
Validation is always a balance between costs, risks and technical possibilities.
Validation is always a balance between costs, risks and technical possibilities.
The validation shall be completed by a statement by the laboratory that the
method is fit for the intended use.
ISO/IEC 17025 states:ISO/IEC 17025 states:
63
Validation is always a balance between
costs, risks and technical possibilities.
Validation is always a balance between
costs, risks and technical possibilities.
64
Self developed software or excel sheets have to be validated.
Self developed software or excel sheets have to be validated.
Records for that must be available.
Do not forget:Do not forget:
71
Fuente de incertidumbre SímboloValor
± [Unidades]
Distribución de probabilidad
DivisorIncertidum-bre estándar
[unidades]
Calibración de la balance uEst + 0.003 g Tipo B / rect 0.7% 0.4%
Calibración de la estufa uPip 0 % Tipo B / rect
Tiempo uBurInd 0 %
Repetibilidad uRep + 0.1 % Tipo B 0.1%
Trabajo en grupos ”Humedada en Alimentos”
33
ucombinada = 3.43 % Uexpandida = 7 %
33