afnor-validation2008

AFNOR VALIDATION SYMPOSIUM, BRUSSELS 2008RVA J0958

AFNOR VALIDATION

EUROPEAN CERTIFICATION

FOR MICROBIOLOGICAL TEST KITS19 JUNE 2008 - BRUSSELS

***

Many thanks to :

AFSSA-LERQAP, SANCO E2, AFNOR Certification, Insitut Pasteur, ADRIA Dveloppement, Deutsches Institut fr Lebensmitteltechnik, 3M Microbiology, bioMrieux Industry, Nestl Research Center, University of Lige, Genesystems, Bio-Rad.

AFNOR all rights reserved. RVA J0958-2008

9:30 Registration

10:00 Welcome and introduction

Speaker : Bertrand Lombard, AFSSA-LERQAP

10:10 European regulation - Specific requirements for the use of alternative methods for microbiological criteriaSpeaker : Ari Hrman, DVM PhD, MPH Legislative officer, END, SAMCO E2

10:30 AFNOR VALIDATION : European status and Certification Scheme

Speaker : Valentine Digonnet, AFNOR VALIDATION Scheme Manager

10:45 Methods performance assessment according to the EN ISO 16140 standard and the AFNOR Validation technical rules : Qualitative methods & Discussion

Speaker : Virginie EWE, Institut Pasteur Lille, Food microbiology laboratotory Technical Manager

11:10-11:30 : Coffee Break

11:30 Methods performance assessment according to the EN ISO 16140 standard and the AFNOR Validation technical rules : Quantitative methods & Discussion

Speaker : Danile Sohier, ADRIA Dveloppement, IDEA Unit Manager

12:00 More information on Inter-laboratory studies

Speaker : Dr. Steinkamp, Deutsches Institut fr Lebensmitteltechnik

12h15-13/30h15 : Lunch Break offered

13:30 Two kit manufacturers validation experiences& Discussion

* 3M Microbiology : Marie-Pierre COPIN; New products and Regulatory Manager

* bioMrieux Industry : Raffaella Giardino, Global Marketing Manager, Program Director QI

14:20 Experience from a perspective of a Food Manufacturer

Speaker : John Marugg, Nestl Research Center, Microbiological Safety

14:45-15:15 :Coffee Break

15:15 Alternative validated methods in Belgium, an example of application of the European Regulation EC n2073/2005

Speaker : Caroline De Backer, University of Liege, National Reference Laboratory for food microbiology

15:40 Revision of EN ISO 16140 : current status and foreseen changes

Speaker : Paul in 't Veld, Chairman of ISO/TC 34/SC 9/WG3

16:00 PCR Legionella : the interest of an AFNOR Validation with 2 examples & Discussion

Speaker : Bertrand Coissac, Genesystems-Sales Manager and Frederic Martinez, Bio-Rad - Food Science Division / Group Product Manager

16:30 Debate & Conclusion

17:00 Closure of the Symposium




Welcome to Afnor Validation Symposium !***

Convenor

Bertrand LOMBARD

Co-ordinator Scientific & Technical Advice, Community Reference Laboratories, International Relations ;

AFSSA-LERQAP (French Agency for Food Safety - Laboratory for Study &Research on Food Quality & Food Processes)23 avenue du Gnral de Gaulle - 94706 Maisons-Alfort, FranceTl: 00 33 1 49 77 26 96 - Fax: 00 33 1 49 77 26 95 - e-mail: [email protected]: www.afssa.fr


Regulation (EC) No 2073/2005 on microbiological criteria for foodstuffsThe use of rapid methods

Ari HrmanEuropean Commission

DG SANCO

AFNOR Validation Symposium, June 2008, Brussels

AFNOR Validation Symposium, 19th of June 2008 Brussels

The aim of the Regulation To ensure a high level of human health protection

Reduction of human salmonellosis and listeriosis cases

To harmonise microbiological criteria in the EU

Uniform rules for food business operators and competent authorities within the EU as well as for third countries importing to the EU


Testing against criteriaFood business operators (Reg 2073/2005)

For validation and verification of procedures based on HACCP and GHPFor batch acceptability testing

Competent authorities (Reg 882/2004 on official controls)

To verify that food is in comliance with the Community criteria


RegulationLatest scientific adviceBroad consensus Flexible

Sampling proceduresAnalytical methodsSampling frequencies (risk-based)Process hygiene criteria Trends


Components of a microbiological criterion (Codex)

!Micro-organism of concern!Analytical method!Sampling plan!Microbiological limits!The foodstuff!The point of the food chain where the limit

applies!Actions to be taken when unsatisfactory

resultsAFNOR Validation Symposium, June 2008, Brussels

Analytical methodsReference methods in the legislation

Preference to horizontal methodsMethods of international standardisation organisations

For in-house control Alternative methods can be used under certain conditions (Regulation 2073/2005)

For official controlsThe use of EN/ISO methods recommended (Guidance document on official controls)


The use of rapid methods

An advantage for food business operators

Results available more rapidly than by using traditional methodsRapid corrective measures and actions possible


Alternative methodsThe use of alternative analytical methods is acceptable when the methods are:

validated against the reference method in Annex I and if a proprietary method, certified by a third party in accordance with the protocol set out in EN/ISO standard 16140 or other internationally accepted protocols, is used.


Alternative methodsIf the food business operator wishes to use analytical methods other than those validated and certified as described in paragraph 3 the methods shall be validated according to internationally accepted protocols and their use authorised by the competent authority


Thank you

[email protected]


AFNOR Validation :Certifying analytical performances

of commercial methods

Valentine Digonnet


AFNOR Validation Symposium, June 2008, Brussels2

AFNOR CERTIFICATIONAFNOR CERTIFICATION

An AFNOR owned commercial subsidiary offering a complete range of certification services with COFRAC accreditation (French signatory of EA and IAF Mutual Agreements)

Staff : 250 employees

Turnover : 60 million euros

AFNOR CERTIFICATION SERVICES

Conformity assessmentAttestation of

conformity to ECregulations

Product CertificationCompetencecertification

Services certification

Systems certification (ISO 9001, ISO

14001,)



Purpose of AFNOR Validation

!Conformity assessment by a third party bodythat test results obtained with commercial kits

are comparable to results obtained using the corresponding reference methods

" individual approach (proprietary methods)


Who needs AFNOR Validation?

Kit manufacturers

# to highlight the performance of their methods

Food industry, private and official labs

# for internal quality control or for services

Users of analytical results

# because they need safeguards


What is an alternative method?

For a givengiven scopescope, it shall estimate the same same analyteanalyte than the one measured by the corresponding reference method,

and shall meet the following requirements:

rapidrapid analysis and/or response

easyeasy carrying and/or automation

analytical performancesperformances (accuracy, sensitivity..)


What is included in the alternative method ?

" The term "alternativealternative methodmethod" refers to the combination of product, equipment andoperating instructions

" It includes all elements necessary forimplementing the alternative method

(equipment, reagents, culture media, software for analysisof results,)


What is a reference method?

" A reference methodreference method is automatically a standardised method (CEN, ISO).

If no CEN or ISO method : an official method (if existing), or in certain (rare) cases a well known and widely practised method will serve as a reference.


List of main reference methods used for AFNOR Validation

EN ISO 6579 : Salmonella

EN ISO 11290-1 : Listeria (detection)

EN ISO 11290-2 : Listeria (enumeration)

EN ISO 6888-1 : S. aureus

ISO 21528-1 : Enterobacteriaceae

ISO 4831 & 4832 : Coliforms

ISO 16649-2 : E. coli

ISO 4833 : Total count


Who acts in AFNOR Validation?

AFNOR CERTIFICATIONadministrative management

responsible for implementation

Expert laboratoriesindependant & qualified

achieve the validation studiesmake presentations of the results

Validation Commissionadvises on :

general operating rules, policies collective advertising

Auditorsindependant & qualified perform the quality audits

Technical Boardadvises on :

technical validation and renewalspecific technical rules.

20 members5 meetings (x 2 days) / year


AFNOR ValidationCertification procedure

Based upon 33 principlesprinciples:

1- A standardized validation protocol (EN ISO 16140)

2- Quality control of the production (audits)

3- Implementation of certification rules (including regular control of the validated methods)


AFNOR Validationtechnical protocol

SinceSince July 2003July 2003 :

standard EN ISO 16140

specific requirements and interpretation set up by the Technical Board


The preliminary study

based on EN ISO 16140 requirements

to characterizecharacterize the alternative method(selectivity, practicability)

to comparecompare the performances of the alternativemethod against the performances of the reference method (relative accuracy, sensitivity, specificity, relative detection level,)

Mandatory third party expert lab (no internal data)


The inter-laboratory study

To define variabilityvariability ofof results obtainedresults obtained inin different different laboratories using the same sampleslaboratories using the same samples

According to EN ISO 16140:

at least 10 collaborative laboratories giving nonoutlier results (or 8 labs for quantitative methods)

implementing both alternative and reference methods


Quality requirementsfor the manufacturer

Adoption of audit requirements based uponEN ISO 13485 (medical devices)

Additional specific requirements concerning thecontrol ofcontrol of productsproducts


AFNOR Validation :Control of alternative methods

DurationDuration ofof certificatecertificate : 4 years

QualityQuality control on sitecontrol on site : according to the certificationalready granted to the supplier

one audit every 2 years (if not certified)

one audit every 4 years (if certified ISO 9001 or 13485)

RenewalRenewal every 4 years : complete review of changes in the alternative method, reference method andValidation protocol

#complementary assays if needed


AFNOR Validation CertificateContentsContents :

AdministrativeAdministrative detailsdetails:

" Date of validation / end ofvalidity

" Certificate reference

" Address of manufacturerand retailer

" Reference of the kit users manual / technical sheet

TechnicalTechnical detailsdetails:

" Principle of the method

" Scope of validation

" Restrictions for use

" Reference method used

" Analytical performances (Resume of main results obtained)


AFNOR ValidationReference documents

OperatingOperating RulesRules (general part + appendices)# to define the general policies

Technical protocolTechnical protocol for the validation studies(based upon EN ISO 16140 + interpretation)# to be used by the expert laboratories


EU Regulation on microbiological Criteria,DG SANCO

Article 5 (published in January 2006) :

use of alternative proprietary methods when certified by a third party in accordance with the protocol set in EN ISO 16140 standard (or similar)and validated against the reference method set inAnnex I (mainly EN ISO methods)


AFNOR Validation complies with EU Regulation

AFNOR VALIDATION scheme was developed inorder to comply with EU requirements defined inregulation, granting certificates to alternativemethods which are:

Validated against EN/ISO reference methodsCertified by a third party (AFNOR CERTIFICATION)Tested against EN ISO 16140 standardized protocol


AFNOR Validation :from 1989 to 2008

! 1989 : first kit certified AFNOR Validation

! 2008 : 73 validated methods from 15 companies# 70% pathogen detection

#Methods based on several different principles such as :

Chromogenic media, Immunoassay, Real time PCR,


International and European supplierswith AFNOR validated methods

3M Healthcare (USA)

AES Chemunex (F)

BIO-ART (BE)

BIOCONTROL Systems (USA)

BIOLINE (DK)

BioMrieux (F)

BIO-RAD (USA, F)

ChromAGAR (F)

CHR Hansen (F)

DUPONT QUALICON (USA)

EUROPROBE (Austria, F)

GENEPROBE (USA)

GENESYSTEMS (F)

OXOID Thermofisher (UK)

SOLABIA BIOKAR (F)

SY-LAB (Austria)

TECRA (Australia)


Implementation of EN ISO 16140in AFNOR VALIDATION

5 years work on implementation of EN ISO 16140

$Working groups and Technical Board meetings

" 63 method validations or renewals according toEN ISO 16140 protocol

" Contribution to ISO/CEN for standard revision since 2005

based on real experience according to validation studies on qualitative and quantitative methods based on several principles


Recent development of AFNOR Validation

% Creation of the logo and registration of thecollective European mark (2005)

% Current contribution to revision of technical protocol EN ISO 16140

% Extension of AFNOR Validation scope to watermicrobiological analysis in 2007


Development of AFNOR ValidationObjectives in the near future

To make a request for accreditation scope extension of AFNOR Certification for AFNOR Validation scheme

To work on mutual agreement with other Validation schemes:% 1- Accepting test results when possible% 2- Working on current obstacles:

- use of same reference methods (ISO CEN standards),- use of same validation protocol (standard EN ISO 16140)- use of a significant part of naturally contaminated samples from

various matrices in the comparative study- 3rd party certification with a regular survey of the product quality

and the production site

To extend AFNOR Validation scope (GMO,)

19/06/2008 Virginie Ewe - Institut Pasteur Lille 1

Methods performance assessment according tothe EN ISO 16140 standard and the AFNOR Validation technical rules

- Qualitative methods



Processus of validation

Performance assesment =

Study in 2 steps : Preliminary study Interlaboratory study

Comparison with an EN ISO standard method: in relation with the european regulation



Preliminary study Aim: to verify if results obtained with the

alternative method are equivalent to those obtained with the reference method Determination of Relative ACcuracy, Relative

SPEcificity and Relative SEnsitivity (%) Determination of Relative Detection Level (LOD50) Inclusivity and exclusivity (pure culture strains)

Practicability (AFNOR certification technical rules)



Relative AC, SPE & SE

Analyses on different samples with: Alternative method Reference method

Comparison of results obtained for the same samplesconfirmation of positive tests



Relative AC, SPE & SE

Number of samples: 5 categories: meat products, dairy products,

choice of products depending on target microorganism

60 results per category (around 50% positive results and 50% negative results)

At least 30 positive results in AFNOR Certification rules



SamplesNatural or Artificial contamination?Natural or Artificial contamination?

Natural contamination if possible, to consider: Effect of background flora Physiologic status of the strains Minimal % of naturally contaminated samples in

AFNOR Certification rules :50% for Listeria monocytogenes25% for Salmonella



Examples of natural contamination rates

Listeria studies:! 55% in lot of studies to 80% in some studies! Smoked salmons, raw meats, delicatessen,

environmental samples! Dairy products

Salmonella studies:! More difficult to find naturally contaminated

samples : meat and egg products! 26 25 %



SamplesNatural or Artificial contamination?Natural or Artificial contamination?

Rules for artificial contamination: ISO 16140: use stressed microorganisms and

determine the stress and level of contamination Respect of origin / samples Different types of stress log (non selective selective medium) > 0.5 Level of contamination: < 30 cells/25g



Analysis protocol

homogenization

Same first step

incubation

Reference method Alternative method



Analysis protocol

homogenization

Same first step

incubation


BPW or Half Fraser



Analysis protocol

Product Homogenisation


Different first step



Calculation and interpretation

PDPA(A+)

NAND(A-)

(R-)(R+)Results

PA = positive accordance (R+ / A+)NA = negative accordance (R- / A-)PD = positive deviation (R- / A+) = additional positiveND = negative deviation (R+ / A-) = false negative



(R+) (R-)

(A+) A+ / R+PA = 155A+ / R-PD = 2

(A-) A- / R+ND = 1A- / R-

NA = 169

Same first step between alternative and reference protocols

Examples



(R+) (R-)

(A+) A+ / R+PA = 155A+ / R-PD = 16

(A-) A- / R+ND = 7A- / R-

NA = 186

Different first stepbetween alternative and reference protocols

Examples



Relative Accuracy (AC)

Degree of correspondence between the response obtained by the reference method and the response obtained by the alternative method on identical samplesAC = (NA + PA) / N



Relative Specificity (SP) Ability of the method not to detect the target

organism if the reference method doesnt detect it

SP = NA / (NA + PD)

i.e. additionnal positive (PD) are considered as false positive, although they were confirmed as true positive

# the target organism is not present



Relative Sensitivity (SE)

Ability of the method to detect the target organism when the reference method detects it

SE = PA / (PA + ND)



Sensitivity

SE = POS x 100% in AFNOR Certification rulesN+

with N+ = PA + ND + PD

et POS = confirmed positive results of the considered method



99.4 %

98.8 %

99.1%

95.7%Relative sensitivity : SE%

92.1%Relative specificity : SP%

93.7%Relative accuracy : AC%

Examples

3 discordances:1 ND and 2 PD

23 discordances:7 ND and 16 PD



99.4 % 95.7 %Relative sensitivity : SE%

98.7 %99.4 %

(PA + ND) / (PA + PD + ND)(PA + PD) / (PA + PD + ND)

91.0 %96.1 %

Reference method (SE%)Alternative method (SE%)

Examples

SE = PA / (PA + ND)



Calculation and interpretation

No criteria of acceptance in the standard

Analysis of discordances

Statistical test to determine if the methods are equivalent or not



Ccld min

~16 7 = 91023

d = |PD ND|discordances

Examples

Less than 6 discordances: no statistical test



Relative detection level

= the smallest number of culturableorganisms that can be detected in the sample in 50% of occasions



Measurement protocol

5 products (one per category) 5 target microorganisms 3 to 5 levels of contamination 6 replicates per level



Interpretation

The relative detection level lies between the two contamination levels: less than 50 % positive results more than 50 % positive results



Interpretation

"Hitchins A. Proposed Use of a 50 % Limit of Detection Value in Defining Uncertainty Limits in the Validation of Presence-Absence Microbial Detection Methods, Draft 10th December, 2003".

LOD 50



Example

Matrix Strain Reference method

(CFU / 25 g)

Alternative method

(CFU / 25 g)

Raw milk L.mono 1/2b 0,4 [0,3 0,6] 0,4 [0,3 0,6]

Pt L.mono 1/2c 0,4 [0,3 0,7] 0,4 [0,3 0,7]

Red cabbage L.mono 4b 0,6 [0,4 1,1] 0,6 [0,4 1,1]

Smoked salmon L.mono 1/2a 0,4 [0,3 0,6] 0,4 [0,3 0,6]

Process water L.mono 1/2c 0,8 [0,4 1,8] 0,8 [0,4 1,8]



Inclusivity

detection of the target microorganism from a wide range of strains

50 strains

Application of the complete protocol of the alternative method



Exclusivity

= the lack of interference from a relevant range of non-target microorganisms

30 strains

Use of non selective broths



Practicability

13 criteria defined in AFNOR Certification rules like : Qualification of technical staff Environment of analyses Materials Real time handling Time to result



Interlaboratory study

Aim:to determine the variability of the results obtained in different laboratories using identical samples



Interlaboratory study organisation

At least valid data from 10 laboratories 3 levels of contamination 8 replicates per level



Analyses

Analyses with both methods, alternative and reference

48 results: 24 with each method



Interpretation

Comparison of alternate method and reference method results against expected results

FP : false positive of L0 level TP : true positive of L1 et L2 levels



Interpretation

TPTPFPTotal

/8/8/8etc

/8/8/8Lab 2

/8/8/8Lab 1

L2L1L0Contamination level

Lab

Pour each method : number of positive results



Interpretation

L0 LevelSP = (1 FP ) X 100%

N-

FP = number of false positive

N- = total number of samples for L0 level



Interpretation

L1 et L2 Levels

SE = TP X 100%N+

TP = number of true positive

N- = total number of samples for each level



Interpretation

Accuracy : AC = PA + NA x 100%N



Accordance, concordance, odds ratio

Notions of repeatability, reproducibility and odds ratio (informative annex)



Conclusion

Preliminary study: informations of method performances

Interlaboratory study: practicabilitylot of work for the participant laboratories



Certificate

Edition of a certificate

Summary report

www.afnor-validation.com


08.19.06 [email protected]

Methods performance assessment according to

the EN ISO 16140 standard and the AFNOR

Validation technical rules

QUANTITATIVE METHODS


08.19.06

ADRIA Developpement

! Agro-Food Application and Technologies Centre focuses on expert courses and training programs

management coordinates innovation programs in close collaboration

with food and diagnosis industrial partners

" predictive tools conception for food process and food products shelf-life control and monitoring# www.symprevius.org

" nano-scales devices development for food microbial ecosystems analysis based on OMIC knowledge

" expert lab which provides daily validation ofmicrobiological alternative methods according tothe ISO 16140 standard


08.19.06

Alternative method ?

! The European regulation (Regulation (EC) No 852) specified

Alternative methods are validated against the reference method; the results are certified by a third party inaccordance with the protocol set out in EN/ISO standard 16140


08.19.06

EN ISO 16140 standard

! Microbiology of food and animal feeding stuffs Protocol for the validation of alternative method Qualitative methods Quantitative methods


08.19.06



$ Microbial Indicators of food quality & safety


08.19.06



And according to the AFNOR validation technical rules



08.19.06

ISO 16140 standard


Or the need of a revised standard



08.19.06

"Methods comparison study

SpecificitySpecificity && selectivityselectivity

Linearity Linearity

Relative Relative accuracyaccuracy Methods precision

Detection & quantification limits

" Inter-laboratory study

BiasBias

RepeatabilityRepeatability & & Reproducibility limitsReproducibility limits Dispersion between laboratories

Validation study process

!


08.19.06

! Food categories : 5 categories to test (minimum)

Dairy products

Meat products

Fish & seafood products

Fuits and vegetables based products

Egg based products

Animal feeds

Environmental samples

Methods comparison study


08.19.06

Specificity /Selectivity

! Pure cultures

30 target strains

20 non-target strains

Enumeration by the reference and the alternative methods

Interpretation : the alternative method is conformedto the manufacturers specifications and to the specific

requirements for its reliable use.


08.19.06

Linearity

! 5 contamination levels, 4 log range

! 1 matrix / category, x 5 categories

! Lack-of-fit test

0

1

2

3

4

5

6

0 1 2 3 4 5 6

Log (Reference method)

L

o

g

(

A

l

t

e

r

n

a

t

i

v

e

m

e

t

h

o

d

)

Matrix 1 Sample Regression P% Correlation coefficient

Regression equation

Matrix 1 GMFR 100 0,999 log alt =0,975 logRf. + 0,101

Ratio : global repeatability standard deviationsGMFR : Orthogonal regressionOLS : Ordinary least squares linear regressionF-table (Snedecor)


08.19.06

Linearity



! Lack-of-fit test

Sample Regression P% Correlation coefficient

Regression equation


0

1

2

3

4

5

6

0 1 2 3 4 5 6


L

o

g

(

A

l

t

e

r

n

a

t

i

v

e

m

e

t

h

o

d

)

Matrix 1

Correlation coefficient = 0,999, P > 5% The linearity is accepted


08.19.06

Sample Regression P% Correlation coefficient

Regression equation


Matrix 2 GMFR 2 0,999 log alt =0,975 log Rf. + 0,093

0

1

2

3

4

5

6

0 1 2 3 4 5 6


L

o

g

(

A

l

t

e

r

n

a

t

i

v

e

m

e

t

h

o

d

)

Linearity



! Lack-of-fit test

Correlation coefficient = 0,999, but P < 5%Linearity test robustness?

Matrix 2


08.19.06

Relative accuracy

Method 1

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6 7


L

o

g

(

A

l

t

e

r

n

a

t

i

v

e

m

e

t

h

o

d

)

P% All products n Regression intercept slope Intercept

= 0 Slope = 1

Method 1 73 GMFR 0,035 0,991 37 27

! Minimum 10 samples/category, x 5 categories

! Priority to naturally contaminated samples

! Slope = 1, Intercept = 0 (linear model)

Discrete clusterIntercept = 0 with P > 5%Slope = 1 with P > 5% No systematic bias, ideal accuracy


08.19.06

Relative accuracy

P% All products n Regression intercept slope Intercept

= 0 Slope = 1

Method 1 73 GMFR 0,035 0,991 37 27 Method 2 73 GMFR -0,135 1,013 1 39

Method 2

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6 7


L

o

g

(

A

l

t

e

r

n

a

t

i

v

e

m

e

t

h

o

d

)

! Minimum 10 samples, x 5 categories


! Slope = 1, Intercept = 0

Discrete cluster, no outlierSlope = 1 with P > 5% Intercept 0 with P < 5%


08.19.06

Relative accuracy

! Minimum 10 samples, x 5 categories


! Slope = 1, Intercept = 0P% All

products n Regression intercept slope Intercept= 0

Slope = 1

Method 1 73 GMFR + 0,035 0,991 37 27 Method 2 69 GMFR - 0,135 1,013 1 39 Method 3 61 GMFR - 0,895 1,044 10 48

Method 3

0

1

2

3

4

5

6

7

0 1 2 3 4 5 6 7

Log (reference method)

L

o

g

(

A

l

t

e

r

n

a

t

i

v

e

m

e

t

h

o

d

)

Intercept = 0 with P > 5% ?Slope = 1 with P > 5%Systematic bias ? Tests robustness ?


08.19.06

Methods comparison study

! Current EN ISO 16140 standard Selectivity & specificity Linearity & relative accuracy

" The alternative method gives results comparable to the reference method

"reliability of the tests?

! Reports & certificates : Deep informations& explanations!


08.19.06

"Methods comparison study

SpecificitySpecificity && selectivityselectivity

Linearity Linearity

Relative Relative accuracyaccuracy Methods precision

Detection & quantification limits

" Inter-laboratory study

BiasBias

RepeatabilityRepeatability & & Reproducibility limitsReproducibility limits Dispersion between laboratories

Validation study process

#

!


08.19.06

! Usually pasteurized milk samples with background microflora

! 3 different levels of contamination, + non inoculated samples, i.e. 1< to 3 Log CFU/ml

! 2 replicates / contamination level

! samples analysed by the reference and the alternative methods

! 8 laboratories (minimum)

Inter-laboratory study


08.19.06

Robust estimatorsStudent t table

Contamination level

Log CFU/ml Number of labs Bias D Log CFU/ml Conclusion

1 n = 12 - 0,05 non significant

2 n = 12 + 0,08 significant Me

t

h

o

d

2

3 n = 12 + 0,16 significant

Bias

Contamination level

Log CFU/ml Number of labs Bias D Log CFU/ml Conclusion

1 n = 12 - 0,19 non significant

2 n = 12 - 0,02 non significant Me

t

h

o

d

1

3 n = 12 + 0,01 non significant

A non significant bias for each analyte level is expected.Bias = 0,16 Log CFU/gsignificant?


08.19.06

Reproducibility limits Contamination level

Log CFU/ml Number of labs Reference

method Alternative

method P %

1 n = 10 0,522 0,277 4

2 n = 10 0,427 0,227 4

3 n = 10 0,247 0,187 21

Repeatability & reproducibility limits

Repeatability limits Contamination level

Log CFU/ml Number of labs Reference

method Alternative

method P %

1 n = 10 0,542 0,601 63

2 n = 10 0,141 0,094 11

3 n = 10 0,106 0,249 1

Ideal comparison, P > 5%According to the current EN ISO 16140 standard : at present nocriteria are available for the rejection of an alternative method

F-table


08.19.06

Inter-laboratory study

! Current EN ISO 16140 standard Bias Repeatability & reproducibility limits

" The alternative method gives results comparable to the reference method

"reliability of the tests

! Reports & certificates : Deep informations& explanations!


08.19.06

Conclusions

" Alternative methods validated according to the EN ISO 16140 standard (experimental design)

Give comparable results to the reference methods

Are quicker and/or easier than the reference methods

Offer a rapid & easy assessment of the microbiological quality of raw materials & finished products

Are internationally recognised.


08.19.06

Conclusions

"reliable tests ?friendly tools needed for users,

i.e. food microbiology laboratories !

The revision is now in progess


08.19.06 [email protected]

Thanks for your attention!


Deutsches Institut fr Lebensmitteltechnik e. V.

AFNOR VALIDATIONEuropean certification for microbiological test kits

19th June 2008, Brussels

More Information on Inter-laboratory-studies

German Institute of Food Technology (DIL)

Dr.-Ing. Helmut SteinkampBusiness Unit Manager

Food Safety Division

Quakenbrck, [email protected]+49 (0)5431 / 183 - 135



Competences of DIL

Applied Research (product and process development)

Food Analysis(chemical, microbiological, physical)

Quality & Safety Management

Technology Transfer

Apparatus / Prototype Design and Construction,

3

DIL works in partnership with small and medium sized food companiesand supports technology transfer from science to business.



Concept of DIL

Stuff at DIL:

- 30 scientists

- 60 technician



TTZ

DIL

A&FNIZO FHL

IGV

IVV

KIN

BfEL

ATB

LUFAMLUA

TUMUH

UK

FHTFHW

UBFHF

FHK

TUD

FHH

FHBFHN

TUBTFH

TNO

BfR

UM

TUHH



DIL in Food Innovation Process

Basic research

Applied research

Development of products/ processes

DistributionWay to market

Food industry

Companies

Service of DIL

Universities / Research Institutions Customer



Food Safety Division

Analytics

Microbiological and chemical analytics

Service for food and feed industry

Marketebility certificate

Implementation of monitoring systems

Quality Management

Implementation of Quality managementsystems (HACCP, BRC, IFS)

Cleaning control

Cross contamination control

Training of internal auditors and staff

Research/Consulting

Detection of R&D support in industry

Food Supply Chain Management

Cross contamination of allergens

Materials in contact withfood



Microbiological Analyis for Food Industry

Analysis of food products

Specific detection of pathogenic microorganisms

Detection of risk material, animal protein of unwanted species, genetically modified organisms (GMOs) and allergens

Cultivation and PCR technology

Identification of microorganisms

Shelf life testing



Sample Range

Meat and Meat products Eggs and Egg products Baked goods and their raw materials / ingredients Salads and Dressings Potato Products Canned Food Water (human /animal), Beverages, Ice Cream Feed Packing materials



1

Microbiological Testing

New EU-Regulations on Food Hygiene Improved control systems are needed in

Food Industry to improve safety Delays in Food Production Chain are getting

more important financial aspects New analytical techniques needed for rapid

conformation of safety



Why participating in Inter-laboratory Studies?

Requirements DIL runs accreditated laboratories Its a Question of Quality of our analysis

Challenge Comparision in these studies shows us the accuracy

of our service for industry Co-operation with Bio-Rad Food Industry wants State-of-the-Art



Inter-Laboratory Studies at DIL

50% of AFNOR tested micro-organisms

Time Micro-organism detected Reference Methode AFNOR-Validation MethodeSuccess

Mai 04 Salmonella spp. ISO 6579 Bio-Rad iQ Check Salmonella PCR ja

Okt 04 Coliforme/E.coli NF ISO 4832 / NF ISO Bio-Rad RAPIDE.coli 2 cultivation ja

Dez 04 Koagulase-positive Staphyloko NF EN ISO 6888-1 Bio-Rad RAPIDStaph cultivation ?

Feb 05 Listeria monocytogenes ISO 11290-1 Bio-Rad iQ Check Listeria monoPCR ?

Okt 05 Listeria monocytogenes ISO 11290-2 Bio-Rad RAPIDL.mono cultivation ja

Okt 05 Salmonella spp. ISO 6579 Bio-Rad RAPIDSalmonella cultivation tw.

Jul 06 Listeria monocytogenes ISO 11290-1 Bio-Rad RAPIDL.mono cultivation ja

Nov 06 Listeria spp. ISO 11290-1 Bio-Rad RAPIDListeria spp. cultivation ja

Apr 07 Listeria spp. ISO 11290-1 Bio-Rad iQ Check Listeria spp. PCR ?

Jun 07 E. coli O157:H7 ISO 16654 Bio-Rad RAPIDE.coli O157:H7cultivation ja

Mai 08 E. coli O157:H7 ISO 16654 Bio-Rad iQ Check E. coli O157 PCR ?



by Bio-Rad



What does DIL get out of Inter-laboratory Studies

Fulfil requirement of accreditation Control of our analytical work Challenge for the laboratory stuff Wide range of experience by detecting in

different food systems/matrices

Implementing new methods No internal validation needed

External testing helps improving



Cultivation Techniques

Well-known and easy to handle Nearly no equipment needed Less expensive Time consuming Results after three days



Enrichment in Pepton Water

21h

Enrichment in selektive bouillons

24h

SelektiveMedia

24h

CultivationTechniques(ASU L00.00-20)



PCR methodsASU L00.00-52

Enrichment in Peptone Water

21h PCR

3h

Proben-Vorbereitung

results



1

PCR vs Cultivation Technology

Samples from rapeseed meat: pork, poultry milk powder eggs and egg produkts

In total 828 samples tested on Salmonella spp. False-negative results:

Very few samples were tested positive by Cultivation but not by PCR

False-positive results: Due to remained DNA of cells especially in

processed eggs



PCR vs Cultivation Technology



Results of Internal Testing

Nearly 5% more positive by PCR technology Intensive internal testing Nearly all false-positive results were detected

as positive for Salmonella spp. Close relation to

food matix (i.e. egg products, rapeseed) stress on micro-organisms cultivation method

1 day vs 3 5 days for results of analysis


Deutsches Institut fr Lebensmitteltechnik e. V.AFNOR Validation Symposium, June 2008, Brussels


More equipment needed More expensive Higher accuracy

PCR Technique


13M Microbiology validation experiencesSYMPOSIUM BRUSSELS JUNE 19th 2008

EUROPEAN VALIDATION FOR MICROBIOLOGICAL TEST KITS

3M Microbiology Validation experience

Marie-Pierre Copin (Laboratoires 3M Sant : Cergy, France)


2Summary

! Introduction to 3M Company! 3M Microbiology products! 3M Microbiology products validations and recognitions! Some milestones on microbiological method validation in Europe! ISO 16140 in the European regulation on microbiological criteria! Afnor certificates and ISO 16140! Conclusion


3Six Market-Leading Businesses


4Consumer and Office Business

! Sales: $3.4 billion! Operating income: $688 million

Simplifying life at home and at work


5Display and Graphics Business

! Sales: $3.9 billion! Operating income: $1.2 billion

The products of choice in four large and growing industries


6Electro and Communications Business


Turning 3M technology into solutions in electrical, electronics and telecommunications markets worldwide


7Industrial and Transportation Business


Helping industrial customers solve their problems


8Safety, Security and Protection Services Business


Responding to growing demand for safety, security and protection


9Health Care Business


Reinvesting in core businesses, pursuing strategic synergies


10

Solving Problems Everywhere

! Operating companies in more than 60 countries and selling products in more than 200

! 63 % of sales are outside the United States! More than 42,000 employees internationally! We provide borderless customer success


3M Microbiology Mission3M MICROBIOLOGY MISSION: Provide a broad range of innovativesolutions that help improve safety, quality and business results for food & beverage processors, food preparation sites and reference laboratoriesglobally. Our offering will have relevant regulatory acceptance and will bebacked with exceptional customer support.


12

Plastic film

Adhesive + indicator

AdhesivePlastic coated paper printed with a grid

Multi-layer coating technology

Cold water soluble gel

Standard methods nutrients

Until November 2006: Main products range was

3M Petrifilm plates:alternative methods for Hygiene indicators


13

Total Count Enterobacteriaceae

E. coli/Coliform

Yeast and Mold RapidColiform

Coliform

High-Sensitivity Coliform

Staph Express Select E. coli

3M Petrifilm plates: Range of enumeration tests

EnvironmentalListeria


14

Using 3MPetrifilm plates for product analysis

1 Inoculation

2 Incubation

3 - Interpretation

3 steps only ..


15

November 2006:

3M company acquires Biotrace Ltd.

I. Hygiene monitoring control 3M Clean-Trace

3M Clean-Trace Water

3M NG

3M Protect

II. Environmental sampling products

III. Pathogen detection range


16

3M TECRA Pathogen test range

Alternative method for Pathogen tests

3M TECRA VIA and ULTIMA Salmonella VIA Salmonella ULTIMA Listeria VIA Campylobacter VIA E. coli O157 VIA Pseudomonas VIA Staphylococcus aureus VIA

3M TECRA Toxin VIA Bacillus Diarrhoeal Enterotoxin VIA Staphylococcal Enterotoxins VIA

3M TECRA Unique Salmonella

ELISA Tests Immunocapture Tests


17

Analytical methods

Standardized methodsStandardized methods Commercial/Alternative methods

Commercial/Alternative methods

References (ISO)References (ISO) Appropriate for routine use

Appropriate for routine use

OUR Customers need:To get evidence that choosen alternative method will give resultsequivalent to reference method

?


18

Some steps in the method approval organisation in Europe

! 1987: AFNOR validation committee implementation( 3M representated in the kit manufacturers college)

! 1989: First certificates issued ( 3M Petrifilm plates)

! 1993-1998: Eureka Microval project founded by Europe to get European schemeand rules in Europe(3M Participation in working groups)

! 1998- 2003:CEN group wrote EN 16140 Standard, became then ISO standard! 1998 Microval third party body implementation

(3M representative member of steering committee)! 1999: Nordval system is set-up! 2006: New European Regulation on microbiological criteria, refering to ISO 16140

standard! 2007 First Microval certificate


19

3M Microbiology: Pioneer in 3rd party method approvals Actions

! 1986 :Petrifilm Aerobic count plate and Petrifilm coliform count plate in milk: First alternative methodsrecognized by AOAC int

! 1989: Petrifilm Aerobic count plate and Petrifilm coliform count plate: First methods validated by AFNOR

! 1993: Some Petrifilm plates recognized as NMKL, then switched to NORDVAL validation.


20

Alternative method in the European Regulationon microbiological criteria (EN 2073/2005)

Article 5The use of alternative analytical methods is

acceptable, when the methods are validated against the reference method set down in Annex I * and certified by a third party in accordance with the protocol set in EN/ISO standard 16140 or other internationally accepted similar protocols


21

Analytical methodsEquivalence of methods

Standardized methodsStandardized methods Commercial/Alternative methodsCommercial/Alternative

methods

Validation according to ISO 16140by an independent third party

Validation according to ISO 16140by an independent third party

"

Microval

ISO, CEN


22

Ex Food Safety Criteria (in EN 2073/2005 )Salmonella in fruit and vegetable


23

Examples of Process hygiene criteria (in EN 2073/2005 )

Aerobic count and E .coli in Meat and Meat products


24

! Ex of afnor certificate! standard method


25

Reference to standard method is given in the AFNOR certificate


26

Validations and recognitions for 3M Petrifilm plates

+Environmental Listeria plate+++Yeast and Mould Count plates++++Staph Express Count plates

++Select E. coli Count plates+++E. coli and Coliform Count plates++High Sensitivity Coliform Count plates++Rapid Coliform Count plates+++Coliform Count plates++++Enterobacteriaceae Count plates+++Aerobic Count plates

RIOMACMMAS

ISO 16140


27

3M TECRA range products validation

AFNORSalmonella UniqueTEC-24/2-04/03 ( negative results in 24 hours)Salmonella UltimaTEC-24/3-12/03 ( negative results in 48 hours)Listeria Ultima TEC-24/4-09/04 ( negative results in 48 hours)

AOAC INTERNATIONAL Official Method of AnalysisSalmonella VIA Method 989.14Salmonella Ultima Method 998.09: Listeria VIA Method 995.22:


28

Conclusion

Validation scheme:! Big progresses to get a unique scheme based on an international standard: ISO

16140In reality: Recognition is more a European one.What about a more common scheme with AOAC?

Third Parties! AFNOR has already certified 63 methods following ISO 16140

MICROVAL issued 5 certificatesAs a manufacturer, we appreciate having the possibility to choose a third partyexpert laboratory to get our methods validated ( system improvement, speed )


Methods validation:A kit manufacturer perspective

Symposium AFNOR VALIDATION 19 June 2008

Dr. Raffaella GiardinoGlobal Marketing ManagerProgram Director Quality indicators

AgendaAgendaAgendaAgenda

1. Introduction

2. AFNOR Validation

3. Example EN ISO 16140 for TEMPO! Part A: Method comparison study! Part B: Inter-laboratory study

4. Conclusions


Chromogenic Pathogens Quality Indicators Identification

VITEK2

LIMS connection

DiversiLab

Typing

A Comprehensive Offer for A Comprehensive Offer for A Comprehensive Offer for A Comprehensive Offer for thethethethe

Food Microbiology LabFood Microbiology LabFood Microbiology LabFood Microbiology LabAPI

Traditional methods

ChromIDTM

VIDAS TEMPO

Count-TactTM

air IDEAL

BacT/ALERT 3D

PPM BioBall

ID32


Validation in Food Validation in Food Validation in Food Validation in Food MicrobiologyMicrobiologyMicrobiologyMicrobiology

" Third party assessment of method performances

" Official Authority requirement" Public Health issue" Customers need (external laboratories

and food industry)


bioMerieuxbioMerieuxbioMerieuxbioMerieuxDevelopment ProcessDevelopment ProcessDevelopment ProcessDevelopment Process

1. Feasibility2. Optimisation

3. Internal Verification4. External Verification

Design Control Phases (controlled by FDA):Design Control Phases (controlled by FDA):Design Control Phases (controlled by FDA):Design Control Phases (controlled by FDA):

Regulatory Affairs Dept. Clinical

Affairs Dept.

4 independents laboratory WW

Food Micro ValidationFood Micro ValidationFood Micro ValidationFood Micro Validation

5. Industrialisation6. External Validation7. Launch authorisation


Food Validations Food Validations Food Validations Food Validations of of of of bMx bMx bMx bMx productsproductsproductsproducts

" NordVal, vs. NMKL (Nordic Countries)" AFNOR Validation, EN ISO 16140, vs ISO-CEN" AOAC validation, USA, vs USDA or FDA" EMMAS Assessment (UK)" Health Protection Branch of Canada" DIN Committee (Germany)" Chinese Food Safety Authority" Japanese Government" Thailand Veterinary Authority"

List of Validations obtained in Food Micro for bioMrieux


bioMbioMbioMbioMrieux rieux rieux rieux AOAC Approvals AOAC Approvals AOAC Approvals AOAC Approvals

VIDAS Salmonella SLM (SC + TT) (May 1996 - Official Method N 996.08) VIDAS Salmonella SLM (RV + TT) (Jan 2004 - Official Method N 2004.03) VIDAS Salmonella ICS / plate (April 2001 - Official Method N 2001.07) VIDAS Salmonella ICS/ plate (April 2001 - Official Method N 2001.08) VIDAS Salmonella ICS/ SLM (April 2001 - Official Method N 2001.09) VIDAS Listeria LIS (December 1998 - Certificate N 981202) VIDAS Listeria LIS (harmonized protocol) (May 1999 - Official Method N 999.06) VIDAS Listeria LIS (harmonized protocol) (June 2004 - Official Method N 2004.06) VIDAS Listeria monocytogenes II (January 2004 - Official Method N 2004.02)VIDAS Listeria Species Xpress (LSX) (October 2005 - Certificate N 100501) with Ottaviani Agosti Agar (OAA)VIDAS Listeria spp & mono (LDUO) (October 2007- Certificate N 100702 ) VIDAS Staph enterotoxin SET2 (September 2004 - Certificate N 070404) VIDAS Staph enterotoxin SET2 (October 2007 - Official Method N 2007.06) VIDAS ECO + O157:H7 ID plate (January 2005 - Certificate N 010502) 8 and 24 hour ground beef protocols

Salmonella

Listeria

StaphenterotoxinE.coliO157:H7


bioMbioMbioMbioMrieux rieux rieux rieux AOAC Approvals AOAC Approvals AOAC Approvals AOAC Approvals

E. coli TEMPO EC (Escherichia coli ) August 2006 - Certificate N 080603; OMA on going

Total Flora TEMPO TVC (Total Viable Count) December 2006- Certificate N 120602; OMA on going

Coliform TEMPO CC (Coliform Count) June 2007 - Certificate N 060702; OMA on going

Enterobacteriacea TEMPO EB (Enterobacteriaceae ) May 2008 - Certificate N 050801

VITEK Identification Listeria (GPI and GNI+) (1992 - Official Method N 992.19)

VITEK Identification of Salmonella, E. coli (1991 - Official Method N 991.13)

and Other Enterobacteriaceae (GNI+)

API Biochemical Identification of Salmonella (1978 - Official Method N 978.24)

VITEK 2 GN - Identification of gram negative February 2008 - Certificate N 020802

VITEK 2 GP - Identification of gram positive December 2007 - Certificate N 120702

VITEK 2 BCL - Identification of Bacillus Pending AOAC RI approval

Identification



1. Introduction

2. AFNOR Validation


4. Conclusions


21 AFNOR validated methods, and all accordingly all accordingly all accordingly all accordingly the EN ISO 16140 protocolthe EN ISO 16140 protocolthe EN ISO 16140 protocolthe EN ISO 16140 protocol

AFNOR AFNOR AFNOR AFNOR Approved MethodsApproved MethodsApproved MethodsApproved Methods

0

5

10

15

20

25

1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

N

b

o

f

I

S

O

1

6

1

4

0

a

p

p

r

o

v

e

d

m

e

t

h

o

d

s


13 Qualitative Methods (pathogens detection)

5 methods for Salmonella detection

7 methods for Listeria/Listeria monocytogenes detection

1 method for E. coli O157:H7 detection

Qualitative methodsQualitative methodsQualitative methodsQualitative methods


VIDAS SLM double voie BIO 12/1-04/94 09/06/2010VIDAS SLM simple voie BIO 12/10-09/02 18/09/2010VIDAS Easy Salmonella BIO 12/16-09/05 20/09/2009VIDAS ICS2-SLM BIO 12/22-05/07 24/05/2011VIDAS ICS2-boite BIO 12/23-05/07 24/05/2011VIDAS Listeria BIO 12/2-06/94 09/06/2010VIDAS LSX BIO 12/12-07/04 01/07/2008VIDAS LMO2 (0.1ml-37C) BIO 12/11-03/04 12/03/2012VIDAS LMO2 (1ml-30C) BIO 12/9-07/02 03/07/2010Accuprobe LMO BIO 12/4-02/95 07/02/2011VIDAS LDUO BIO 12/18-03/06 09/03/2010OAA dtection BIO 12/14-04/05 07/04/2009

E. coli O157 VIDAS ECO BIO 12/8-07/04 05/07/2012

Mthodes qualitatives

Salmonelles

Listeria

AFNOR AFNOR AFNOR AFNOR Approved MethodsApproved MethodsApproved MethodsApproved Methods


8 Quantitative methods:1 for Total Count enumeration

3 for E. coli enumeration

2 for coliforms enumeration

1 for Enterobacteriaceae enumeration

1 for Listeria monocytogenes enumeration

Quantitative methodsQuantitative methodsQuantitative methodsQuantitative methods


Flore totale TEMPO TVC BIO 12/15-09/05 19/09/2009TEMPO EC BIO 12/13-02/05 04/02/2009Coli ID E. coli 37C BIO 12/19-12/06 14/12/2010Coli ID E. coli 44C BIO 12/5-03/99 19/01/2011TEMPO TC BIO 12/17-12/05 08/12/2009Coli ID coliformes 37C BIO 12/20-12/06 14/12/2010

Entrobactries TEMPO EB BIO 12/2112/06 14/12/2010Listeria monocytogenes OAA dnombrement BIO 12/24-03/08 28/03/2012

Mthodes quantitatives

Escherichia coli

Coliformes

AFNOR AFNOR AFNOR AFNOR approved Methodsapproved Methodsapproved Methodsapproved Methods


Classified by technology:11 VIDAS methods

4 TEMPO methods

5 media methods

1 molecular hybridation method

AFNOR AFNOR AFNOR AFNOR approved Methodsapproved Methodsapproved Methodsapproved Methods



1. Introduction

2. AFNOR Validation


4. Conclusions


Validation Protocol Validation Protocol Validation Protocol Validation Protocol ISO 16140ISO 16140ISO 16140ISO 16140

" A. Method comparison study1. Linearity2. Relative Accuracy3. Detection & Quantification limits4. Relative sensitivity5. Inclusivity and Exclusivity6. Practicability


1. Linearity 1. Linearity 1. Linearity 1. Linearity ---- TVCTVCTVCTVC

" TVC: in double at 40h and in double at 48h = 250 results

2.08 4.66troutListeria seeligeri BR1RTE Salmon with pates

2.41 5.28raw milkStaphylococcusaureus 501

Custard cream

2.41 4.93plantXanthomonas maltophilia 11.2

Green beans

2.32 4.95 mechanically separated meat

Bacillus cereus 29Wet cat food

2.30 - 5.04bufbourguignon

Enterobacter agglomerans 72

Porc pt

Range (log)Range (log)Range (log)Range (log)Isolated fromIsolated fromIsolated fromIsolated fromStrainStrainStrainStrainFoodFoodFoodFood

"5 food categories + 5 strains"5 contamination levels to cover the whole range"2 repetitions with TEMPO and Ref method per sample


1. Linearity 1. Linearity 1. Linearity 1. Linearity ---- TCTCTCTC

" Graphs for all foods/parameters + non-linearity tests" (example TC, dilution 1)

" The correlation coefficients are all greater than 0.98" The non-linearity test is not significant, therefore the linearity linearity linearity linearity

of TEMPO TVC, TC, EC and EB is goodof TEMPO TVC, TC, EC and EB is goodof TEMPO TVC, TC, EC and EB is goodof TEMPO TVC, TC, EC and EB is good.


2. Relative accuracy2. Relative accuracy2. Relative accuracy2. Relative accuracy

! 5 food categories Ex. TC range (log)o Meat, poultry products 1.00 4.63o Pet-food 1.00 5.43o Dairy products 1.52 5.48o Fruit and vegetables 1.85 6.15o Sea-food 1.40 5.04

! As much as possible naturally contaminated sample, artificial with strain stress

! TVC: in double at 40h and in double at 48h

" Protocol


2. Relative accuracy 2. Relative accuracy 2. Relative accuracy 2. Relative accuracy ---- TVCTVCTVCTVC

" 157 different naturally contaminated samples x (2 Ref. + 4 TEMPO) = 942 tests


3. Detection and quantification3. Detection and quantification3. Detection and quantification3. Detection and quantificationlimitslimitslimitslimits

" Critical Level CL" Detection Limit LOD" Quantification Limit

Data from statistical calculations; they are not included in the validation certificate, because not necessary. They will be probably change in the next revision of the EN ISO standard


4. Relative sensitivity4. Relative sensitivity4. Relative sensitivity4. Relative sensitivity

! Example plot from TC

! Use data from the linearity study! Plot the precision profile CV () vs. x(y)


5. 5. 5. 5. Inclusivity Inclusivity Inclusivity Inclusivity and exclusivityand exclusivityand exclusivityand exclusivity

" Protocol Protocol Protocol Protocol

1. Inclusivity: 30 positive pure strains (collection or food isolates)! In double with TEMPO, VRBG, VRBL/TBX and PCA! 3 Lactoseneg strains (TC), 2 -glucuronidaseneg strains (EC)! 180 tests per each parameter

2. Exclusivity: 20 negative strains (collection of food isolates)! In double with TEMPO, VRBG, VRBL/TBX and PCA! 120 tests per each parameter

3. Not applicable to TVC


5. 5. 5. 5. InclusivityInclusivityInclusivityInclusivity : EC: EC: EC: EC

Conclusion:Conclusion:Conclusion:Conclusion:

Perfect correspondence between TEMPO EC and ISO 16649-2 standard


5. Exclusivity : EC5. Exclusivity : EC5. Exclusivity : EC5. Exclusivity : EC

Conclusion:Conclusion:Conclusion:Conclusion:

No interference by non target strains


6. Practicability6. Practicability6. Practicability6. Practicability

" 13 evaluation criteria (only AFNOR)

! Training duration of an operatorA technician can be fully trained in 2 days

! Traceability of resultsA complete traceability for TEMPO Filler and Reader

! Delay to the resultsTEMPO TVC gives results in 40h compared to 72h of the ref. method, with a saving of more than one day

! QC check and lab maintenanceMonthly use of the kit QC to validate the TEMPO Filler and Reader performances


6. Practicability6. Practicability6. Practicability6. Practicability

" Real hands-on time (work-flow study) in relation to the number of samples to analyse*

4.14.14.14.113.913.913.913.9Time for 1 sampleTime for 1 sampleTime for 1 sampleTime for 1 sample

82278Total time

290Reading

20107Dilution and inoculation

-21Petri dishes preparation

1515Dilution and stomaching

4545Sample Weighing

Method TEMPO TVCMethod TEMPO TVCMethod TEMPO TVCMethod TEMPO TVCMethod ISO 4833Method ISO 4833Method ISO 4833Method ISO 483320 samples20 samples20 samples20 samples

* Media and tubes preparation time non included


Validation Protocol Validation Protocol Validation Protocol Validation Protocol ISO 16140ISO 16140ISO 16140ISO 16140

" B. Inter-laboratory study1. Relative Accuracy2. Repeatability & Reproducibility limits3. Ratio R/r4. Dispersion between laboratories


InterInterInterInter----laboratory studylaboratory studylaboratory studylaboratory study

" ProtocolProtocolProtocolProtocol

! 12-13 European labs participating to each study + organising lab

! 8 Pasteurized milk samples with natural flora for TC, EB and EC, UHT for TVC

! Samples at 4 levels of artificial contamination over the whole range

! Analysed with TEMPO x 2 and ISO method at day 1


Example TEMPO TCExample TEMPO TCExample TEMPO TCExample TEMPO TC



Example TEMPO TCExample TEMPO TCExample TEMPO TCExample TEMPO TC


Alternative ISO Alternative ISO1.97 0.46 0.23 0.5 0.272.9 0.28 0.16 0.39 0.18

3.92 0.28 0.14 0.31 0.16

Contamination Level (log cfu/g)

Repeatibility Reproducibility

TVC

Alternative ISO0.78 0.880.91 1.241.27 1.78

Dispersion between laboratories

EC

Example TEMPO ECExample TEMPO ECExample TEMPO ECExample TEMPO EC



1. Introduction

2. AFNOR Validation


4. Conclusions


AFNOR ValidationAFNOR ValidationAFNOR ValidationAFNOR Validation

1. A clear set of rules" Which makes it possible to organise the validation process without major

unexpected events" The different parameters undergo similar procedures, allowing a general

coherence in the validation activities:! submission of the data file! content of the validation certificate! Content of the Summary Reports! post validation management (I.e. new software versions, etc.), ! in the audits and in the overall costs

2. Good Representativity in the AFNOR Technical Committee

" Presence of Official Bodies, alternative method users, Kit manufacturers, etc., assuring an equilibrated evaluation of the data files

" the practical side (users point of view) is also taken into account, and sometimes the good sense prevails on purely statistical considerations


5. Professional excellence of expert laboratories" Very good knowledge of the ISO 16140 protocol which allows the

production of clear reports and presentations

3. Different expertise in the AFNOR Technical Committee

" Good associations of scientific, regulation and technical knowledge; a group of people not only one person taking decision

4. Frequency of the Technical Committee meetings" Approximately every two months. The validation delays are easily

foreseeable

6. Public web-site" Where is possible to download not only the Validation Certificate, but the

Summary Report too.

AFNOR ValidationAFNOR ValidationAFNOR ValidationAFNOR Validation


Nestl Research Center2007-03-20 NRC/QS JMr1

- Rapid method validation -Experiences from a perspective of a food

manufacturer

AFNOR Validation - 19 June 2008 John D. Marugg,

Nestl Research Centre, Lausanne, Switzerland




This presentation

1 Does the food industry need rapid methods?2 Requirements and selection process 3 Nestl validation procedure4 Examples of Nestl/AFNOR approved alternative methods5 Harmonisation and challenges



Nestl, the biggest Food Company

276'000 employees

Sales of CHF 107.6 billion in 2007

93% of sales is food and beverages

480 factories in 86 countries

Prepared dishes & Cooking aids

17%

Pharmaceuticalproducts

7%

Milk products & Ice cream19%

Powdered & liquid beverages

17%

Nestl Waters10%

Confectionery 11%

PetCare

11%

Nestl Nutrition 8%



Nestl's Billionaire Brands



Many products. many analyses needed

! > 250 Labs involved in microbiological testing finished products, ingredients, environmental samples

! ~ 50 Salmonella labs

! ~ 1.000.000 Salmonella analyses per year > 99% negative results

! ~ 0 Salmonella outbreaks per year



Common limitations of traditional (cultural) methods

! Slow Delays release finished products and

ingredients Delays response to data from environmental

monitoring programs

! Many yield false negative and false positive results/large measurement uncertainty

! Labour intensive

Conclusion: Food industry indeed needs rapid methods

Choice is enormous How to decide which method is best?



Rapid Methods: huge diversity in principles + strategies

! metabolic properties Miniaturisation and Diagnostic Kits, Biochemical Identification Techniques chromogenic agar broths

! antibody-based Elisa, dipstick, lateral flow... Immunoconcentration Immunomagnetic separation

! Nucleic acid-based Hybridisation PCR riboprinting, AFLP,PFGE, DNA chips.



Minimum requirements for the ideal rapid method

Flexibility multi-purpose, multi-target, detection + identification, non-destructive

Future Nestl microbiologist in search of Salmonella

Sensitivity 100%

Specificity 100%

Speed fast " on-line

Ease-of-use fully automated

Costs zero



Rapid Methods: Reality

Challenges: Foods represent a wide variety of complex matrices Target organisms often present in low numbers Intrinsic bacterial flora may hinder identification Processing of foods results in stressed or injured

cells Lengthy enrichment procedures required to detect

viable cells



Requirements Rapid Methods

! Validated preferably ISO 16140, AOAC, Nordval etc.

! Reliable Also suited for Nestl specific matrices Composite samples (pooling)

! Recognized by authorities and trading partners! Fast result! Low costs

Equipment Reagents Personnel Laboratory space costs of waste disposal



Requirements rapid methods (cont)

! No extra requirements for technicians easy training

! Compatible with normal lab organisation! Sufficient capacity/throughput! Multifunctional (adapted for Salmonella, Listeria, etc)! Automation

limit possibilities human errors! Compatible LIMS! Guaranteed availability/delivery

reliable supplier world wide available (also in the future) quick repair/back up equipment in case of problems

! Easy exit strategy



Collect information candidate methods

- Supplier- Validation organisations

- Colleague microbiologists- Publications- Other suppliers



Pragmatic solution

Procedure: Review literature, data from supplier and external independent

validation data Comparison with other methods Test pure cultures of target and non-target microorganisms Analysis of different matrices (spiked and naturally contaminated) Repeatability & reproducibility

Compile results

Decision taken by an expert panel



Development of a coordinated approach for method selection

! Method evaluation and approval Expensive Requires specific competence Should be coordinated

! 1998, implementation NesVal system: a Common framework for internal evaluation and validation of new and alternative methods



NesVal, main features

! Centrally coordinated evaluation of rapid methods microbiology allergens, chemical contaminants (mycotoxins, antibiotics) Modifications of existing methods, scope extensions (new matrices)

! Systematic choice of candidate methods Determined by needs of laboratories & offers from method suppliers

! Fixed procedure, complementary to ISO 16140 Special attention to Nestl specific matrices Comprises usually an first evaluation by NRC followed by inter-

laboratory studies Judges methods also with respect to other requirements



NesVal, main features (cont)

! Technical Expert Committee Technical evaluation at the Nestl Research Centre Switzerland

- Assessment external evaluation data (suppliers, external validation)- Preliminary screening trials- Coordination of trials

Maintain contacts with suppliers- Feedback on results of trials- Explain what our needs are

! Advisory committee Ca 20 members, representing markets, R&D, Corporate QM,

business units Define needs and priorities Ensure communication and implementation



Validation status rapid methods NesVal May 2008

Method Target Throughput Approval/ LI number(s/series) Status

ELISA-based (


VIDAS system (BioMrieux)

! The VIDAS is an Enzyme Linked Fluorescent Assay (ELFA) for detection of Salmonella antigens carried out on a multi-parameter automated immuno analyser.

! Each test consists of:! Solid Phase Receptacle (SPR)

solid phase coated with anti-S antibodies pipetting device

! Ready-to-use reagent strips

Tests available: SLM, easy SLM, LIS, LMO2, LSX, enterotoxins, etc.

0.1 ml

Pre-enrichment 18h

RVS 7h

1 ml

1 ml

M-broth 16h

15 min 100C

Elisa 1h

0.2 ml



Transia system (Biocontrol)

Transia Plate Salmonella Goldis a sandwich type Elisa assay on a microtiter plate format

Each test consists of: microtiter plate with solid phase (strips) coated with anti-Salmonella antibodies polyclonal Ab as capture specific monoclonal Ab as tracer

Other assays: Listeria, enterotoxins, allergens

1 ml

0.1 ml

1 ml

Pre-enrichment 18h

RVS 20h

15 min 100C

Elisa 1h



PCR method dedicated to use in food industry

Automated DNA amplification with fluorescent detection

# Tablets contain all reagents necessary for PCR- Taq polymerase- Primers - Nucleotides- Control reaction- Fluorescent dye

# AssaysSalmonella, Listeria genus, Listeriamonocytogenes, E. coli O157 and E.sakazakii

BAX system (Dupont Qualicon)

50 l

10 l

5 l

Pre-enrichment 18h

BHI 3h

Lysis reagent 20 min 37C 5min 100C

PCR 3+1h


2007-03-20NQAC Cergy/OGG

NRC/QS JMr 21

Day 1Day 1Day 1Day 1

RapidRapidRapidRapidL. monoL. monoL. monoL. mono

37C, 24 37C, 24 37C, 24 37C, 24 2 hhhh

Suspend 25g or 25 ml sample 1:10

1/2 Fraser brothFraser brothFraser brothFraser broth

30C, 24 30C, 24 30C, 24 30C, 24 2222 hhhh


Confirmation(s)Confirmation(s)Confirmation(s)Confirmation(s)Spot on Ottaviani & AgostiSpot on Ottaviani & AgostiSpot on Ottaviani & AgostiSpot on Ottaviani & Agosti

or Gen Probeor Gen Probeor Gen Probeor Gen Probeor othersor othersor othersor others


0.1 ml

Palcam + O&APalcam + O&APalcam + O&APalcam + O&A 0.1 ml in 10 ml 0.1 ml in 10 ml 0.1 ml in 10 ml 0.1 ml in 10 ml FraserFraserFraserFraser

37C, 24 37C, 24 37C, 24 37C, 24 2 hhhh


37C, 48 37C, 48 37C, 48 37C, 48 2 hhhh

Palcam + O&APalcam + O&APalcam + O&APalcam + O&A37C, 24 37C, 24 37C, 24 37C, 24 2 hhhh

ConfirmationConfirmationConfirmationConfirmation

ConfirmationConfirmationConfirmationConfirmationDay 5Day 5Day 5Day 5

EN ISO 11290-1RLM



Hygiene indicators: Need for rapid validated methods

! Hygiene indicators are used for product releaseTVC, Enterobacteriaceae, Coliforms, E. coli

! Main available methods:Lab efficiency TAT Calibration ISO16140

Petrifilm + No Yes

MicroFoss + ++ Required No/Bactrac /Yes

Simplate ++ ++ No

afnor-validation2008

Documents

symposium afnor

afnor certification

sancoafnor validation

food microbiology15

food manufacturerspeaker

food safety laboratory

rva j0958welcome

use of alternative methods