corry et al. 2007.pdf

ARTICLE IN PRESS

FOODMICROBIOLOGY

0740-0020/$ - se

doi:10.1016/j.fm

�CorrespondE-mail addr

Food Microbiology 24 (2007) 230–253

www.elsevier.com/locate/fm

A critical review of measurement uncertainty in the enumeration of foodmicro-organisms

Janet E.L. Corrya,�, Basil Jarvisb, Sue Passmorec, Alan Hedgesd

aSchool of Clinical Veterinary Science, University of Bristol, Langford BS40 5DU, UKbRoss Biosciences Ltd, Upton Bishop HR9 7UR, UK

cHickory Hollow, Old Coach Road, Cross, Axbridge BS26 2EQ, UKdDepartment of Cellular and Molecular Medicine, University of Bristol, Bristol BS8 1TD, UK

Received 13 April 2006; received in revised form 22 May 2006; accepted 23 May 2006

Available online 1 August 2006

Abstract

Derivation of uncertainty provides a way to standardize the expression of variability associated with any analytical procedure. The

published information on uncertainty associated with data obtained using microbiological procedures is reviewed to highlight the causes

and magnitude of such variability in food microbiology. We also suggest statistical procedures that can be used to assess variability (and

hence, uncertainty), within and between laboratories, including procedures that can be used routinely by microbiologists examining

foods, and the use of ‘robust’ methods which allow the retention of ‘outlying’ data. Although concerned primarily with variability

associated with colony count procedures, we discuss also the causes of variability in presence/absence and indirect methods, such as

limiting dilution, most probable number and modern instrumental methods of microbiological examination. Recommendations are also

made concerning the most important precautions to be taken in order to minimize uncertainty in microbiology. These include strict

internal controls at all stages of microbiological testing, as well as validation of methods, trend analysis, use of reference materials and

participation in proficiency testing schemes. It is emphasized that the distribution of microbes in foods is inherently heterogeneous, and

that this review only addresses uncertainty of measurement with respect to the sample taken, not the lot or consignment of food from

which the sample was taken.

r 2006 Elsevier Ltd. All rights reserved.

Keywords: Uncertainty; Confidence limits; Variability; Statistical procedures; Colony counts; Validation; Reference materials; Precautions

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

2. Methods and techniques to estimate uncertainty. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

2.1. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

2.2. Components of uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

2.2.1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

2.2.2. Samples (heterogeneity in the substrate) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

2.2.3. Sub-samples (homogeneity and size) and dilutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

2.3. Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

2.3.1. Method robustness (effect of errors in preparation of media, temperature variation, state of target organism) 235

2.3.2. Media and diluent preparation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

2.3.3. Plating and incubation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

e front matter r 2006 Elsevier Ltd. All rights reserved.

.2006.05.003

ing author. Tel.:+44117 928 9409; fax:+44117 928 9324.

ess: [email protected] (J.E.L. Corry).

www.elsevier.com/locate/fm

dx.doi.org/10.1016/j.fm.2006.05.003

mailto:[email protected]

ARTICLE IN PRESSJ.E.L. Corry et al. / Food Microbiology 24 (2007) 230–253 231

2.3.4. Counting colonies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

2.3.5. Confirmation techniques—sensitivity, specificity and selectivity of media . . . . . . . . . . . . . . . . . . . . . . . . . . 237

3. General considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

3.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

3.2. Quantitative methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

3.3. Semi-quantitative methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

3.4. Qualitative methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

3.5. Alternative methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

4. Data for method validation and control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

4.1. Method validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

4.2. Reference materials and proficiency testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

4.3. Analytical quality control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

5. Approaches to the determination of uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

5.1. Organisation of inter-laboratory trials. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

5.1.1. Basic requirements for microbiological collaborative trials (ISO 16140, 2003). . . . . . . . . . . . . . . . . . . . . . . 245

5.2. Preliminary evaluation of quantitative data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

5.3. Analysis of variance (ANOVA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

5.4. Alternative approaches to ANOVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

5.5. Determination of repeatability and reproducibility using quality monitoring (QM) data . . . . . . . . . . . . . . . . . . . . . 246

5.6. Determination of uncertainty from reproducibility estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

5.7. Uncertainty associated with qualitative tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

6. Recommendations to minimize uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

6.1. Personnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

6.2. Equipment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

6.3. Diluents and media . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

6.4. Incubation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

6.5. Primary sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

6.6. Analytical (test) sample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

6.7. Examining cultures and recording data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

6.8. Quality monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

Appendix A. Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

A.1. Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

A.2. Uncertainty source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

A.3. Uncertainty components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

A.4. What is the difference between error and uncertainty? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

A.5. Definition of some key terms associated with uncertainty measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

1. Introduction

Measurement of uncertainty has been a commonplacerequirement in physical and chemical analyses for manyyears but it is only recently that the subject has beenaddressed by microbiologists. Whilst the accepted conceptis the measurement of the ‘‘level of uncertainty’’ associatedwith a microbiological test, the recipient really wants toknow the ‘‘level of confidence’’ which the microbiologist canput on the particular result.

Increasingly, laboratory accreditation procedures, andboth national and international definition and standardisa-tion of laboratory methods seek to define the level ofuncertainty which can be ascribed to a series of tests. TheBritish Standards Institute, the International StandardsOrganisation (ISO), Codex Alimentarius, the InternationalDairy Federation (IDF), the Nordic Committee forMicrobiological Standardisation (NMKL) and AOACInternational are but a few of the organisations currently

seeking to define and to provide measurements ofuncertainty associated with methods used for the examina-tion of foods for pathogenic and other micro-organisms.Evaluation of measurement uncertainty is a requirement

for laboratories operating ISO/IEC 17025 (1999, 2005)accreditation systems and this includes those carrying outmicrobiological examinations. Whilst this review is pri-marily concerned with methods and techniques in foodmicrobiology, the concepts and techniques for estimatinguncertainty are applicable to other areas, since micro-biological techniques are generic and often common todifferent matrices.In most areas of microbiological testing, results are

incomplete without some interpretation of their signifi-cance in the context of specifications, legislative limits orassessments of public health significance. Without ameasure of the reliability of the results, such interpretationsmay be meaningless, and estimations of measurementuncertainty are becoming a requirement of bodies setting

ARTICLE IN PRESSJ.E.L. Corry et al. / Food Microbiology 24 (2007) 230–253232

commercial specifications, legislative limits, guidelines andstandards and by bodies commissioning surveillance work(e.g. the European Food Safety Authority and UK FoodStandards Agency).

A proper interpretation of results obtained using anyanalytical procedure requires the analyst to consider care-fully the diverse sources of error associated with the dataobtained. These comprise three major factors: (a) randomerrors; (b) systematic bias of the analytical procedure; and(c) application errors within any particular laboratory.

Bias can be introduced by: (a) the collection, transporta-tion and storage of the original sample material; (b) theaccuracy and representativeness of sub-sampling; (c) thechoice of method; and (d) the accuracy of applying allprocedures involved. Some aspects of bias are within thecontrol of the microbiologist; others are not. One of themajor sources of uncertainty is the distribution of microbesin the sample and the way in which the analytical sample istaken.

Formal concepts and terminology for measurementuncertainty were brought together with the publication ofthe Guide to the Expression of Uncertainty in Measure-ment (GUM) (ISO, 1993). Many discussions and guidancedocuments from international standardisation and otherbodies to interpret GUM requirements for particularmeasurement areas have followed. It must be emphasizedthat ‘uncertainty’ as defined in these documents appliesonly to the statistical variation associated with the resultsof an analysis on the sample examined. Whilst for manyyears, food microbiologists have estimated the reliability oftheir quantitative enumeration methods at best as70.5 log

10colony forming units (cfu), the task now is to

refine such statements to provide reliable and objectiveestimates for uncertainty of results obtained by micro-biological methods.

Estimates of ‘‘uncertainty’’ are frequently derived asmeasures of repeatability (r) and reproducibility (R) usingdata generated in one or more standardized laboratoryprocedures in a series of inter- and intra-laboratory tests.The collective data are analysed statistically to derivevalues for these parameters, which are then appliedgenerally to the use of that procedure and type of sample.

It has been suggested (e.g. ISO, 1993; Eurachem, 2000)that a component-by-component (‘‘bottom-up’’ or Type B)approach for evaluating measurement uncertainty shouldbe used. In this approach, the measurement method isbroken down into all those components likely to have aneffect on the result. The estimated error of each individualcomponent stage is assessed before being combined intoan overall (combined standard uncertainty) value. Thisapproach is used in documents M3003 and LAB12published by the United Kingdom Accreditation Service(UKAS, 1997, 2000), and has been explored in the contextof food and water microbiology by NMKL (1999), Niemiand Niemela (2001) and Niemela (2002). Niemela (2002)introduced the term ‘‘manageable uncertainty’’ to definethose components that could be assessed to provide

estimates of combined uncertainty for single microbiologi-cal results. Complex statistical procedures were documen-ted, but these may not be suitable for use in routinelaboratories. Unfortunately, such a ‘‘bottom up’’ approachtakes no account of differences in sample matrix type,operator bias or other built-in error, and is extremely time-consuming and difficult to carry out.For most physical and chemical analyses, the distribu-

tion of levels in a set of parallel sub-samples conforms tothe ‘‘normal’’ statistical distribution. In such circum-stances, it is possible to analyse replicate samples in arange of laboratories and, from the accumulated data, toobtain statistical estimates of the various components ofimprecision through analysis of variance (ANOVA). Theunderlying assumption is that the actual measurementsconform, at least approximately, to a normal distribution.The cause of any lack of homogeneity (e.g. as shown byoutlier results) must be analysed before ANOVA isundertaken, and may lead to elimination of particularanalysts, samples or laboratories from the dataset.Estimates of the residual (sr) usually include a componentdue to the variation between replicate error (presumedidentical) sub-samples, unless the study design, e.g. Jarviset al. (2004), allows for the extraction of this component,which can be statistically significant.Colony count data derived by examination of biological

specimens, including foods, frequently conform approxi-mately to the lognormal distribution and sometimes to aPoisson distribution (in which the variance betweenreplicate values equals the analytical mean of the values).In the former case it is necessary, before statistical analysis,to transform the data to logarithms, whereas in the secondcase, a square root transformation should be used toconform to an approximately normal distribution. Un-fortunately, there are many circumstances where countsconform to a negative binomial distribution for which thetransformation of counts is much more complex (Jarvis,1989; Niemela, 1996). Further detail on the statisticalprocedures is given in Section 5.

2. Methods and techniques to estimate uncertainty

2.1. Background

Whilst a ‘‘bottom-up’’ (component by component) ap-proach to estimation of uncertainty is feasible for non-destructive sampling procedures where measurements can bereplicated without effect on the sample, it is regarded as lesssuitable for ‘‘destructive’’ testing where replicate analyses aremade on separate sub-samples. Alternative approaches havebeen proposed for analytical chemistry (EURACHEM, 2000;Horwitz, 2003). For microbiological examinations, whereresults cannot be simply compared as they have often beenobtained from a heterogeneous matrix, further developmentof these alternatives has been necessary.Although EA-04/10 (2002) and other publications

discussing application of uncertainty estimates to individual


results advocate omission of sample variability, this is notfeasible with the inherent distributional variability in sub-samples or even replicate dilutions for food microbiology(Niemela, 1996; NMKL, 1999). In water microbiology,where low numbers of organisms are often assessed forpublic health significance, this heterogeneity is welldocumented (Tillett and Lightfoot, 1995) and ‘‘uncer-tainty’’ estimations are largely based on distributionalvariations in the absence of method-precision data(NMKL, 1999; Niemela, 2002; ISO 8199, 2005).

Another major problem with the component-by-compo-nent approach for microbiological examinations is that theuncertainty associated with some components of themethod (e.g. incubation temperature fluctuations andmedia quality) is effectively not quantifiable for anyindividual experiment. Moreover, such effects are usuallyoverwhelmed by the uncertainties associated with micro-bial distribution in the sample. The effects on the end resultof other components, such as serial dilutions, have longbeen appreciated and can be quantified in any uncertaintybudget (Hedges, 1967, 2002, 2003; Jarvis, 1989). Anexample of component assessment was given in the PHLSStandard of Practice (1998), and components are fullylisted in CCFRA Guideline 47 (2004a, b).

However, whatever approach is used to assess uncer-tainty, it is an essential prerequisite that sources ofvariability (i.e. the components of uncertainty) in testmethods and subsequent results should be identified (ISO/IEC 17025, 1999, 2005; EURACHEM, 2000; Horwitz,2003). This can be done in a number of ways, but processflowcharts or lists (e.g. Jarvis, 1989; PHLS, 1998) or‘‘fishtail’’ diagrams (EURACHEM, 2000) have been usedeffectively in many laboratories.

Once the sources of uncertainty have been identified, itfollows logically that the microbiologist seeks to controlthem. This is the pragmatic route taken over many years byUKAS and other accreditation agencies as well as mostlaboratories operating reputable quality systems. Thealternative ‘‘top-down’’ (Type A) approach is based onestimates of repeatability and reproducibility of testsundertaken in laboratories that are in control (ISO/TS21748, 2002).

Often, standard microbiological methods have omittedcredible information on method performance. A pro-gramme of international collaborative trials conductedaccording to the International Protocol (IUPAC, 1993)was undertaken to provide the necessary data (Lahellec,1998). Statistical methods for evaluating the precision ofcomponent parts of methods or entire methods areavailable and are being used to produce precision datafor standard methods for food and feed examination.These are based on collaborative trial results, for exampleBacillus cereus (Schulten et al., 2000), Listeria (Scotteret al., 2001), bifidobacteria (Leuschner et al., 2003) andClostridium perfringens (ISO 7937, 2004). Individuallaboratories can build on these published estimates ofrepeatability and reproducibility when validating methods

for their own use and can add in their own routine qualitycontrol data to demonstrate that their long-term perfor-mance meets the published criteria (Codex Alimentarius,2004). The use of control charts, often linked to laboratoryinformation management systems, is common in analyticalchemistry to show immediately when a result falls outsidepreviously set precision limits, and this concept can also beeffective for microbiology to show if some systematic oroperational error has occurred in an analysis. This has beendiscussed for water testing using duplicate split samples byLightfoot et al. (1994), in SCA (2002) and in more generalterms by Niemela (1996). EU Regulation 2073 (2005) onmicrobiological criteria for foods demands the use of trendanalyses for microbiological data.Experimental repeatability data, or data obtained from

routine Internal Quality Control (IQC) duplicate samples,have been used in estimations of uncertainty for (food)enumeration methods by ANOVA by PHLS (1998),Voysey and Jewell (1999) and the AAG (2005). The lasttwo publications emphasised the necessity of separatingassessment of results at different levels and from differentmatrices tested by the same method, in order to provideseparate uncertainty estimates as laid down in ISO/TS21748 (2002). Voysey and Jewell (1999) and ISO/TS 21748(2002) noted that this approach would minimize uncer-tainty, resulting in individual laboratories quoting lowuncertainty estimates. Voysey and Jewell (1999) recom-mended that uncertainty should not be quoted formicrobiological results until further work had been under-taken, but CCFRA (2004b) gives guidance on when andhow to report uncertainty, as also does the ISO/TS 19036(2005).Another option is the critical relative difference (RD)

approach advocated as ‘‘very useful for the analyst’’ byPiton and Grappin (1991) when comparing novel enumera-tion methods for dairy products with the IDF Referencemethods. This approach has also been adopted in ISO/TS19036 (2005) to establish equivalence between microbiolo-gical methods. Critical difference and critical differencelimit, as defined in ISO 5725-3 (1994), is also included in an(informative) Annex to ISO 4833 (2003), where the conceptis used to interpret a single result under repeatability orreproducibility conditions and to compare a result with alimit or specification.If the use of the ‘‘top down’’ approach with collaborative

trial data is pursued as a means of estimating uncertaintyfor microbiological enumeration methods, then considera-tion must be given to the inclusion of trueness assessmentsin the estimates. This is usually performed by applying themethod to a Certified Reference Material (CRM), but thisis problematic for microbiology where the ‘‘true’’ value isimpossible to know and can only be estimated. At best, any‘‘bias’’ in results caused by systematic errors in laboratoryoperations or deficiencies in the method chosen may berevealed by participation in external Proficiency Schemesconducted according to ISO/IEC Guide 43 (1997). Resultsfor individual parameters should be plotted to show any


trends (ISO/IEC 17025, 1999, 2005), and thus demonstratethat bias is kept under control, before the ‘‘top down’’principle can be applied to provide estimates of uncer-tainty. A new approach outlined in CCFRA Guideline no.47 (2004b), combines precision and bias estimates by use ofa spreadsheet.

For quantitative microbiological examinations, the wayahead might be to combine approaches, as differentmethods for uncertainty estimation have their owndeficiencies and can underestimate when taken in isolation.This was advocated by Voysey and Jewell (1999) and isreinforced in ISO/TS 21748 (2002), with the statement that‘‘practical uncertainty estimates accordingly often use someelements of both extremes’’, referring to the GUM (ISO,1993) ‘‘bottom up’’ component method and the collabora-tive trial ‘‘top down’’ method.

2.2. Components of uncertainty

2.2.1. Introduction

Microbiological examination of food involves manyprocesses, including collecting samples; transporting themto the laboratory; taking subsamples, and preparing aninitial suspension in diluent or a liquid growth medium,which includes effective mixing. Each stage adds to thevariability of the result. However, estimation of measure-ment uncertainty is concerned with the variability of theanalytical procedure per se and excludes consideration ofall stages prior to the taking of an analytical sample. Forquantitative examinations, a series of dilutions is preparedand measured volumes are plated onto or into appropriatemedia. The numbers of colonies (sometimes only thosewith particular characteristics) are counted after incubationunder specified conditions. In the case of most probablenumber (MPN) methods, a number of tubes of liquid orsolid growth medium are prepared in parallel at eachdilution and results are recorded as the number of tubes inwhich the target organism grows, sometimes after con-firmatory tests. In presence/absence tests, e.g. for patho-gens, a small, defined quantity of food is examined. Thetest may include preenrichment and enrichment in a liquidmedium followed by plating a small volume onto a solidselective medium. Confirmatory tests are carried out on aproportion of typical colonies of the species sought.Alternative more rapid tests (immunological or PCR-based) are now available for detection of pathogens such assalmonella, campylobacter and Listeria monocytogenes.

These are most often applied to the enrichment culturerather than the initial suspension because non-viable cellsmight be present in the initial sample; furthermore,substances in foods can inhibit PCR reactions. Variousalternative ‘rapid’ methods are also widely used forquantitative methods (Section 3.5).

Common systematic components of the steps in theexamination of food, include:

Traceability: A system is required to ensure that theidentity of each sample can be followed throughout the

examination process, so that the result can be relatedconfidently to each specific sample. The need for this mayseem obvious, but without reliable traceability, estimationsof reliability of measurement are meaningless.

Calibration: Equipment (e.g. balances, pipettes, pHmeters) and instruments (e.g. thermometers and thermo-graphs) used must be calibrated against reference standards.

Tolerances and settings: Practical considerations dictatethat some latitude must be allowed in terms of themeasurements made with the calibrated equipment andinstruments, e.g. the temperature of the water bath used totemper molten agar for pour-plates could vary between 42and 47 1C. Temperatures of incubation of media can alsovary, even sometimes for standard methods—e.g. 35 1C isused in the USA, while most European laboratories use37 1C. Some laboratories incubate media for ‘thermophilic’campylobacters at 37 1C, others at 42 or 43 1C, while theISO 10272-1 (2006) standard method suggests 41.5 1C.The degree of dryness of agar surfaces is impossible tostandardize.

2.2.2. Samples (heterogeneity in the substrate)

Sampling of food for microbiological examinationinvolves two stages. ‘‘Representative’’ samples have firstto be taken from the original material, then these primarysamples have to be homogenized and a sub-sample drawnthat accurately reflects the microbial population of theindividual sample. Obtaining representative samples from abulk food is discussed in detail by ICMSF (2002) andrequires that sampling be carried out at random. Inpractice, it may be impossible to be confident that thesample(s) are truly representative. The confidence inthe result depends largely on the number and size of thesamples examined.

2.2.3. Sub-samples (homogeneity and size) and dilutions

Microbes are rarely distributed randomly in foods oftenoccurring in clumps that represent micro-colonies (Mosselet al., 1995). To allow for this, the sub-sample should bethoroughly homogenized before removal of portions fortesting. Liquid products such as milk are easiest to dealwith, and can usually be mixed by thorough shaking. Forbulk dried material the ‘quadrant diminution’ method hasbeen suggested (Harreweijn et al., 1972, cit. Mossel et al.,1995). Sampling multi-component products, such as readymeals, meat pies or sandwiches is more problematic. Thechoice is whether to take a composite sample containingrepresentative proportions of all the components, tosample all the components separately, or to sample onlythe components considered to pose the highest potentialrisk. For practical reasons, composite samples are oftentaken of ready meals, while a composite sample of asandwich or pie filling, without the bread or pastry, isusually taken. Even when the primary sample consists ofone component in a relatively small package, it is rare forthe whole package to be used for microbiologicalexamination. Portions of 25 g or less are most commonly


used, and should be made up from different parts of thesample. The microflora of raw fruit and vegetables andpieces of raw meat or poultry is found on the surface, andsamples are usually measured in terms of cm2 (area ratherthan weight). With raw meat, there is also a choice oftaking swab samples or excising the area to a depth of1–2mm. Excision is generally considered to recover moremicrobes than swabbing with gauze, sponge or cotton wool(Gill, 2001). Also, in spite of a specified protocol, the exactmethod of swabbing (e.g. time and direction(s) of rubbing,and pressure applied) will inevitably vary between indivi-dual technicians. Whole poultry carcasses or portions offruit and vegetables are sometimes sampled by rinsing withwater or diluent.

The next step in microbiological analysis is to prepare asuspension of the food and its microflora in a suitablediluent or enrichment medium. The diluent can either bemeasured by mass or by volume. Automatic devices(gravimetric diluters) are available for adding appropriatequantities of sterile diluent to the food sample. Methodsfor preparing a suspension vary, but ‘stomachers’—peristaltic-type mixers—are most often used. For liquids,shaking by hand often suffices. Other methods includeshaking with glass beads, using a PulsifierTM, or using amechanical mixer. The aim is to achieve a homogeneoussuspension so that the microbes are not present in clumps,and that individual microbial cells are distributed ran-domly in each aliquot of the suspension taken. Ideally,microbes should not be attached to food debris. Extractionof the microbes from swabs is attempted in a similarmanner. Clearly, microbes attached to swabs or largepieces of food, and not in suspension will not be detected inquantitative tests, although in presence/absence tests, theymight still be detectable if the swabs or foods are includedin the enrichment medium.

Sources of error in sampling, therefore, include: (a)the quantity and method of taking the test sample (sincelarge sub-samples and sub-samples composed from differ-ent parts should be more representative of the originalsample); (b) the method of swabbing and material used;(c) the method of homogenisation with diluent; (d) themethod for detaching microbes from swabs or foodparticles; and (e) the composition of the diluent used,which may also be critical, e.g. for analysis of fatty or lowaw foods.

2.2.3.1. Dilution errors. Sources of error include thevariation in volume used—diluent dispensed prior toautoclaving will lose some of its volume due to evapora-tion; further loss will occur during storage. The volume lostwill depend on the type of closure used and will not beconstant, even from apparently identical bottles or tubes(Corry, 1982) although use of fixed autoclave routines willreduce variation (NMKL, 1994). Because the concentra-tion of solutes in the diluent will be increased, any extravolume must be added as water. ISO 6887-1 (1999) specifiesan accuracy of 72% for diluent after autoclaving.

The pipettes used both for dispensing and for subsequentserial dilution introduce a significant source of error (ISO6887-1 (1999) specifies an accuracy of 75%). The use ofserial dilution methods generates a variance that increaseswith the length of the series (Jarvis, 1989; Hedges, 2002,2003).Methods of preparing dilutions have been specified (e.g.

NMKL, 1994; ISO 6887-1, 1999; Collins et al., 2004)but are often not standardized in practice, leading tolarge errors in colony counts. For instance, if a freshpipette, or tip, is not used to prepare each dilution, it hasbeen estimated that an error of up to 1 log10 cfu/gmay result (NMKL, 1994). Failure to mix thoroughlyeach dilution can also result in large errors. Most dilutionsare now carried out using automatic pipettes withplastic tips rather than the traditional glass pipettes. Plasticpipette tips are rarely used plugged, although contamina-tion of automatic pipettes occurs readily (Kolari et al.,1999).Methods of preparing dilutions vary: a suspension may

be mixed by the use of a vortex mixer, or the sample maybe mixed by drawing the fluid up and down several times inthe pipette before the sample is withdrawn (Collins et al.,2004). In the latter case, the possibility of microbesadhering to the walls of the pipette will be minimized.Other variable procedures include touching the tip of thepipette against the side of the diluent bottle after thecontents have been expelled, and whether the end of the tipis dipped into the diluent when the contents are expelled.The volume dispensed by a pipette may be lower when highviscosity diluent is used. Similar considerations apply whensamples are withdrawn from dilutions for plating ontosolid media.

2.3. Techniques

2.3.1. Method robustness (effect of errors in preparation of

media, temperature variation, state of target organism)

The robustness of a method is defined (Lightfoot andMaier, 1998) as the effect of changes in the testingenvironment (physical, chemical and human) on theresult obtained—in other words, their effect on uncer-tainty. We know little about most of these variables exceptwhen, for instance, there has been a gross and obviouserror in preparing the medium, such as omitting anindicator or adding so much selective agent that thepositive control plate shows no colonies. The likelihood ofsuch errors occurring is diminished if dehydrated media areobtained from specialist manufacturers, who have thecapability of thoroughly testing each new lot theymanufacture. This is particularly important because manyingredients of microbiological media are not chemicallydefined (Holbrook, 2000). Nevertheless, it is good labora-tory practice to compare the performance of each new lotof medium purchased against the previous lot, as well aschecking each batch prepared for its selectivity andproductivity with appropriate target and competitor


microbes (Corry et al., 1995, 2003; ISO/TS 11133-2, 2003).It is also worthy of note that a culture medium fromone supplier may not give identical results to that fromanother.

We are not aware of any systematic studies of the effectof relatively small deviations from specified incubationtemperature on results of selective or non-selective colony-counts, and we would expect these to be minor, exceptwhen an elevated temperature is part of the selectivesystem. Microbes in foods are frequently the survivors ofadverse conditions encountered during food processing,e.g. heat, cold, drying, acid, chemicals—and may bedamaged to varying degrees. Sub-lethal damage is wellknown to impair the ability of microbes to multiply in thepresence of selective agents (Mossel and Corry, 1977;Holbrook, 2000; Mackey, 2000; Stephens and Mackey,2003). Methods have sometimes been devised to allow forthis, e.g. ‘preenrichment’ of salmonellas for 16–18 h in anon-selective liquid medium, prior to transfer to a selectiveenrichment medium. An alternative approach is that ofAnderson and Baird-Parker (1975), BSI (1991) forenumeration of E. coli, using a membrane filter placed onthe surface of a non-selective nutrient agar; the inoculum isspread on the filter, and the plate incubated at 37 1C for 4 h.The filter, with inoculum, is then transferred to a selectivemedium and incubated for a further 24 h at 44 1C. Thisprinciple has been used in ISO 16649-1 (2001) and ISO16649-2 (2001) for b-glucuronidase-positive E. coli. In anytests of real foods the extent and range of injury present inthe target population is unknown. Some individual cells areprobably undamaged and therefore unaffected by theselective agents in the medium, others have varying degreesand types of damage that require different conditions forrepair.

Recovery (lag) time for heat-damaged salmonellas inpure culture sometimes exceeds 36 h (Stephens and Mack-ey, 2003). The situation in liquid media, where very lownumbers of target microbes are sought in the presence ofvery high numbers of competitive flora, is much morecomplicated (see discussion in Holbrook, 2000). This isbecause the competitive flora may compete effectively withthe damaged target organisms such that competitorscompletely outnumber targets, even after subsequentselective enrichment and plating. Even apparently non-inhibitory media may inhibit recovery due to the presenceof substances such as peroxide and reducing sugars(Stephens et al., 2000).

2.3.2. Media and diluent preparation

Preparing media is often done by relatively juniormembers of staff with minimal training. This is undesirablebecause errors in media and diluent preparation can havehuge consequences for the results of otherwise carefullycontrolled microbiological examination. A number ofuseful guidelines on this topic should be read even byexperienced microbiologists (NMKL, 1994; Lightfoot andMaier, 1998; CCFRA, 2004a).

It is important to follow carefully the manufacturers’instructions for preparation of their media. Twocommon faults are to overheat media during or aftersterilisation, and to use too large volumes. NMKL (1994)recommends no more than 200ml of agar media or1 l of liquid media, while Oscroft and Corry (1991) andCCFRA (2004a) suggest an upper limit of 1 l for bothtypes. Use of media preparators avoids many of theproblems of overheating encountered with autoclaves, aslong warm-up and cool-down times are eliminated.Another common error is to autoclave media that shouldonly be raised to boiling point before use (e.g. violet redbile (VRB) agar).Particular attention needs to be paid to tempering of

media in water baths. Leaving liquid agar media for 44 hin a tempering water-bath before pouring plates is a causeof heat damage (Oscroft and Corry, 1991). Solid mediawith low pH are particularly sensitive, and should not beheld for more than 1 h in a tempering water-bath becausethe agar gel strength is affected. Using media at too high atemperature for preparing ‘pour’ plates may cause damageto many microbes in the test suspension. Water in a bathloaded with large numbers of hot bottles can take a verylong time to cool to the target temperature of about 45 1C.Selective media should not be prepared in advance andre-melted.

2.3.3. Plating and incubation

Most quantitative methods of examining foods requiremeasured volumes of serial dilutions to be plated—eitherby pour-plating or by surface-plating. Pour plating requirescareful mixing of the inoculum (usually 1ml) with moltentempered agar medium in a Petri dish before the mediumsets.Surface plating requires prepared dishes of agar

with a dry surface. Standard methods usually requirea 0.1ml volume to be dispensed per plate, and subsequentlyspread evenly over the surface prior to incubation. Inpractice, it is usually possible to inoculate 0.2ml perplate, but volumes larger than this may cause difficultiesdue to the surface remaining wet, allowing coalescenceof colonies during incubation. Clearly, there are a numberof possible sources of error in both pour and spreadplating.The effect of drying during incubation is probably

more significant with surface plating, in incubators withcirculated air (preferable in order to prevent temperaturegradients) and at higher temperatures. In order tocompensate for the drying effect of fans that circulateair, some workers use of trays of water to humidifythe air, while others put the plates in open plastic bags. Tothe best of our knowledge, the benefits of these twoprocedures have neither been assessed systematically norcompared.Use of plastic rather than glass Petri dishes enables them

to be incubated in stacks as high as 12 or more. Westwoodand Hodgkinson (1977) showed that agar in plastic plates


in the middle of stacks of ten took about 13.5 h to achievethe target temperature of 37 1C in an incubator withoutfan, compared with 2.7 h for plates at the top or bottom.Plates in an incubator with an air circulator warmed upmuch more rapidly but numbers of colonies of heat-damaged E. coli were significantly greater on the plates inthe middle of the stacks! Peterz (1991) concluded thatwarming of plates in the middle of stacks in excess of sixplates incubated, with or without air circulation, at 44 1Cwas significantly delayed, and bacteria, like Hafnia alvei

(which cannot normally multiply above 37 1C) were able togrow. Temperature variability was greater for plates notenclosed in a plastic bag. They recommended incubation ofstacks of not more than three plates, in open plastic bags,in incubators with air circulation in order to achieve thetarget temperature in less than 2 h. Plastic bags reduce lossof moisture and therefore reduce evaporative cooling.Hence the numbers of organisms recovered in replicatetests could vary widely, particularly if the organisms aresublethally damaged and a selective medium is used,depending on the time taken to achieve the targettemperature of incubation. In many cases, organisms couldbe fortuitously resuscitated.

Containers for incubation of anaerobic and microaero-bic organisms sometimes hold 36 plates in stacks 12-high.Although some containers are made of thin stainless steel,others are made of polypropylene several millimetres thick.It can take many hours for agar in dishes in the centre toreach the target incubation temperature.

By contrast, standard methods usually specify quiteprecisely the length of incubation (normally 72 h).Experience suggests that deviations of several hours fromthe specified time rarely affect the results significantly.However, in certain circumstances—for instance, presump-tive positive colonies on media such as VRB or chromo-genic agars only develop typical colour after a minimumincubation period. In other cases, the colour in the targetcolonies fades with time, and sometimes colonies ofunwanted organisms can develop colour after extendedincubation.

2.3.4. Counting colonies

Traditionally colonies were counted manually. Auto-matic colony-counters are now being used more frequently,especially when spiral platers are used, as manual countingis difficult, slow and tedious. Counting errors in replicatemanual counting can be surprisingly high. Fowler et al.(1978) reported variation between counts done on the sameplates by five different analysts of718%, and variation forrepeated counts of colonies on the same plates by the sameanalyst of 77.7%. Peeler et al. (1982) found counts within75% on 73% of plates for one analyst, and 710% on91% of plates for two analysts. Progress has been made toresolve counting inaccuracies of automatic counters due tocoalescence of colonies and variations in colony size(Wilson, 1995; Corkidi et al., 1998; Marotz et al., 2001).Automatic colony counters for use on some types of

spirally inoculated plates have also been favourablyassessed (Wilson, 1995).

2.3.5. Confirmation techniques—sensitivity, specificity and

selectivity of media

The sensitivity of a medium can be defined as its abilityto support growth of the widest possible range of the targetorganisms, and has sometimes been termed ‘productivity’particularly when applied to general purpose rather thanselective media (Corry et al., 2003). For a particularselective medium, batch or lot, productivity can bedetermined by comparing total counts of populations ofvarious target strains on or in the test medium with thoseobtained on or in a reference medium. Similarly, selectivity

can be quantified by comparing the highest dilution ofcultures of unwanted organisms able to grow on or in anon-selective reference medium with the highest dilutionable to grow in or on the selective test medium. Sensitivity

does not always equate with specificity: the target organismshould not only grow, but its colonies should also showcharacteristic morphology that distinguishes them fromcolonies of other organisms.When comparing presence/absence methods, the sensi-

tivity of a method is defined as the fraction of the ‘‘true’’number of expected positives that is observed. Similarly,the specificity of the method is a fraction of the expectednumber of ‘‘true’’ negatives that is observed (Notermanset al., 1997, cit. Debevere and Uyttendale, 2003).Frequently, such measures are used to compare a new

method against an existing (‘standard’) method, but thenumbers of ‘‘true’’ positives and negatives cannot bedefined with any certainty when using naturally contami-nated samples (especially with numbers of target cells closeto the Limit of Detection (LOD)).The standard method is often assumed to detect the

number of true positives (Feldsine et al., 2002), but thealternative method may be more sensitive than the standardmethod. With cultural methods, it will be possible toverify that presumptive positive results are indeed thetarget organism. However, in the case of PCR- or ELISA-based methods, which are likely to be more sensitive thanthe standard cultural method, there are no isolates toexamine.With standard cultural methods, the result depends on

the ability of the analyst to recognize colonies of the targetorganism. In direct plating methods, the results arenormally expressed either as ‘presumptive’ or as ‘con-firmed’. In some instances, presumptive colonies ofthe target organism have to be distinguished, usually bytheir size and colour, from other colonies. In others, allcolonies growing on a particular selective medium arepresumed to be the target organism. In both casesthere is plenty of scope for inaccuracy in the result (seealso Section 3.2). In presence/absence tests, the analystmay completely fail to spot colonies of the target organism,even though they are present, due to overcrowding bycompetitive organisms.


3. General considerations

3.1. Introduction

Estimation of the uncertainty of a microbial test result(ISO/IEC 17025, 1999, 2005; Horwitz, 2003) is affected bya number of method-related criteria that must beestablished or at least considered before any form ofestimate is applied to results. These are: (a) the sensitivityand specificity of the method in use; (b) inter-relatedexpressions of percentage recovery (‘bias’ in chemistry);and (c) Limit of Detection (LOD) routinely achieved bythe method. In addition to the overall effectiveness of themethod in use, the nature of the sample matrix and thepresence of competitive (non-target) flora will contribute tothe systematic error of the whole process and the result(Niemela, 2002).

For sensitivity, the target group of organisms notdetected (false negatives) by the method, and anyassociated confirmatory tests, should be considered in thefinal expression of uncertainty. Examples of this are theparticular Salmonella serovars missed by the differingtechniques in routine use, or the percentage of E. coli

strains not enumerated by methods using glucuronidase tohighlight typical colonies. Few microbiological methodsare 100% sensitive and they will therefore underestimatethe level of target organisms to varying degrees. The sameapplies to many traditional and novel confirmation testsused as adjuncts to basic methods that identify targetorganisms to an accepted level of probability (Niemi andNiemela, 2001).

Similarly, many methods are not 100% specific and thiscan lead to over-estimation of the target population byincluding false positive results, e.g. due to the similarcolonial morphologies of B. cereus and B. thuringiensis oncertain selective agars. Even the confirmation regime itselfmay be less than 100% specific.

With confirmation tests, the number of presumptivecolonies selected for identification is important. The fewercolonies tested, the more likely are false negative results tobe reported and the greater the uncertainty of the result(Niemi and Niemela, 2001). Most traditional culturemethods for foods require a minimum of five coloniestested, but the probability of the target organism beingcorrectly identified depends on its proportion amongst thepresumptive count (Jarvis, 1989). Standard methods (e.g.ISO) specify calculation of the result based on the fractionof confirmed colonies. Hence, it is essential that theconfirmation is as specific as possible to reduce the riskof introducing a level of uncertainty that will outweigh allother technical uncertainty components (see Section 2.3.5).

The concept of ‘selectivity’ (F) of a microbiologicalmethod was introduced in NMKL (1994) and extended inISO/TR 13843 (2000) to mean the log of the fraction ofpresumptive target colonies amongst the total flora, butfew data are available to establish a criterion foracceptability of methods for this parameter. It should not

be confused with the use of the term ‘selective’ referring tomedia or selectivity factor (SF) applied to media assess-ment (Corry et al., 1995, 2003), although all these terms arerelated. All of the above can affect the LOD (qualitative)or Limit of Determination (quantitative) of any methodand these parameters should be verified (ISO/TS 11133-2,2003) to establish how ‘certain’ the laboratory is that lownumbers of target organisms would actually be isolated bytheir methods.Such over-arching factors as sensitivity, specificity,

recovery and LOD/determination should be included asrelevant, and sometimes substantial, components in un-certainty estimates whatever method is used to preparethem. Niemela (2002) addressed these components andprovided mathematical formulae to deal with some ofthem. He regarded food matrix effects as a component thatmight be determined by complex analytical design butpointed out that no values for matrix variances wereavailable in the literature.Most publications advocate ignoring the distributional

uncertainty of the organisms in the sample, in order toconcentrate on technical uncertainty of the laboratoryoperating the method (EA, 2003). This is rarely, ifever, wholly possible in microbiological examinations assample heterogeneity will be reflected in results fromreplicate sub-samples (Lightfoot et al., 1994; Voyseyand Jewell, 1999). Even if replicates are prepared fromthe same primary suspension as advocated by ISO 6579(2002), there will be an element of sub-sample anddispersion variability in the aliquots tested. This wasparticularly emphasised in the search for low numbers ofindicator organisms in potable waters by the work of Tillettand Lightfoot (1991), but would also apply in othersituations. It is also essential to recognize that anyquantitative statement of precision or uncertainty cannotbe extrapolated to the bulk foodstuff from which thesample was taken (see Introduction).

3.2. Quantitative methods

Most publications on UMM have concentrated ontraditional quantitative enumeration methods by culture,as these are in some ways similar to chemical analyses andamenable to statistical analysis of data. Both the ‘‘bottom-up’’ and the ‘‘top-down’’ approaches have been applied tobasic methods such as Aerobic Colony Counts on variousmatrices (e.g. NMKL, 1999; Voysey and Jewell, 1999;AAG, 2005; Niemela, 2002; Roberts and Greenwood,2003; Augustin and Cartier, 2006).Selective counts require assessment of sensitivity and

specificity (including any confirmation stages) to beconsidered in uncertainty estimates, and only the compo-nent approach of Niemela (2002) has specifically docu-mented these. ‘‘Top-down’’ methods include these effects,but may severely underestimate uncertainty from lessrobust methods if no allowance is made for such factors(ISO/TS 21748, 2002).


The precision of standard enumeration methods hasbeen derived from collaborative trials to validate methodsfor international use and published methods includerepeatability and reproducibility estimates (see Section 4).Similar estimates are also available for food methodsobtained during trials for the current UK FEPASperformance scheme (Scotter, 1996) and for a range ofmilk methods (Dahms and Weiss, 1998). Routine labora-tories can use such data as a basis for their own secondaryvalidation of these methods and to set warning/actionlimits for their own Analytical Quality Control (AQC).To confirm continuing competence, laboratory qualitycontrol based on routine duplicate analyses of similarsamples (type and microbial load) can be performed,and data analysed by an appropriate method for determi-nation of intermediate reproducibility to ensure uncer-tainty remains within the defined limits. It is important tonote that uncertainty can only be assessed reliablyfrom valid results, that is, those within the counting limitsspecified by the methods. Standard methods requirethat counts below or above the optimal range must bereported as estimates, and hence they should not beused in precision analysis. This applies also to zero values.ISO ‘total count’ methods suggest that colonies arecounted on plates containing between 15 and 300 colonies(e.g. ISO 4833, 2003). ISO selective plate count methodssuggest that colonies should be counted on plates contain-ing not more than 150 colonies of presumptive targetorganisms, and no more than 300 colonies in total (e.g. ISO16649-2, 2001).

Most laboratories can estimate repeatability and inter-mediate precision (reproducibility) (EURACHEM, 2000)internally, but true reproducibility data can only beobtained from collaborative trials or performance schemedata. This approach to continuous estimation of uncer-tainty requires all the components of variance to beidentified and minimized by suitable control/monitoring(see Section 2) before the results can be comparedstatistically to derive the estimates. In particular, if resultsobtained from proficiency testing (PT) schemes are used,ideally all should have been obtained using the samemethod, otherwise an over-estimate of uncertainty may beobtained.

Laboratories should establish that their methods canmeet the LOD claimed and are appropriate for thespecification being checked. The same principles wouldapply for estimating uncertainty of results from non-standard methods such as modified Miles and Misra(Corry, 1982). Membrane methods have various applica-tions in microbiological examination including, in thesimplest case, transferring inocula between media as in ISO16649-1 (2001) to provide a resuscitation step and toimprove the LOD of a surface plate to o10. Uncertaintyestimates are again similar, but other sources of error needto be controlled, such as loss of inoculum from themembrane and crowding by non-target strains that canlead to low results. The main application of membranes is

for concentrating low numbers in samples by filtering toretain the organisms on the membrane so that they may becultured on the surface of suitable media or harvested forfurther manipulations. This has many uses in foodmicrobiology, as larger volumes can be analysed to give abetter LOD or provide increased assurance of sterility inbeverage testing. If the organisms are retained for cultureon the membranes, uncertainty estimates can be preparedin the same way as other counting methods withconsideration of any added errors associated with use ofmembranes, and recording results as estimates if numbersbelow the limits for valid results are obtained (ISO 8199,2005). If the organisms are harvested, then recovery is amajor component to be included.Quantitative estimation of microbial loads on surfaces is

common. As discussed earlier the swab matrix can retainorganisms, leading to reduced recovery, and thereforecontribute a large component of bias. Contact plates forhygiene assessments may give even lower counts thantechniques that remove the organisms from the surface andthereby break up the micro-colonies of the (probable)contagious distribution of the organisms on the surface.This may also be a factor with dip slides used to estimatethe microbial load of liquids and these also can show largevariations in counts depending on the volume of the liquidretained by the slide.All count procedures should be fit for their intended

purpose and the user laboratory must ensure that anappropriate method is chosen for the analysis to ensurethat valid results are obtained (ISO/IEC 17025, 1999,2005). This has two aspects: testing of suitable dilutions toobtain counts within the limits specified by the method;and using a method that will provide the LOD required bythe specification or other guideline for the test product.Experience of the particular product or data from sourcessuch as ICMSF (2002) should indicate the required rangeof dilutions. The laboratory should ensure that the desiredLOD is achievable with the method and not rely ontheoretical LODs assessed simply from the inoculumvolume.

3.3. Semi-quantitative methods

Some of the enumeration techniques discussed aboveare described as ‘‘screening’’ or QC methods and arenot in use as standard or reference methods since theirprecision is low and they may be very susceptible to bias.However, they may be fit for the intended purposeand uncertainty estimates can be produced, although thesewill be larger than those obtained from more robustmethods.Current standard methods include those based on the

imprecise MPN technique (Woodward, 1957; de Man,1983), for example ISO 7521 (2005) for E. coli in foods, forwhich uncertainty estimates are required. Such dilutioncounting methods are fundamentally sets of individualpresence/absence tests where the likelihood of growth in


each test depends on the concentration of the target in theinoculum. The proportion of positive and negative tests isrecorded and the result obtained from tables or computerprogrammes (e.g. de Man, 1983). Tables published instandard methods include confidence limits that show onlythe influence of statistical variation on the results, and alsofour categories of results that indicate the probability ofthis result being achieved. These categories are useful tohighlight possible technical errors that may invalidate theresults.

The best precision from these methods is obtained whenthe numbers of positive and negative results are approxi-mately equal (Cochran, 1950), so choice of the optimalvolumes and dilutions for each sample is again important.Logically, the greater the number of ‘‘tests’’ that areperformed, the greater the precision, and this is the basis ofrecently introduced miniaturized (ISO 9308-1, 2000) ormulti-well (SCA, 2002) MPN methods for E. coli and otherorganisms.

Whilst variations in, and the errors associated with,limiting dilution series and MPN tests have been discussedby many authors (e.g. Swaroop, 1951; Woodward, 1957;Jarvis, 1989), little consideration has been given toestimating uncertainty apart from the component ap-proach adopted by NMKL (1999) and Niemela (2002).

Many of the variance components associated withconventional culture apply also to semi-quantitativemethods and should be controlled before estimates areattempted. A naıve view may be to assume an acceptablelevel of uncertainty as the 95% confidence interval fromthe published tables, qualified as discussed above, in orderto demonstrate that results are within an acceptablerange of statistical variation and do not show grosstechnical errors. Niemela (2002) extended this approachand offered two further methods for estimating relativestandard uncertainty that produced similar though notidentical figures for a ‘single dilution MPN’. For multipledilution series, he concluded that confidence limitsconstitute an appropriate expression of the uncertainty ofthe MPN or the final suspension; however, other compo-nents, especially the uncertainty associated with thedilution should be added to obtain a combined uncertaintyof the result.

A similar approach might be taken for other situationswhere test results are considered semi-quantitative; forexample enumeration methods where colony numbers,below those considered valid (i.e. 30 in ISO/TR 13843,2000), are sought or isolated. This would apply to mostenumeration procedures discussed above, e.g. for countsbelow 15 or 30 colonies/plate or membrane filtrationresults below 10 colonies (ISO 8199, 2005). Such results canbe reported as estimates, together with the 95% confidenceintervals. According to ISO/TR 13843 (2000), results below10 colonies/plate are regarded as so imprecise that they canhardly be characterized as better than semi-quantitative,and when o3 colonies are detected all methods becomepresence/absence tests.

The opposite situation, colonies so crowded as toprovide only an estimate of numbers, usually occursbecause insufficient dilutions of a sample have beenprepared. Carabin et al. (2001) discussed assessment ofimprecise results when comparing methods for faecalcoliforms. Such estimates may be acceptable in routinetesting, as the sample will probably have failed thespecification.

3.4. Qualitative methods

All qualitative tests are based on whether or not thetarget organism (usually a pathogen) is detected. Mostpublications have specifically excluded consideration ofthese tests, or dealt with them only from the perspective ofhow many tests should be performed to give a certainprobability of isolating the target organism (ICMSF,2002). However, statistics based on those for MPNuncertainty were advocated by NMKL (1999), and Niemiand Niemela (2001) classified presence/absence testing as‘‘a one-tube detector’’ system.What is uncertain in qualitative terms, however, is

whether a particular laboratory using a particular techni-que would isolate any target organisms present. Thisdepends on the sensitivity, specificity, LOD and percentagerecovery of the method in use and these characteristicsshould be defined and included in the documentedmethod. Another very relevant factor is the size and typeof the sub-sample taken to represent the whole sample—generally the larger the sub-sample the greater the chanceof detection, although pathogens are rarely distributedhomogeneously. Examination of replicate random foodsamples is generally recommended (ICMSF, 2002). Thetype of sub-sample and the protocol for obtaining the testportion is of vital importance, e.g. a carcass-rinse techniqueis much more likely to isolate pathogens from poultry thanexcision of breast muscle from under the skin (Anon,1979).Little has been written about uncertainty estimates

for qualitative test results as these have been regarded asin the category that ‘‘preclude rigorous, metrological andstatistically valid calculation of uncertainty of measure-ment’’ (ISO/IEC 17025, 1999, 2005). However, thatstandard requires definition and control of those compo-nents that affect the uncertainty of test results. Fulldefinition of the methods is required in the case oftraditional enrichment techniques, and modern innova-tions, for example, the immunomagnetic separation(IMS) technique, multi-well ELISA methods, DNA probesand PCR methods. These can either improve thedetection limit by capturing the target organism usingantigenic or nucleic acid structures, improve specificity,reduce the time to a result, or screen out negative samples.All need performance definition, including detection limits,in the same terms as traditional culture methods. Howsensitive is the method? Does use of method ISO 6579(2002) isolate all serotypes of salmonella and do semi-solid


methods (De Smedt et al., 1986) show poor sensitivity, asnon-motile species may be missed? How specific is themethod? Does use of IMS for isolating E. coli O157 givefalse positive results with E. hermanii? Most enrichmentmethods should give 100% recovery, especially those witha resuscitation/preenrichment step to allow repair ofdamaged organisms, but what is the LOD of the methodin the hands of the user laboratory? Theoretically, thisshould be one organism in the sub-sample examined, butfactors such as overgrowth by competitive flora can alterthe LOD, and this can occur by mischance, or due tomodifications of the method. Examples of modificationsthat might result in failure to detect low numbers of thetarget organism include economising by use of half or evenquarter plates of expensive media for plating out (e.g.salmonella on Rambach agar), speeding up procedures byreducing enrichment times or even omitting a preen-richment step.

Langton et al. (2002) introduced a procedure forevaluation of collaborative trial results for qualitativemethods, based on new concepts of Accordance andConcordance, that have been included as part of theprotocol for validation of alternative methods in ISO 16140(2003). The procedure was further refined by van der Voetand van Raamsdonk (2004), but is merely a different wayof comparing the data from tests replicated in differentlaboratories and is not considered to assist in the derivationof an estimate of uncertainty. Information on the levels oftarget (and competitive) organisms in the samples must beconsidered, as concentrations around the LOD of themethod/laboratory are likely to be more inconsistent.Hitchins and Burney (2004) have proposed the use of theSpearman Karber analysis to derive LD50 values, fromwhich uncertainty parameters may be estimated, as analternative basis for assessing uncertainty of qualitativetests. This, and related procedures using probit, logitand other statistical techniques are currently beingevaluated by ISO.

3.5. Alternative methods

Many methods for rapid detection, enumeration andcharacterisation of micro-organisms are available whichcould be described as alternatives to the traditional,culture-based techniques. Full descriptions of such meth-ods can be obtained elsewhere, but performance character-istics and estimating uncertainty of results will be brieflydiscussed. Many are regarded as modifications of (e.g.miniaturized or defined substrate MPN) or adjuncts to (e.g.rapid identification kits or latex-based serology) traditionaltechniques and, as such, their associated uncertainty can beassessed by the same methods, or regarded as one of thecomponents of the main technique.

The reviews of van der Zee and Huis in’t Velt (1997), DeBoer and Beumer (1999), Olsen (2000), Reichmuth andSuhren (1996) and Suhren and Reichmuth (2000) provideexamples of precision data, such as specificity and

sensitivity/detection limits and repeatability (where appro-priate), which can be used in estimating uncertaintycomponents.Jarvis (1989) reviewed sources of error in alternative

methods available at that time. As the indirect measure-ments involved in many of the techniques require correla-tion of the property measured to the number of microbespresent in order to provide a result, he noted this as anadditional component to be considered and controlled forsuch methods. He concluded that the detectors providednormally distributed results and functioned well with purecultures, but showed greater variance due to over-disper-sion of organisms in actual food samples (but this effectalso holds for results using traditional cultural methods).The matrix effects influence the uncertainty of test resultsand this is still a problem for many alternative techniques,as stated by Olsen (2000) when discussing why DNA-basedtechniques had not achieved widespread application in thefood industry.Microscopy can be used for direct cell counts for liquids

or semi-solid foods, but have major sources of error: inbreaking up clumps while preparing samples; inaccuratedispensing of small volumes; high multiplication toextrapolate a result; failure to distinguish non-viable cells,and above all operator subjectivity (Jarvis, 1977). Theseerrors may result in relative standard deviations (RSDs) of55% or more (Jarvis, 1989). Fluorescent staining orfluorescent antibody methods can improve cell visualisa-tion, distinguish non-viable organisms, can be automated,and have removed some sources of error. Boisen et al.(1992) found direct epifluorescent filter technique (DEFT)to give results equivalent to the standard plate count andReichmuth and Suhren (1996) reported acceptable repeat-ability with raw milks for both DEFT and automatedfluorescence techniques. Uncertainty estimates for suchmethods should include definition of sensitivity andspecificity of the stains or antibodies to include any bias,in addition to precision data collected from replicatecounts.Measurement of physical attributes of microbial suspen-

sions in liquids by turbidimetry for higher levels (e.g. forstandardising test suspensions in disinfectant testing) andcoulter counter or flow cytometry for lower levels orspecific organisms, requires calibration data to ensure theunits of measurement are correlated accurately to cellnumbers and avoid any bias. The sensitivity of flowcytometry is high, with 100 yeast cells and 100–1000bacteria detected in liquid samples (van der Zee and Huisin’t Velt, 1997), but food matrices can interfere andthe system does not distinguish dead cells unless bior-eagents are used to label target organisms. However,Rattanasomboon et al. (1999), using flow cytometryto count Brochothrix thermosphacta in meat products,showed good correlation with other methods. Goodcalibration is also a requirement of screening systems thatmeasure metabolic activity of target organisms in cultureby impediomentry, and such systems have been discussed


by Gibson et al. (1992) for salmonella detection, Gibsonand Ogden (1997) in the context of assessing compliancewith specifications, and Glassmoyer and Russell (2001)for Staph. aureus detection. Again, specificity and sensitiv-ity/detection limit definition is required, as well ascorrelation of the detection times to the initial inoculum,if estimates of the numbers of bacteria are requiredrather than just presence/absence rapid screening. Glass-moyer and Russell (2001) showed the Staph. aureus

detection limit to be as low as 10 cfu/ml, but cautionedagainst using the system for enumeration amongst mixedflora without further modifications. Reichmuth andSuhren (1996) noted varying precision for quantitativeassessment on raw milks that depended on the instrumentand methods used.

Bioluminescence ATP detection systems are used in thefood industry as rapid indicators of hygienic conditions.Colquhoun et al. (1998) described a test protocol toevaluate three commercially available hygiene-monitoringsystems and ranked sensitivity (detection limit) andreproducibility data when checking three types of foodsoil that affected the performance of the devices differently.Reichmuth and Suhren (1996) evaluated four such methodsfor raw milk quality assessment, and found that althoughrepeatability was good and the ‘‘accuracy’’ when comparedwith a reference method was adequate, the systemsdetected different strains of the trial flora to differentextents.

Alternative immunological techniques vary in theircomplexity and purpose from simple latex visualisationof serological confirmation tests, IMS to enhance detectionof specific pathogens and ELISA to shorten the time takenfor detection of pathogens by traditional enrichmenttechniques. Sowers et al. (1996) evaluated commerciallatex reagents for O157 and H7 antigens of E. coli bycomparing the sensitivity and specificity of this modifica-tion; such data can be used to show the uncertainty ofthe confirmation step in estimates for results of manydetection tests.

IMS is routinely used to lower the detection limit andreduce the time to a result for traditional enrichment andcan be coupled with other systems such as electrochemicaldetection to give a result in 2 h (Perez et al., 1998; Cheet al., 1999). ELISA methods for pathogen detection areused to produce results more rapidly and to screen outnegative results as an adjunct to enrichment methods.Commercial kits are available for many pathogens andtheir toxins (Novicki et al., 2000; Made et al., 2004) and, asqualitative tests, are defined in terms of specificity andsensitivity. Detection limits range from 103 to 105 cfu/ml(De Boer and Beumer, 1999) and initial enrichment istherefore required.

DNA-based techniques for detecting pathogens have notyet been widely applied in routine food microbiology dueto problems in separating target organisms from foods, andthe need to improve detection limits by using an enrich-ment step.

4. Data for method validation and control

This section focuses on sources of suitable data andmaterials by discussing method validation, referencematerials and AQC.

4.1. Method validation

Valid microbiological methods should be shown to be fitfor their intended purpose and have established perfor-mance characteristics across the full range of the foods tobe tested, with due consideration of the effects of differentlevels of the target organisms. This concept has long beenaccepted in analytical chemistry (see for example, Youdenand Steiner, 1975), where principles for collaborative trialsto validate methods were harmonized into a protocol byIUPAC (1988, 1995). To some extent these have beenapplied to microbiological methods with acknowledgmentof different requirements, e.g. sample preparation (An-drews, 1987). Early microbiological work on methodcomparisons with consideration of precision characteristicswas carried out predominantly on ‘‘total’’ colony counts byculture or modifications of that basic technique (Hedgeset al., 1978; Kramer and Gilbert, 1978; Silliker et al., 1979).Formal international validation of microbiological

methods was accelerated in the 1990s, driven primarily bythe prescription of methods by European legislation (EEC,1985, 1993a, 2005; Scotter and Wood, 1996), and also bythe desire to use methods variously described as ‘new’,‘rapid’ and ‘routine’ (Lombard et al., 1996; IDF 161A,1995) as alternatives (now the accepted term) to standard(normative) or reference methods (Andrews, 1996, 1997;Anon, 1998; Ogden et al., 1998; Davey, 2001). Standardand reference methods are often perceived as laborious,expensive and time-consuming (Rentenaar, 1996) and theprocess for their development is too rigid and slow(Hitchins, 1996; Leclercq et al., 2000). The alternativemethods must be validated (ICMSF, 2002) as fit forpurpose, and ISO 16140 (2003) has been developed toaddress this need for both quantitative and qualitativemethods. Harmonisation of validation and acceptableprotocols is also desirable, but is proving difficult toachieve amongst the diverse bodies formulating micro-biological methods (Andrews, 1996; Hitchins, 1996; Jack-son and Wachsmuth, 1996). Certification schemes forvalidated alternative methods have been introduced in theEU (MicroVal) and the UK (EMMAS—Scotter andWood, 1996). Validation for non-standard or ‘in-house’methods used by accredited laboratories is also required byISO/IEC 17025 (1999, 2005).The fact that many standard and reference methods had

not been formally validated (Fleet, 1996) has also beenaddressed (Scotter et al., 1993; Lahellec, 1998) andperformance data for quantitative methods are nowbecoming available (Schulten et al., 2000; Scotter et al.,2001; Leuschner et al., 2003). These data illustrateachievable performance characteristics such as repeatability


and reproducibility (ISO 7937, 2004) and critical RD (DeSmedt, 1998) and can also highlight deficiencies in methodperformance, for example the poor specificity of ISO 11290-1, 1996; ISO 11290-2, 1998) for L. monocytogenes in thepresence of L. innocua (Scotter et al., 2000). Such workjustifies and informs future work on method developmentand can be used by laboratories in estimates of uncertaintyof their own results.

Similarly, work on seven ‘routine’ colony count culturemethods and 10 ‘alternative’ milk count procedures byReichmuth and Suhren (1996) established repeatabilityvalues for 15 methods. IDF validation (Hitchins, 1996)often involves ‘routine’, simple and affordable methods insupport of the dairy industry, but some harmonisation withISO and CEN is being achieved where appropriate.

Early work to validate some methods now regarded assemi-quantitative, demonstrated very poor precision. Forexample, Woodward (1957) showed three-tube MPNmethods to have 95% confidence ranges as large as1.27 log10 and Silliker et al. (1979) showed the ICMSFcoliform MPN method range to be 1.85 log10 with a 95%confidence interval for a single laboratory of 71 log10cycle. Salo et al. (2000) presented collaborative studyresults on surface hygiene control, emphasising the need toacknowledge the demonstrably poor recoveries of surfacetechniques such as dip slides, contact plates and swabs(16–30%, with poorest recoveries at the highest contam-ination levels). The contact plate appeared to give the bestrepeatability and reproducibility values, but all methodsshowed poor precision.

4.2. Reference materials and proficiency testing

Apart from uncertainty data derived from collaborativetrials and method validation, use of reference materials isanother way of assessing method performance. Suchmaterials have proved far more difficult to produce in astable form for microbiological examinations than forchemical analyses (Peterz and Steneryd, 1993).

Distribution of fresh, contaminated food samples posesa number of problems, not least the stability of suchsamples over time and distance, although Vivegnis et al.(1997) showed that naturally contaminated refrigeratedfoods could be used with some success. Nevertheless, mostwork has been to develop freeze-dried simulated foodsamples for PT (Peterz and Norberg, 1983, 1986; Peterz,1992). Encapsulated contaminated milk powders have beenused for qualitative salmonella/listeria testing (e.g. In’tVeld and De Boer, 1991). Although these materials are notfully stable, their decline can be followed using controlcharts and, on this basis, some milk capsule CRMs areavailable from the European Community Bureau ofReference (In’t Veld, 1998).

Stable samples have been developed for food examina-tions and proficiency schemes in the UK (Scotter, 1996).Derivatives from national reference culture collections arealso available from various commercial organisations, but

these are generally not quantified and are not within thedefinition of reference materials used here, although theyhave considerable use in laboratory internal qualityassurance.The design and rationale of PT schemes has pulled in

two directions: the need to establish precision character-istics with stable and quantified material; and the some-times conflicting need to reflect ‘reality’, especially inclinical microbiology (Isenberg and D’Amato, 1996; Salkinet al., 1997). The artificialities, and difficulties of statisticalinterpretation, for microbiological performance testinghave been highlighted by various authors (Voysey andJewell, 1999; Tillett et al., 2000; Jewell, 2001) and werediscussed at a Eurachem/EQALM Workshop reported byOrnemark et al. (2001). A wide-ranging review of PTschemes has been published by Augustin and Cartier(2006).Publication of ISO/IEC Guide 43 (1997) and ILAC

(2000) have resulted in improved harmonisation. Compe-tence guidelines for providers have been set and eightscheme providers across a range of testing are currentlyaccredited including Lenticules (Lightfoot et al., 2001) andthe ‘‘BioBallTM’’ for which a very high degree of withinand between batch precisions is claimed (Vesey, 2005).Nevertheless, topics remain to be addressed for microbiol-ogy, especially the conflict between PT samples and reality,and the differing statistical approaches used to assessdata and produce the ‘assigned’ value. The ‘reality’question is valid for microbiology, but performance testingremains useful provided it is fully understood thatprecision data from reference samples are likely to bebetter than that achieved with real samples, due to greaterhomogeneity, and because PT samples contain laboratory-attenuated strains. This is illustrated by the data of Scotteret al. (2001), where method ISO 11290-2 (1998) gaveresults (as log10 cfu) with sr ¼ 0:34, sR ¼ 0:51 for referencematerials, but sr ¼ 0:58, sR ¼ 0:81 for natural foodsamples.Use of reference materials, including those from PT

schemes, is an essential requirement for accredited andOfficial Control microbiology laboratories (EEC, 1993b;ISO/IEC 17025, 1999, 2005; Dveyrin et al., 2001) as biaseffects cannot be assessed and eliminated in any other way(Havelaar et al., 1993). Precision data on performance of amethod in the user laboratory can also be gleaned andadded to data from other sources.

4.3. Analytical quality control

Uncertainty estimations for microbiological resultscannot be achieved unless the full range of AQCmeasures, both internal and external, are in place inthe laboratory, to limit variability, demonstrate thatmethods are ‘in control’, and provide precision andbias data to establish and continually to update uncertaintyestimates. Relevant aspects of Guidance on AQC for wateranalysis (ISO 13530, 1997) and Part 3 of SCA (2002) may


be adaptable for food microbiological methods. Snell(1991) reviewed AQC in public health microbiology andHPA (2005) lists sources of uncertainty data from AQCprocedures.

The components (or sources) of uncertainty have beendiscussed in Sections 2 and 3 in the context of the methodsand techniques. Ongoing control and monitoring of suchcomponents to minimize errors is a major factor ofassessment for accreditation to ensure satisfactory techni-cal performance (ISO/IEC 17025, 1999, 2005). Manycontrol measures are common to all microbiologicalmethods, but some are dependant on the requirements ofindividual methods or techniques. Many guidelines havebeen published dealing either with the collective aspects(NMKL, 1994, 1999) or with individual components suchas media QC (Corry et al., 1995, 2003; ISO 11133-1, 2000;ISO/TS 11133-2, 2003) or reliability of counting procedures(Fruin et al., 1977; Fowler et al., 1978; Niemela, 1983;Muller and Hildebrandt, 1990; ISO 14461-1, 2005; ISO14461-2, 2005).

Internal control of methods is usually achieved byregularly incorporating IQC samples into routine workwith a frequency depending on the degree of assurancedesired. This is a procedure allowing effectiveness ofcontrol to be monitored using Shewhart or other statisticalprocess control charts (e.g. Beauregard et al., 1992)perhaps automatically compiled by a LIMS system inlarger laboratories. This approach can be useful forquantitative microbiology methods where the targetorganisms are known to be present in reasonable numbers,but is less informative for P/A tests and those whereinherently variable low level or zero results are the norm.However, alternative control chart systems can be applied.For instance, Lee and Cole (1994) used frozen samplesspiked at an appropriate low level that were stable for 3months (Donnison et al., 1993) and constructed charts withcontrol limits for the test materials.

Lightfoot et al. (1994) explored the use of duplicate splitsamples for IQC of water microbiology where low andvariable counts are expected. This allowed the productionof control charts using the 95% CI of the first count and acheck that the second count was included in this interval ona minimum of 19 of 20 occasions. This approach assessesresults only on the variation of the target organisms in thesample and masks technical errors in the laboratory unlessthese are gross (BSI, 2003). Niemela (1996) proposed asimple precision control criterion ‘Q’ for duplicate countsthat again produced control limits and steering diagrams(NMKL, 1999), taking in the distributional (sample)uncertainty, but also adding a factor for technicaluncertainty of 77% based on previous estimations byJarvis (1989).

A further level of control, and the only measure whichcan assess bias or systematic error in results, is the use ofCRMs, other reference materials or PT scheme samples asdiscussed in Section 4.2. Such materials should be morestable and reliable than internally prepared spiked samples,

but are expensive. However, the performance of eachmethod should be assessed on a cumulative basis toidentify trends and indicate any bias (ISO/IEC 17025,1999, 2005; Tillett and Lightfoot, 1995).Some quantitative PT schemes have been criticised

(Tillett et al., 1993, 2000; Ornemark et al., 2001; Jewell,2001) because the continuous assessment of performanceby individual laboratories is difficult. The criticismsfocussed on how the results were assessed: firstly, howthe ‘assigned’ value for the PT sample is established; andsecondly the differences in and fitness for purpose of thestatistical approaches used to assess scheme results.Statistical approaches for food PT scheme resultswere discussed by Peterz (1992) and for water schemes byTillett and Lightfoot (1991) and Tillett et al. (1993, 2000).Some degree of harmonisation has been achieved foranalytical chemistry (IUPAC, 1993, 1995; ISO/IEC Guide43, 1997; ILAC, 2000). These procedures recommendderivation of z-scores that determine the differencesbetween the data of each participant and an analyticallyderived ‘‘true’’ value in relation to a specified standarddeviation for the test. z-values can easily be plottedto show trends in laboratory results. For some years, thisapproach has been adopted in microbiological proficiencyschemes such as FEPASTM (Food Examination Perfor-mance Assessment Scheme, Central Science Laboratory,York, UK) that uses both an assigned reference valuefor the microbiological data and an assigned standarddeviation.Trend analysis of presence/absence data is more difficult

and should be combined with consideration of the leveland characteristics of the target organisms—a very lowlevel or an unusual strain may not be detected if themethod used is insufficiently sensitive and specific and notproven to have an adequate LOD. Early work onassessment of cumulative data from clinical PT schemes(Tillett and Crone, 1976) used a binary (1 ¼ correct,2 ¼ incorrect) system to assess (Cochran, 1950) laboratoryperformance. The accordance/concordance approach mayhave applications for assessing trends of qualitative data inPT schemes (ISO 16140, 2003).

5. Approaches to the determination of uncertainty

Reproducibility values for data produced using astandard protocol can provide a means of assessing theaccuracy, trueness and precision of a method when used intwo or more laboratories. Intermediate reproducibility canbe determined by producing data within a single labora-tory. Definitions of repeatability, reproducibility and therelevant test conditions are given in Appendix A. Datarequired to generate these parameters are obtainedprimarily through carefully controlled intra- or inter-laboratory trials, although alternative approaches areavailable for the assessment of repeatability on an ongoingbasis within a laboratory.


5.1. Organisation of inter-laboratory trials

The primary purpose of a collaborative trial is togenerate data for ANOVA. A trial involves a definednumber of analysts in each of 15 or more laboratories.Each analyst should examine an agreed number of replicatesamples; ideally the primary dilution of each sample shouldalso be tested in duplicate. Such an experimental designrequires that the replicate samples are prepared in a singlecoordinating laboratory that also specifies in detail the testprotocol to be followed in the collaborating laboratories.Ideally, the test should be structured as a fully nesteddesign where equal numbers of samples are examined by anequal number of analysts in each laboratory; where this isnot possible other experimental designs (e.g. a partiallynested trial) can be used—but this adds to the complexityof the statistical analysis. Details of nested and otherdesigns are described in detail in ISO 5725-3 (1994). These,and other related, ISO standards provide information onthe precautions to be taken in setting up such trials, theway in which the data are to be assessed and the statisticalanalyses required.

5.1.1. Basic requirements for microbiological collaborative

trials (ISO 16140, 2003)

Laboratories selected to participate in a trial should beable to demonstrate an appropriate level of competence(e.g. use ‘‘accredited’’ laboratories). The trial coordinatorneeds to consider whether:

(i)
The test matrix will be a real food with naturallyoccurring organisms, a ‘‘sterile’’ food to which astandard inoculum of organisms will be added, or justa standard suspension (fresh or freeze dried) of testorganisms.
(ii)
The ‘‘target organisms’’ will be a pure or a mixedculture inoculum, whether the test organisms be wild-type isolates or culture collection strains, and whetherthe response of the organisms is known for theparticular circumstances to be used in the test.
(iii)
The concentration of organisms to be used should beat a level found in the food materials. In the case ofpresence or absence tests the level of organisms shouldreflect both the target ‘‘Level of Detection’’ (LOD) anda higher abuse level. Some standard protocols requirethat methods be tested at more than two levels oforganism.
(iv)
The choice of diluents and culture media is critical. Ifso, they will need to be specified (possibly evenincluding the manufacturer of the medium).
(v)
The method of examination will need to be defined,including handling of the initial samples, preparationof dilutions, plating or other inoculation procedures,incubation conditions, etc. Participating laboratoriesshould be required to provide information onthe calibration, monitoring and tolerances of theirequipment.
(vi)
In order to minimize any risk of confusion, partici-pants must be provided with a standard form on whichto record all their data and observations. In all cases,participants must provide the raw data (e.g. colonycounts on each replicate dilution; presence of a targetorganism, etc.), in order to ensure that the coordinat-ing laboratory can carefully check the data supplied.
5.2. Preliminary evaluation of quantitative data

For quantitative trials, derivation of colony count levelsmay be done using either simple or weighted means but it isrecommended that the G2 test for homogeneity of replicatetest data should be done (ISO 14461-1, 2005, part 1).Individual colony count data sets must be inspected for‘‘non-conforming’’ results (i.e. outliers). Non-conformingdata may arise as a result of faulty samples, poorlaboratory practices, including inaccurate recording ofthe raw data, or failure to conform to the protocol. Whereit is suspected that data may have been wrongly recorded,the coordinator of the trial must refer back to thelaboratory concerned to assess whether the data can becorrected.

5.3. Analysis of variance (ANOVA)

The standard ANOVA is based on the concept that thedata to be analysed conform at least reasonably to anormal distribution. Colony count data therefore need tobe transformed before subjecting the data to analysis.Routinely, the assumption is made that colony counts willconform approximately to a log10-normal distribution, butif conformation with another distribution is suspected (e.g.random or negative binomial distributions) then alternativeforms of transformation are necessary. After transforma-tion, but before doing a standard ANOVA, it is essential totest for outliers using the methods of Horwitz (1995) orYouden and Steiner (1975). The ANOVA should be set upas a one-way analysis with nesting, depending upon thenumber of variables (i.e. laboratories, analysts, samplesand replicates). Generally, interactions between thefactors are not evaluated. The residual mean square givesthe estimate of repeatability standard deviation (sr)whilst the reproducibility standard deviation (sR) isderived from the combined estimates of the componentsof variance derived for analysts and laboratories, i.e.

sR ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiðs2r þ s2Lab þ s2AnÞ

q, where s2r is the residual (i.e.

between-replicate) variance, and s2Lab, s2An are the variances

associated with the laboratory and analyst components,respectively. For details of the derivation of componentvariances see ISO 5725-3 (1994).

5.4. Alternative approaches to ANOVA

Increasingly, robust methods of analysis are beingapplied, since such methods do not require the elimination


of outlier data. The consideration is that ‘‘all data (otherthan simple computational errors) have validity’’ andoutliers merely indicate the possibility that extreme valuescan be encountered in real life situations. Various differentapproaches to robust ANOVA have been proposed andsome are included in standard methods (see for instance,ISO 5725-3, 1994; ISO 5725-5, 1998; ISO 16140, 2003). Themost commonly used method is that described in ISO 5725-5 (1998) and by AMC (2001)—a spreadsheet procedurefor this method is available for downloading.1 Analternative approach is the Recursive Median (Remedian)described in ISO 16140 (2003). Unfortunately, whilst therobust methods provide estimates of repeatability andreproducibility standard deviations, they do not provideany estimate for the components of variability. A simpleapproach to derive these values is given by Hedges andJarvis (2006).

5.5. Determination of repeatability and reproducibility using

quality monitoring (QM) data

A simple method for determination of repeatability andreproducibility using data generated by comparativeanalysis of paired samples is published in ISO 19036(2005) and is based on the following equation:

sR ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXn

i¼1

ðyiA � yiBÞ2

2n

s,

where yiA and yiB are the log10-transformed results ofparallel tests on a sample, and I is the number of samplesfrom 1 to n. The repeatability estimate reflects tests done onnumerous samples over a time period by a single analyst ina single laboratory. The internal (or intermediate) estimateof reproducibility is obtained by analysis of data derived bydifferent analysts operating within a single laboratory.Jarvis et al. (2004, 2005) have reported estimates ofvariability derived using these approaches. A spreadsheetprocedure has been published by AAG (2005).

5.6. Determination of uncertainty from reproducibility

estimates

Estimates of uncertainty relate to data produced using aspecific method of analysis but not to the method per se.Estimates of expanded uncertainty (U) are obtained bymultiplying the reproducibility standard deviation by acoverage factor (k) chosen to indicate that a result will notexceed or be less than a desired approximate statisticalprobability. Hence for a 95% probability, U ¼ 2sR, whilstfor a 99% probability U ¼ 3sR. In reality this is nodifferent from saying that an approximate 95% confidenceinterval demonstrates that a mean result (y) will lie withinthe range y� 2sR.

1/http://www.rsc.org/Membership/Networking/InterestGroups/Analytical/

AMC/Software/RobustStatistics.aspS

5.7. Uncertainty associated with qualitative tests

‘‘Presence or absence’’ data are not readily amenable tostatistical analysis to derived reproducibility or repeatability.However, some tests can be used to assess the extent towhich a result may be repeatable. For instance, if a series ofparallel trials using a defined test procedure generates anoverall average of (say) 50% positive values, then it ispossible to derive an estimate of the standard error of thatresult, based on the binomial distribution. For instance, if 20replicate samples were tested, then the CI associated with anoverall result of 50% (10 positives from 20 samples) rangesfrom 27.2% to 72.8%, whereas if 60 samples had been testedwith 30 positives, the CI would range from 36.8 % to63.2%. It is important to note that such limits are dependantonly on the total number of samples tested and not thenumber of laboratories involved. It is also possible to derivea ‘‘most probable value’’ even at a replicated singleinoculation level where at least one positive result isobtained, or to derive a ‘‘maximum level of contamination’’when all results at a single level of inoculation are negative(for details see, Jarvis, 1989).If tests are done at more than one inoculation level, then

an MPN value can be derived. Possibly more useful is theconcept of determination of the LOD associated with 50%positive results (LOD50) as described by Hitchins andBurney (2004) who used the Spearman–Karber method ofanalysis. Other alternative forms of analysis (e.g. probitand logit) are currently under investigation through theStatistics Working Party of ISO Committee TC34/SC9.

6. Recommendations to minimize uncertainty

Control of uncertainty is not totally in the hands of themicrobiologist since bias factors are inherent in allmethodology—but many sources of error and bias arecontrollable. Key issues are:

6.1. Personnel

There is no substitute for effective training in laboratorypractice, both on engagement and on a continuing basis.Untrained staff should never be used for critical work.

6.2. Equipment

All equipment must be properly maintained and used.Proper calibration of equipment, including incubator,water bath and autoclave temperatures, is essential. Thelaboratory should know the tolerances applicable tobalances, pipettes, etc., and should check them on aregular basis.

6.3. Diluents and media

All media and diluents must be prepared correctly,including careful attention to dispensing, sterilisation and

http://www.rsc.org/Membership/Networking/InterestGroups/Analytical/AMC/Software/RobustStatistics.asp

http://www.rsc.org/Membership/Networking/InterestGroups/Analytical/AMC/Software/RobustStatistics.asp


tempering prior to use. The surface of agar plates forsurface inoculation must be carefully dried to minimize anyrisk of over-drying or microbial contamination.

6.4. Incubation

Agars must be prevented from drying out and the correcttemperature of incubation achieved rapidly (see Section 2).

6.5. Primary sampling

When taking a primary sample is part of the analyticalprotocol, it is essential to ensure that the sample is takenrandomly unless it represents a particular part of a lot (e.g.the first samples from a packaging line).

6.6. Analytical (test) sample

Unless there is a good reason to select specific parts froma multi-component food, ensure that the entire sample isthoroughly mixed before taking a subsample for analysis.Ensure that a sufficiently large subsample is taken in orderto minimize the vagaries of microbial distribution—forinstance, it is preferable to dilute 10 g of a food with 90mlof diluent to produce a primary 10�1 dilution than to dilute1 g with 9ml diluent.

6.7. Examining cultures and recording data

The analyst carrying out colony counting, recording ofcolonial morphology, examination of confirmatory tests,etc., must be both technically competent, and physicallyand mentally ‘‘fit’’ to do the work. Selection of a number of‘‘typical’’ colonies for confirmatory analysis is critical,as is the choice of test protocol. Accurate counting andrecording of colony numbers for quantitative tests(whether done manually or instrumentally) is vital.

6.8. Quality monitoring

Any competent laboratory should monitor the quality ofits microbiological analysis through the use of QMprocedures, using both standardized reference materialsand parallel testing by individuals and groups of micro-biologists. Data generated in QM analyses should alwaysbe subjected to effective statistical analysis including trendanalysis.

Acknowledgement

We are grateful for the financial support of the UK FoodStandards Agency.

Appendix A. Definitions

Measurement of uncertainty was developed primarily inthe physical and chemical sciences and has a language of its

own. Although some terms correspond to those usedtraditionally in statistical literature, many will be unfami-liar to microbiologists. Definitions of terms related touncertainty are to be found in numerous official reports, ofwhich the following are the key sources: Eurachem (2000),ISO/IEC 17025 (1999, 2005), ISO/TS 21748 (2002), NKML(1999) and MIKES (2002). The definitions given below arefrom Eurachem (2000), unless otherwise stated.

A.1. Uncertainty

The Eurachem (2000) definition of Uncertainty of ameasurement is ‘‘A parameter associated with the result of a

measurement that characterizes the dispersion of the values

that could reasonably be attributed to the measurand’’. Thiscan be rewritten, as ‘‘Uncertainty is a measure of the likely

range of values that is indicated by an analytical result.’’This parameter is defined as a standard deviation (qv), a

confidence interval (qv) or some other quantitative measureof variability. The ‘‘measurand’’ is defined as the quantityof an analyte (qv) or other measurable quantity (e.g. a pHvalue). In microbiological terms, ‘‘measurand’’ refers to thenumber of cfu, the MPN of an organism or some otherquantity (e.g. plaque-forming units of a bacteriophage) perunit quantity of test sample (qv) analysed.

A.2. Uncertainty source

When a sample is analysed for a colony count, the meanresult is dependant upon: (a) the true level of organismspresent in the test sample (and in the serial dilutionsused as inocula) that are able to grow in the conditionsof the test to produce detectable colonies; (b) thevariation in the distribution of organisms in the originalsample, in the test sample(s), and in the serialdilutions tested; (c) a laboratory bias, defined as x

L, with

variance, s2L; and (d) a measurement error, xe, withvariance s2e . The component factors that contribute toboth laboratory bias and measurement error are describedin Section 2.

A.3. Uncertainty components

In assessing uncertainty, it may be necessary to evaluatethe effects of a number of components (sources ofuncertainty); each contribution to the overall uncertaintyis measured by the Standard Uncertainty, and is generallythe standard deviation (qv) of the measurement of thatcomponent. Where components interact, it may benecessary to assess the covariance (qv) of the individualcomponents. For an analytical result (measurement) theterm combined standard uncertainty, denoted by uc(y).

A.4. What is the difference between error and uncertainty?

Error is defined as the difference between an individualresult and the true value of the measurand. However, error


is an idealized concept and cannot be known exactly.Uncertainty, by contrast, describes the most likely limitsassociated with a measurement value; uncertainty valuesmust not be used to ‘‘correct’’ a measurement result. Inchemical analysis, a measurement value may be correctedto allow for, e.g. incomplete recovery of the analyte duringthe preparation stages of an analysis, such that thecorrected measured value may be deemed to be very closeto the true value. In such a situation, the error may be verysmall but the uncertainty associated with the analysis maystill be very large because of the lack of certainty as to theprecision (qv) of the analytical method. Uncertainty mustnever be interpreted as a measurement of the error itself,nor the error remaining after application of any correctionfactor.

Error has two components: a random component and asystematic component. Random error arises from unpre-dictable variations that affect the measurement result andgive rise to variations in repeated observations of themeasurand. It is defined as the result of a measurementminus the mean value that would be obtained from aninfinite number of measurements of the same measurandcarried out under repeatability conditions (qv). Randomerror is equal to the total error minus systematic error andsince only a finite number of measurements can be made, itis possible only to determine an estimate of random error.Random error will be present even when all conditions are‘identical’.

Systematic error is defined as the component of errorthat remains constant, or occurs in a predictable manner,during the course of a series of analyses of the samemeasurand. Systematic errors in microbiological analysismay arise from, e.g. use of an incorrect formulation for adiluent or a culture medium; incorrect calibration ofautoclave temperature such that repeated batches ofculture medium are exposed to excessive temperatureduring sterilisation; inaccurate calibration of a pH meter,etc. It is not possible to correct for systematic errors inmicrobiological analysis since the extent of the errorscannot be readily quantified.

Spurious errors (or mistakes) arise through humanfailure or instrument malfunction and invalidate a mea-surement. Measurements that are obviously incorrectshould always be discarded, although incorrectly tran-scribed data can be corrected provided the original data areavailable.

A.5. Definition of some key terms associated with

uncertainty measurement

Measurand is the term used to describe the amount, i.e.the mass, number or volume concentration) of an analytethat is being measured, e.g. the number of cfu per unitquantity of sample.

Analyte is the generic term used to describe a specificsubstance measured by the test procedure, e.g. an aerobiccolony count, an Enterobacteriaceae colony count, etc.

Sample refers to the source material from which ananalytical (test) sample is taken. The test or analytical

sample is the quantity of sample material that is measuredand homogenized, extracted or otherwise prepared for usein the test.

Standard uncertainty, denoted by the term uðyiÞ, de-scribes the uncertainty of a measurand expressed as itsstandard deviation.

Components of uncertainty are the individual sources ofuncertainty that together contribute to the combined

standard uncertainty, denoted by ucðyÞ. The combinedstandard uncertainty of a measurand equals the positivesquare root of the variances or co-variances of thecomponents of uncertainty, weighted according to howthe measurand varies with these quantities. Expanded

uncertainty (U), defines an interval within which the valueof the measurand is believed to lie with a defined level ofconfidence, and is the preferred way of reporting theuncertainty of an analytical result. The value of U can becalculated from the combined standard uncertainty and acoverage factor, using the expression: U ¼ kuc.The coverage factor, denoted by k, is a numerical

value used as a multiplier of the combined standarduncertainty in order to obtain an expanded uncertainty.The coverage factor used will depend upon the statisticalprobability required; for an approximate 95% probability,k has a value of 2 and for 99% probability k will have avalue of 3.An uncertainty budget is a list of sources of uncertainty

and their associated standard uncertainties that may beused to evaluate a combined standard uncertainty value fora measurand.

Bias is the difference between the expected test result andan accepted reference value (if known). Bias is the totalsystematic error as opposed to random error, and severalsystematic errors may contribute to the bias value.

Trueness is the closeness of agreement between theaverage values obtained from a large set of test results andan accepted reference value and is normally expressed interms of bias. Trueness should not be referred to as‘‘the accuracy of the mean’’ as is the common usage inbiostatistics.

Precision is the closeness of agreement between indepen-dent test results obtained under stipulated conditions.‘‘Independent test results’’ are those obtained in a mannerthat is not influenced by any previous result on the sameor a similar test material. Quantitative measures ofprecision depend critically on the application of ‘‘stipulatedconditions’’; repeatability and reproducibility conditions (qv)are particular examples of extreme stipulated conditions.The measure of precision is expressed in terms of impreci-sion and is computed as the standard deviation of thetest results. Low precision is reflected by a high standarddeviation.

Repeatability is a measure of variability derived underspecified repeatability conditions, i.e. where independenttest results are obtained with the same method on identical


test items in the same laboratory by the same analyst usingthe same equipment, batch of culture media and diluents,and tested within short intervals of time.

Repeatability standard deviation (sr) is the standarddeviation of the mean result obtained under repeatabilityconditions. The terms ‘‘standard error’’ and the ‘‘standarderror of the mean’’ have also been used to describe thesame parameter.

Repeatability limit (r) is a value less than or equal towhich the absolute difference between two test resultsobtained under repeatability conditions is expected to agreewith a probability of 95%, r ¼ 2:28sr.

The sample standard deviation (s) is an estimate of thepopulation standard deviation and is derived from theexpression:

s ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPni¼1ðyi � yÞ2

n� 1

s,

where y is the mean of n values (yi) with i ¼ 1 . . . n.The standard error of the mean (y) of n values taken from

a population is given by the expression:

sy ¼sffiffiffinp .

The RSD is an estimate of the standard deviation (s) of aset of n results divided by the mean value ðyÞof that set.This is frequently referred to as a coefficient of variation

and is often stated as a percentage value

RSD ¼s

yand RSD% ¼ 100

s

y.

Reproducibility is a measure of precision under reprodu-

cibility conditions, i.e. conditions where test results areobtained with the same method on identical test itemsin different laboratories with different operators usingdifferent equipment. A valid statement of reproducibilityrequires specification of the conditions used.

Reproducibility standard deviation (sR) is the standarddeviation of test results obtained under reproducibilityconditions. The value of sR is derived from the combinedsample standard deviations of the mean for betweensample (sr), between analyst (sa) and between laboratory(sL) estimates. For a particular determination,

SR ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffis2rnr

þs2ana

þs2LnL

� �s,

where nr, na and nL are the number of samples, analysts andlaboratories, respectively. For comparative purposes, SR isusually quoted for nr ¼ na ¼ nL ¼ 1.

Reproducibility limit (R) is the value less than or equal towhich the absolute difference between two test resultsobtained under reproducibility conditions is expected to bewith a probability of 95%, R ¼ 2:28SR.

References

Accreditation Advisory Group of the Institute of Food Science

Technology (AAG), 2005. Guideline no. 13. Uncertainty of measure-

ment in food microbiology by analysis of variance.

Analytical Methods Committee, 2001. Robust statistics: a method of

coping with outliers. Analytical Methods Brief No. 6, April 2001.

Royal Society of Chemistry.

Anderson, J.M., Baird-Parker, A.C., 1975. A rapid and direct plate

method for the enumeration of Escherichia coli biotype 1 in food.

J. Appl. Bacteriol. 39, 111–117.

Andrews, W.H., 1987. Recommendations for preparing test samples for

AOAC collaborative studies of microbiological procedures for foods.

J. AOAC Int. 70, 931–936.

Andrews, W.H., 1996. Validation of modern methods in food micro-

biology by AOAC International, collaborative study. Food Control 7,

19–29.

Andrews, W.H., 1997. New trends in food microbiology: an AOAC

International perspective. J. AOAC Int. 80, 908–912.

Anon, 1979. EEC Report: Microbiological Methods for Control of

Poultry Meat. Study P.203, VI/5021/79-EN, Commission of the

European Communities.

Anon, 1998. MicroVal Rules and Certification Scheme. MicroVal

Sceretariat, NNI, Delft, The Netherlands.

Augustin, J.-C., Cartier, V., 2006. Lessons from the organisation of a

proficiency testing program in food microbiology by interlaboratory

comparison: analytical methods in use, impact of methods on

bacterial counts and measurement uncertainty of bacterial counts.

Food Microbiol. 26, 1–38.

Beauregard, M.R., Mikulak, R.J., Olson, B.A., 1992. A Practical Guide to

Statistical Quality Improvement. Van Nostrand Reinhold, New York.

Boisen, F., Skovgaard, N., Ewald, S., Olsson, G., Wirtanen, G., 1992.

Quantitation of microorganisms in raw minced meat using the direct

epifluorescence filter technique: NMKL Collaborative study. J. AOAC

Int. 75, 465–473.

BSI, 1991. BS 5763: Methods for the Microbiological Examination of

Foods and Feeding Stuffs, Part 13, Enumeration of E. coli: Colony

Count Technique Using Membranes, British Standards Institution,

London.

BSI, 2003. Water Quality—Enumeration of micro-organisms in water

samples—Guidance on the estimation of variation of results with

particular reference to the contribution of uncertainty of measurement.

DD 260:2003, British Standards Institution, London.

CCFRA, 2004a. A Code of Practice for Microbiological Laboratories

Handling Food, Drink and Associated Samples, Guide no. 9, third

edition. Campden and Chorleywood Food Research Association,

Chipping Campden, UK.

CCFRA, 2004b. Microbiological measurement uncertainty: a practical

guide. In: Jewell, K. (Ed.), Guideline no. 47. Campden and Chorley-

wood Food Research Association, Chipping Campden, UK.

Carabin, H., Gyorkos, T.W., Joseph, L., Payment, P., Soto, J.C., 2001.

Comparison of methods to analyse imprecise faecal coliform data from

environmental samples. Epidemiol. Infect. 126, 181–190.

Che, Y.H., Yang, Z., Li, Y., Paul, D., Slavik, M., 1999. Rapid detection of

Salmonella typhimurium using an immunoelectrochemical method

coupled with immunomagnetic separation. J. Rapid Methods Auto-

mat. Microbiol. 7, 47–59.

Cochran, W.G., 1950. Estimation of bacterial densities by means of the

‘‘most probable number’’. Biometrics 6, 105–116.

Codex Alimentarius, 2004. Draft Guidelines on Measurement Uncertainty

(at Step 8 of the Codex Procedure).

Collins, C.H., Lyne, P.M., Grange, J.M., Falkinham, J.O. (Eds.), 2004.

Collins and Lyne’s Microbiological Methods, eighth ed. Arnold,

Oxford, pp. 144–155.

Colquhoun, K.O., Timms, S., Fricker, C.R., 1998. A simple method for

the comparison of commercially available ATP hygiene-monitoring

systems. J. Food Prot. 61, 499–501.


Corkidi, G., Diaz-Uribe, R., Folch-Mallol, J.L., Nieto-Sotelo, J., 1998.

COVASIAM: an image analysis method that allows detection of

confluent microbial colonies and colonies of various sizes for

automatic counting. Appl. Environ. Microbiol. 64, 1400–1404.

Corry, J.E.L., 1982. Quality assessment of culture media by the Miles and

Misra method. In: Corry, J.E.L. (Ed.), Quality Assurance and Quality

Control of Microbiological Culture Media. G.I.T. Verlag, Darmstadt,

pp. 21–37.

Corry, J.E.L., Curtis, G.D.W., Baird, R.M. (Eds.), 1995. Culture Media

for Food Microbiology. Progress in Industrial Microbiology, vol. 34.

Elsevier, Amsterdam.

Corry, J.E.L., Curtis, G.D.W., Baird, R.M. (Eds.), 2003. Handbook of

Culture Media for Food Microbiology. Progress in Industrial

Microbiology, second ed., vol. 37. Elsevier, Amsterdam.

Dahms, S., Weiss, H., 1998. Estimation of precision values for

microbiological reference methods—standardised pour-plate techni-

que. Milchwissenschaft 53, 555–559.

Davey, G., 2001. Which method? A food microbiologist’s nightmare.

Food Aust. 53, 73–75.

De Boer, E., Beumer, R.R., 1999. Methodology for detection and typing

of food borne microorganisms. Int. J. Food Microbiol. 50, 119–130.

De Man, J.C., 1983. MPN tables corrected. Eur. J. Appl. Microbiol. 17,

301–305.

De Smedt, J.M., 1998. AOAC validation of qualitative and quantitative

methods for microbiology in foods. Int. J. Food Microbiol. 45, 25–28.

De Smedt, J.M., Bolderdijk, R.F., Rappold, H., Lautenschlaeger, D.,

1986. Rapid salmonella detection in foods by motility enrichment on a

modified semisolid Rappaport–Vassiliadis medium. J. Food Prot. 49,

510–514.

Debevere, J., Uyttendale, M., 2003. Validating detection techniques. In:

McMeekin, T.A. (Ed.), Detecting Pathogens in Food. Woodhead

Publishing, Cambridge, pp. 69–92.

Donnison, A.M., Ross, C.M., Russell, J.M., 1993. Quality control of

bacterial enumeration. Appl. Environ. Microbiol. 59, 922–925.

Dveyrin, Z., Ben-David, H., Mates, A., 2001. Proficiency testing as tool

for ISO 17025 implementation in National Public Health Laboratory:

a mean for improving efficiency. Accredit. Qual. Assur. 6, 190–194.

European Co-operation for Accreditation (EA), 2002. Accreditation

for Microbiological Laboratories. Document EA-04/10. /http://

www.european-accreditation.org/S.

European Co-operation for Accreditation (EA), 2003. EA Guidelines for

the Expression of Uncertainty in Quantitative Testing. Document EA-

4/16. /http://www.european-accreditation.org/S.

EEC, 1985. Council Directive 85/397/EEC of 5 August 1985 on health and

animal-health problems affecting intra-Community trade in heat-

treated milk. O. J. Eur. Comm. L 226, 13–29, 24.08.1985.

EEC, 1993a. Council Directive 93/43/EEC of 14 June 1993 on the hygiene

of foodstuffs. O. J. Eur. Comm. L 175, 1–11, 19.07.1993.

EEC, 1993b. Council Directive 93/99/EEC of 29 October 1993 on the

subject of additional measures concerning the official control of

foodstuffs. O. J. Eur. Comm. L 290, 14–17, 24.11.93.

EEC, 2005. Commission Regulation2073/2005 of 15 November 2005 on

microbiological criteria for foodstuffs. O. J. Eur. Comm. L 338, 1–26,

22.12.05.

EURACHEM, 2000. Quantifying Uncertainty in Analytical Measure-

ment, second ed. Laboratory of the Government Chemist, London.

Feldsine, P., Abeyta, C., Andrews, W.H., 2002. AOAC international

methods committee guidelines for validation of qualitative and

quantitative food microbiological official methods of analysis.

J. AOAC Int. 85, 1187–1200.

Fleet, G.H., 1996. Microbiological methods—the Australian perspective.

Food Control 7, 41–46.

Fowler, J.L., Clark, W.S., Foster, J.F., Hopkins, A., 1978. Analyst

variation in doing the standard plate count as described in

Standard Methods for the Examination of Dairy Products. J. Food

Prot. 41, 4–7.

Fruin, J.T., Hill, T.M., Clarke, J.B., Fowler, J.L., Guthertz, L.S., 1977.

Accuracy and speed in counting agar plates. J. Food Prot. 40, 596–599.

Gibson, D.M., Ogden, I.D., 1997. Sampling and statistics. In: Evaluation

of Fish Freshness AIR3CT94, Nantes, France, pp. 140–145.

Gibson, D.M., Coomes, P., Pimbley, D.W., 1992. Automated conduc-

tance method for the detection of salmonella in foods—collaborative

study. J. AOAC Int. 75, 293–302.

Gill, C.O., 2001. HACCP in primary processing: red meat. In: Brown,

M. (Ed.), HACCP in the Meat Industry. Woodhead, Cambridge, UK,

pp. 81–122.

Glassmoyer, K.E., Russell, S.M., 2001. Evaluation of a selective broth for

detection of Staphylococcus aureus using impedance microbiology.

J. Food Prot. 64, 44–50.

Havelaar, A.H., Heisterkamp, S.H., Hoekstra, J.A., et al., 1993.

Performance characteristics of methods for the bacteriological

examination of water. Water Sci. Technol. 27, 1–13.

Hedges, A.J., 1967. On the dilution errors involved in estimating bacterial

numbers by the plating method. Biometrics 23, 158–159.

Hedges, A.J., 2002. Estimating the precision of serial dilutions and viable

bacterial counts. Int. J. Food Microbiol. 76, 207–214.

Hedges, A.J., 2003. Estimating the precision of serial dilutions and colony

counts: contribution of laboratory re-calibration of pipettes. Int.

J. Food Microbiol. 87, 181–185.

Hedges, A.J., Jarvis, B., 2006. Application of robust methods to the

analysis of collaborative trial data using bacterial colony counts.

J. Microbiol. Methods, in press.

Hedges, A.J., Shannon, R., Hobbs, R.P., 1978. Comparison of the

precision obtained in counting viable bacteria by the Spiral Plate

Maker, the Droplette and the Miles & Misra methods. J. Appl.

Bacteriol. 45, 57–65.

Hitchins, A.D., 1996. The International Dairy Federation’s procedure for

the validation of microbiological analytical methods for dairy foods.

Food Control 7, 13–18.

Hitchins, A.D., Burney, A.A., 2004. Determination of the limits of

detection of AOAC validated qualitative microbiology methods.

AOAC International, 118th Annual Meeting Program, Poster Abstract

P-1021, p. 153.

Holbrook, R., 2000. Detection of microorganisms in foods—principles of

culture methods. In: Lund, B.M., Baird-Parker, T.C., Gould, G.W.

(Eds.), The Microbiological Safety and Quality of Food. Aspen

Publishers, Maryland, pp. 1761–1790.

Horwitz, W., 1995. Protocol for the design, conduct and interpretation of

method-performance studies. Pure Appl. Chem. 67, 331–343.

Horwitz, W., 2003. The certainty of uncertainty. J. AOAC Int. 86,

109–111.

ICMSF, 2002. Micro-organisms in Foods 7. Microbiological Testing in

Food Safety Management. Kluwer, Amsterdam.

IDF 161A, 1995. Milk-Quantitative determination of bacteriological

quality—guidance on evaluation of routine methods.

ILAC, 2000. Guide 13. Guideline for the requirements for competence of

providers of proficiency-testing schemes, ILAC-G13: 2000.

In’t Veld, P.H., 1998. The use of reference materials in quality assurance

programmes in food microbiology laboratories. Int. J. Food Micro-

biol. 45, 35–41.

In’t Veld, P.H., De Boer, E., 1991. Recovery of Listeria monocytogenes on

selective agar media in a collaborative study using reference samples.

Int. J. Food Microbiol. 13, 295–300.

Isenberg, H.D., D’Amato, R.F., 1996. Does proficiency testing meet its

objective? J. Clin. Microbiol. 34, 2643–2644.

ISO, 1993. Guide to the Expression of Uncertainty in Measurement

(GUM). ISO, Geneva.

ISO 5725-3, 1994. Accuracy (trueness and precision) of measurement

methods and results: Intermediate measures of the precision of a

measurement method, ISO, Geneva.

ISO 10272-1, 2006. Microbiology of food and animal feeding stuffs—

Horizontal method for detection and enumeration of Campylobacter

spp. –Part 1 detection method. ISO, Geneva.


horizontal method for the detection and enumeration of Listeria

monocytogenes: Part 1. Detection method, ISO, Geneva.

http://www.european-accreditation.org/




ISO 13530, 1997. Water Quality—Guide to Analytical Quality Control for

Water Analysis, ISO, Geneva.


horizontal method for the detection and enumeration of Listeria

monocytogenes: Part 2. Enumeration method, ISO, Geneva.

ISO 5725-5, 1998. Accuracy (trueness and precision) of measurement

methods and results—alternative methods for the determination of the

precision of a standard measurement method, ISO, Geneva.


preparation of test samples, initial suspension and decimal dilutions

for microbiological examination. Part I: general rules for the

preparation of the initial suspension and decimal dilutions, ISO,

Geneva.

ISO 9308-1, 2000. Water quality—detection and enumeration of E. coli

and coliform bacteria—Part 1: membrane filtration method, ISO,

Geneva.


guidelines on quality assurance and performance testing of culture

media—Part 1: General guidelines on quality assurance of culture

media in the laboratory, ISO, Geneva.


horizontal method for the enumeration of b-glucuronidase-positiveE. coli—Part I. Colony-count technique at 44 1C using membranes and

5-bromo-4-chlor-3-indolyl-b-D-glucuronide, ISO, Geneva.


horizontal method for the enumeration of b-glucuronidase-positive E. coli—Part 2. Colony-count technique at 44 1C using 5-

bromo-4-chloro-3-indolyl-b-D-glucuronide, ISO, Geneva.

ISO 6579, 2002. Microbiology of food and animal feeding stuffs—

horizontal method for the detection of Salmonella spp., ISO, Geneva.


horizontal method for the enumeration of micro-organisms—colony-

count technique at 30 1C, ISO, Geneva.


protocol for the validation of alternative methods, ISO, Geneva.


horizontal method for the enumeration of Clostridium perfringens—

colony-count technique, ISO, Geneva.

ISO 14461-1, 2005. Milk and milk products—quality control in the

microbiology laboratory—Part 1: Analyst performance assessment for

colony counts, ISO, Geneva.

ISO 14461-2, 2005. Milk and milk products—quality control in the

microbiology laboratory—Part 2: Determination of the reliability of

colony counts of parallel plates and subsequent dilution steps, ISO,

Geneva.


horizontal method for the detection and enumeration of presumptive

E. coli—most probable number technique, ISO, Geneva.

ISO 8199, 2005. Water quality—general guidance on the enumeration of

micro-organisms by culture, ISO, Geneva.

ISO 19036, 2005. Microbiology of food and animal feeding stuffs—guide

on measurement of uncertainty for quantitative determinations, ISO,

Geneva.

ISO/TS 21748, 2002. Guide to the use of repeatability, reproducibility and

trueness estimates in measurement uncertainty estimation dilution

steps, ISO, Geneva.

ISO/IEC Guide 43, 1997. Part 1. Proficiency testing by inter-laboratory

comparison—development and operation of proficiency-testing

schemes. Part 2: Selection and use of proficiency-testing schemes by

laboratory accreditation bodies, ISO, Geneva.

ISO/IEC 17025, 1999. General requirements for the competence of testing

and calibration laboratories, ISO, Geneva (revised 2005).

ISO/TR 13843, 2000. Water quality—guidance on validation of micro-

biological methods, ISO, Geneva.

ISO/TS 11133-2, 2003. Microbiology of food and animal feeding stuffs—

guidelines on preparation and production of culture media—Part 2:

Practical guidelines on performance testing of culture media, ISO,

Geneva.

IUPAC, 1988. Protocol for the design, conduct and interpretation of

collaborative studies. Prepared by Horwitz, W. Pure Appl. Chem. 60,

855–864.

IUPAC/ISO/AOAC, 1993. International harmonised protocol for profi-

ciency testing of (chemical) analytical laboratories, Protocol for

proficiency testing. Prepared by Thompson, M. and Wood, R. J.

AOAC Int. 76, 926–940.

IUPAC, 1995. Protocol for the design, conduct and interpretation of

method performance studies. Prepared by Horwitz, W. Pure Appl.

Chem. 67, 331–343.

Jackson, G.J., Wachsmuth, I.K., 1996. The US Food and Drug

Administration’s selection and validation of tests for foodborne

microbes and microbial toxins. Food Control 7, 37–39.

Jarvis, B., 1977. A chemical method for the estimation of mould in tomato

products. J. Food Technol. 12, 581–591.

Jarvis, B., 1989. Statistical Aspects of the Microbiological Analysis of

Foods. Progress in Industrial Microbiology, vol. 21. Elsevier,

Amsterdam.

Jarvis, B., Corry, J.E.L., Hedges, A., 2005. Simple statistical procedures to

estimate intermediate reproducibility based on microbiological quality

assessment and routine analysis of food samples. AOAC International,

119th Annual Meeting Programme Abstract No. 1402, pp. 172–173.

Jarvis, B., Hedges, A., Corry, J.E.L., 2004. Microbiological uncertainty:

estimates of uncertainty obtained from colony count data submitted

for proficiency assessment schemes, data derived for internal labora-

tory quality monitoring and during routine enforcement examination

of foods. In: Proceedings of the SfAM Conference on Dairy &

Food Microbiology: Challenges and opportunities, Cork, July 2004

(Abstract No. P69).

Jewell, K., 2001. Microbiological proficiency testing: a personal perspec-

tive. Accredit. Qual. Assur. 6, 154–159.

Kolari, M., Mannonen, S., Takala, T., Saris, P., Suovaniemi, Salkinoja-

Salonen, M.S., 1999. The effect of filters on aseptic pipetting lifetime of

mechanical and electronic pipettors and carryover during pipetting.

Lett. Appl. Microbiol. 29, 123–129.

Kramer, J.M., Gilbert, R.J., 1978. Enumeration of micro-organisms in

food: a comparative study of five methods. J. Hyg. 81, 151–159.

Lahellec, C., 1998. Development of standard methods with special

reference to Europe. Int. J. Food Microbiol. 45, 13–16.

Langton, S.D., Chevennement, R., Nagelkerke, N., Lombard, B., 2002.

Analysing collaborative trials for qualitative microbiological methods:

accordance and concordance. Int. J. Food Microbiol. 79, 175–181.

Leclercq, A., Lombard, B., Mossel, D.A.A., 2000. Revue bibliographique.

Normaliser les methodes d’analyse dans le cadre de la securite

microbiologique francaise des aliments: atout ou contrainte. Sci.

Aliments 20, 179–202.

Lee, R.J., Cole, S.R., 1994. Internal quality control samples for water

bacteriology. J. Appl. Bacteriol. 76, 270–274.

Leuschner, R.G.K., Bew, J., Simpson, P., Ross, P.R., Stanton, C., 2003.

A collaborative study of a method for the enumeration of probiotic

bifidobacteria in animal feed. Int. J. Food Microbiol. 83, 161–170.

Lightfoot, N.F., Maier, E.A. (Eds.), 1998. Microbiological Analysis of

Food and Water. Guidelines for Quality Assurance. Elsevier,

Amsterdam.

Lightfoot, N.F., Tillett, H.E., Boyd, P., Eaton, S., 1994. Duplicate split

samples for internal quality-control in routine water microbiology.

Lett. Appl. Microbiol. 19, 321–324.

Lightfoot, N.F., Richardson, I.R., Harford, J.P., 2001. The use of

Lenticules for the process control of enumeration techniques in food

and environmental microbiology. J. Appl. Microbiol. 91, 660–667.

Lombard, B., Gomy, C., Catteau, M., 1996. Microbiological analysis of

foods in France: standardized methods and validated methods. Food

Control 7, 5–11.

Mackey, B.M., 2000. Injured bacteria. In: Lund, B.M., Baird-Parker,

T.C., Gould, G.W. (Eds.), The Microbiological Safety and Quality of

Food. Aspen Publishers, Inc, Maryland, pp. 315–335.

Made, D., Petersen, R., Trumper, K., Stark, R., Grohmann, L., 2004. In-

house validation of a real-time PCR method for rapid detection of


Salmonella spp. in food products. Eur. Food Res. Technol. 219,

171–177.

Marotz, J., Lubbert, C., Eisenbeiss, W., 2001. Effective light recognition

for automated counting of colonies in Petri dishes (automated colony-

counting). Comput. Methods Prog. Biomed. 66, 183–198.

MIKES (Centre for Metrology and Accreditation), 2002. Uncertainty of

Quantitative Determinations Derived by Cultivation of Microorgan-

isms. Advisory Commission for Metrology, Chemistry Section, Expert

Group for Microbiology, Publication J3/2002, Helsinki, Finland.

Mossel, D.A.A., Corry, J.E.L., 1977. Detection and enumeration

of sublethally injured pathogenic and index bacteria in foods and

water processed for safety. Alimenta 16, 19–34 (special issue on

Microbiology).

Mossel, D.A.A., Corry, J.E.L., Struijk, C.B., Baird, R.M., 1995. Essentials

of the Microbiology of Foods. Wiley, Chichester.

Muller, A., Hildebrandt, G., 1990. Sampling errors and systematic errors

in bacterial count determination. The accuracy of colony-counting.

A survey of the literature. Flieschwirtschaft 70, 680–684.

Niemela, S.I., 1996. A semi-empirical precision control criterion for

duplicate microbial colony counts. Lett. Appl. Microbiol. 22, 315–319.

Niemela, S.I., 2002. Uncertainty of quantitative determinations derived by

cultivation of microorganisms, second ed. Centre for Metrology

and Accreditation, Advisory Commission for Metrology, Chemistry

Section, Expert Group for Microbiology, Publication J3/2002,

Helsinki, Finland.

Niemi, R.M., Niemela, S.I., 2001. Measurement uncertainty in micro-

biological cultivation methods. Accredit. Qual. Assur. 6, 372–375.

NMKL, 1994. Quality Assurance Guidelines—for microbiological

laboratories, second ed., Report no. 5, Nordic Committee on Food

Analysis, Oslo, Norway.

NMKL, 1999. Procedure No. 8, Measurement of Uncertainty in

Microbiological Examination of Foods, Nordic Committee on Food

Analysis, Oslo, Norway.

Novicki, T.J., Daly, J.A., Mottice, S.L., Carroll, K.C., 2000. Comparison

of sorbitol MacConkey agar and a two-step method which utilizes

enzyme-linked immunosorbent assay toxin testing and a chromogenic

agar to detect and isolate enterohemorrhagic E. coli. J. Clin.

Microbiol. 38, 547–551.

Ogden, I.D., Brown, G.C., Gallacher, S., Garthwaite, P.H., Gennari, M.,

Gonzalez, M.P., Jørgensen, L.B., Lunestad, B.T., MacRae, M., Nunes,

M.C., Peterson, A.C., Rosnes, J.T., Vliegenthart, J., 1998. An

interlaboratory study to find an alternative to the MPN technique

for enumerating E. coli in shellfish. Int. J. Food Microbiol. 40, 57–64.

Olsen, J.E., 2000. DNA-based methods for detection of food-borne

bacterial pathogens. Food Res. Int. 33, 257–266.

Ornemark, U., Boley, N., Saeed, K., van Berkel, P.M., Schmidt, R.,

Noble, M., Makinen, I., Keinanen, M., Uldall, A., Steensland, H., Van

der Veen, A., Tholen, D.W., Golze, M., Christensen, J.M., De Bievre,

P., De Leer, W.B., 2001. Proficiency testing in analytical chemistry,

microbiology, and laboratory medicine—working group discussions

on current status, problems, and future directions. Accredit. Qual.

Assur. 6, 140–146.

Oscroft, C.A., Corry, J.E.L. (Eds.), 1991. Guidelines for the Pre-

paration, Storage and Handling of Microbiological Culture Media.

Campden Food and Drink Research Association Technical Manual

no. 33.

Peeler, J.T., Leslie, J.E., Danielson, J.W., Messer, J.W., 1982. Replicate

counting errors by analysts and bacterial colony counters. J. Food

Prot. 45, 238–240.

Perez, F.G., Mascini, M., Tothil, I.E., Turner, A.P.F., 1998.

Immunomagnetic separation with mediated flow injection analysis

amperometric detection of viable E. coli O157. Anal. Chem. 70,

2380–2386.

Peterz, M.E.G., 1991. Temperature in agar plates and its influence on the

results of quantitative microbiological food analyses. Int. J. Food

Microbiol. 14, 59–66.

Peterz, M., 1992. Laboratory performance in a food microbiology

proficiency testing scheme. J. Appl. Bacteriol. 73, 210–216.

Peterz, M., Norberg, P., 1983. Freeze-dried mixed cultures as samples for

proficiency tests and collaborative studies in food microbiology.

J. AOAC Int. 66, 1510–1513.

Peterz, M., Norberg, P., 1986. A split-sample study of microbiological

food laboratories used freeze-dried mixed cultures. Int. J. Food


Peterz, M., Steneryd, A.C., 1993. Freeze-dried mixed cultures as reference

samples in quantitative and qualitative microbiological examinations

of food. J. Appl. Bacteriol. 74, 143–148.

PHLS, 1998. PHLS Guidance Note—Uncertainty of Measurement in

Testing. Q. SOP4 Issue no. 2, 01.07.1998, Technical Services, H.Q.,

London.

Piton, C., Grappin, R., 1991. A model for statistical evaluation of precison

parameters of microbiological methods: application to dry rehydra-

table film methods and IDF reference methods for enumeration of

total aerobic mesophilic flora and coliforms in raw milk. J. AOAC Int.

74, 92–103.

Rattanasomboon, N., Bellara, S.R., Harding, C.L., Fryer, P.J., Thomas,

C.R., Al-Rubeai, M., McFarlane, C.M., 1999. Growth and enumera-

tion of the meat spoilage bacterium Brochothrix thermosphacta. Int.

J. Food Microbiol. 51, 145–158.

Reichmuth, J., Suhren, G., 1996. Considerations concerning the evalua-

tion of routine methods for assessing the bacteriological quality of raw

milk. Kieler Milchwirtschaft. Forschung. 48, 175–185.

Rentenaar, I.M.F., 1996. MicroVal, a challenging Eureka project. Food

Control 7, 31–36.

Roberts, D., Greenwood, M. (Eds.), 2003. Practical Food Microbiology,

third ed. Blackwell Publishing, Oxford.

Salkin, I.F., Limberger, R.J., Stasik, D., 1997. Commentary on the

objectives and efficacy of proficiency testing in microbiology. J. Clin.

Microbiol. 35, 1921–1923.

Salo, S., Laine, A., Alanko, T., Sjober, A., Wirtanen, G., 2000. Validation

of the microbiological methods Hygicult dipslide contact plate and

swabbing in surface hygiene control: a Nordic collaborative study.

J. AOAC Int. 83, 1357–1365.

Standing Committee of Analysts (SCA), 2002. The Microbiology of

Drinking Water. Environment Agency, Nottingham /www.environment-

agency.gov.uk/scienceS.

Schulten, S.M., in’t Veld, P.H., Nagalkerke, N.J.D., Scotter, S., de Buyser,

M.L., Rollier, P., Lahellec, C., 2000. Evaluation of the ISO 7932

standard for the enumeration of Bacillus cereus in foods. Int. J. Food


Scotter, S., Wood, R., 1996. Validation and acceptance of modern

methods for the microbiological analysis of foods in the UK. Food

Control 7, 47–51.

Scotter, S., Aldridge, M., Back, J., Wood, R., 1993. Validation

of European Community methods for microbiological and

chemical analysis of raw and heat-treated milk. J. Ass. Pub. Anal.

29, 1–32.

Scotter, S.L., 1996. Proficiency testing in food microbiology, MAFF

Central Science Laboratory ‘Quality assessment scheme’. Food Sci.

Technol. Today 10, 227–228.

Scotter, S.L., Langton, S., Lombard, B., Lahellec, C., Schulten, S.,

Nagelkerke, N., in’t Veld, P.H., Rollier, P., 2000. Validation of ISO

method 11290 Part 1. Detection of Listeria monocytogenes in foods.


Scotter, S.L., Langton, S., Lombard, B., Lahellec, C., Schulten, S.,

Nagelkerke, N., in’t Veld, P.H., Rollier, P., 2001. Validation of ISO

method 11290 Part 2. Enumeration of Listeria monocytogenes in foods.


Silliker, J.H., Gabis, D.A., May, A., 1979. ICMSF Methods Studies. XI.

Collaborative/comparative studies on determination of coliforms using

the most probable number procedure. J. Food Prot. 42, 638–644.

Snell, J.J.S., 1991. The UK national external quality assessment scheme

for microbiology. PHLS Microbiol. Dig. 8, 46–48.

Sowers, E.G., Wells, J.G., Strockbine, N.A., 1996. Evaluation of

commercial latex reagents for identification of O157 and H7 antigens

of E. coli. J. Clin. Microbiol. 34, 1286–1289.

http://www.environment-agency.gov.uk/science

http://www.environment-agency.gov.uk/science


Stephens, P.J., Mackey, B.M., 2003. Recovery of stressed microorganisms.

In: Corry, J.E.L., Curtis, G.D.W., Baird, R.M. (Eds.), Handbook of

Culture Media for Food Microbiology. Progress in Industrial

Microbiology, vol. 37, pp. 25–48.

Stephens, P.J., Druggan, P., Nebe-von Caron, G., 2000. Stressed

Salmonella are exposed to reactive oxygen species from two

independent sources during recovery in conventional culture media.


Suhren, G., Reichmuth, J., 2000. Interpretation of quantitative micro-

biological results. Milchwissenschaft 55 (1), 18–22.

Swaroop, S., 1951. The range of variation of most probable number of

organisms estimated by the dilution method. Indian J. Med. Res. 39,

107–134.

Tillett, H.E., Crone, P.B., 1976. Quality control of the isolation rate of

pathogens in medical microbiology laboratories. J. Hyg. 77, 359–367.

Tillett, H.E., Lightfoot, N.F., 1991. Preliminary statistical assessment of

UK water-quality control trials. Water Sci. Technol. 24 (2), 57–60.

Tillett, H.E., Lightfoot, N.F., Eaton, S., 1993. External quality assessment

in water microbiology: statistical analysis of performance. J. Appl.

Bacteriol. 74, 497–502.

Tillett, H.E., Lightfoot, N.F., Eaton, S., Place, B.M., 2000. External

quality assessment of microbial counts from water: to score or not to

score for proficiency. J. Chart Inst. Water Environ. Manage. 14,

304–308.

Tillett, H.E., Lightfoot, N.F., 1995. Quality control in environmental

microbiology compared with chemistry: what is homogeneous and

what is random? Water Sci. Technol. 31, 471–477.

UKAS, 1997. M3003 The expression of uncertainty and confidence in

measurement, 76pp.

UKAS, 2000. The Expression of Uncertainty in Testing, Edition 1, UKAS

Publication ref: LAB 12.

van der Voet, H., van Raamsdonk, L.W.D., 2004. Estimation

of accordance and concordance in inter-laboratory trials of

analytical methods with qualitative results. Int. J. Food Microbiol.

95, 231–234.

van der Zee, H., Huis in’t Velt, J.H.J., 1997. Rapid and alternative

screening methods for microbiological analysis. J. AOAC Int. 80,

934–940.

Vesey, G., 2005. Precise reference materials for microbiology. 119th

AOAC International Annual Meeting, Orlando, September 2005.

Symposium on Microbiological Reference Materials, Abstract S802.

Vivegnis, J., Oger, R., Decallonne, J., 1997. Microbiologie alimentaire;

evaluation de la precision d’essais d’aptitude sur un aliment frais

refrigere contamine naturellement. Sci. Aliments 17, 641–654.

Voysey, P.A., Jewell, K., 1999. Uncertainty associated with microbiolo-

gical measurement, Project no. 29732, Campden and Chorleywood

Food Research Association Review no. 15, CCFRA, Chipping

Campden, UK.

Westwood, N., Hodgkinson, P., 1977. Variable recovery of heat damaged

E. coli in stacked plastic dishes. J. Appl. Bacteriol. 42, 145–148.

Wilson, I.G., 1995. Use of the IUL Countermat automatic colony counter

for spiral plated total viable counts. Appl. Environ. Microbiol. 61,

3158–3160.

Woodward, R.L., 1957. How probable is the most probable number?

J. Am. Waterworks Assoc. 49, 1060–1068.

Youden, W.J., Steiner, E.H., 1975. Statistical Manual of the

Association of Official Analytical Chemists. AOAC International,

Washington, DC.

corry et al. 2007.pdf

Documents