Vol. 59, No. 2APPLIED AND ENVIRONMENTAL MICROBIOLOGY, Feb. 1993, p. 380-3880099-2240/93/020380-09$02.00/0Copyright © 1993, American Society for Microbiology

Evaluation of Commercial Presence-Absence Test Kits forDetection of Total Coliforms, Escherichia coli, and

Other Indicator BacteriaJAMES A. CLARK'* AND ABDUL H. EL-SHAARAWI2

Laboratory Services Branch, Water Quality Section, 125 Resources Road, Etobicoke, Ontario M9P 3V6,1and National Water Research Institute, Burlington, Ontario L7R 4A6,2 Canada

Received 29 July 1992/Accepted 12 November 1992

Evaluations of several commercial presence-absence (P-A) test kits were performed over a 6-month periodin 1990 by using the Ontario Ministry of the Environment (MOE) P-A test for comparison. The generalprinciples of the multiple-tube fermentation technique formed the basis for conducting the product evaluations.Each week, a surface water sample was diluted and inoculated into 25 99-ml dilution blanks for each of threedilutions. The inoculated dilution blanks from each dilution series were randomly sorted into sets of five. Threeof these sets were inoculated into the P-A test kits or vice versa, as required. The other two sets were passedthrough membrane filters, and one set of five membrane filters was placed onto m-Endo agar LES to givereplicate total coliform counts and the other set was placed onto m-TEC agar to give replicate fecal coliformresults. A statistical analysis of the results was performed by a modified logistic transform method, whichprovided an improved way to compare binary data obtained from the different test kits. The comparative testresults showed that three of the four commercial products tested gave very good levels of recovery and that thefourth commercial product gave only fair levels of recovery when the data were compared with the data fromMOE P-A tests and membrane filter tests. P-A bottles showing positive results after 18 h of incubation that weresubcultured immediately in ECMUG tubes frequently could be confirmed as containing total coliforms, fecalcoliforms, or Escherichia coli after 6 h of incubation; thus, the total incubation time was only 24 h. The presenceof anaerogenic coliforms and Aeromonas spp. and presumptive positive occurrences were also recorded forconsideration as possible indicators of deteriorating water quality. A limited number of split-sample analyseswere performed with drinking water samples for two of the commercial P-A test kits; the results showed thatthe level of indicator organism recovery was equivalent to that of the MOE P-A test.

Examination of bacteriological water samples to deter-mine whether the quality of the water is acceptable fordrinking and other domestic purposes has traditionally beendone by most-probable-number (MPN) procedures or themembrane filter (MF) technique (1). Interest in presence-absence (P-A) methods for determining the microbiologicalquality of drinking water has increased dramatically in thelast 10 years. This methodology became officially availableon 31 December 1990, when the Total Coliform Rule,promulgated by the U.S. Environmental Protection Agency,became effective (10). The new regulation changed themanner of reporting total coliforms from numbers per 100 mlto the presence or absence of total coliforms in 100 ml ofsample. Prior to the promulgation of the new regulation,several studies were done to compare the P-A test with theMF technique and the MPN procedure (otherwise known asthe multiple-tube fermentation method). These studies es-sentially demonstrated that the P-A test was equivalent to orbetter than the MF and MPN techniques for detecting totalcoliform organisms (2, 12, 15, 16).About the same time, an alternative test procedure for

detecting total coliforms and Escherichia coli was beinginvestigated (8, 9). One part of the test involved the substrateo-nitrophenyl-p-D-galactopyranoside; when this substratewas acted upon by an enzyme produced by total coliformbacteria, it released o-nitrophenyl, giving a yellow color tothe broth medium. The other part of the test involved the

* Corresponding author.

substrate 4-methylumbelliferyl-o-D-glucuronide. When thiscompound was acted on by the enzyme 13-glucuronidaseproduced by E. coli, a bright blue fluorescent color appearedin the broth medium when the preparation was viewed in thedark with long-wavelength UV light (11).The test systems were first combined in a commercial

product known as Colilert, which was marketed by AccessAnalytical Systems. A similar product, known as Coliquik,was developed and marketed by the Hach Company. Severalcomparative studies were performed both in the UnitedStates and in Canada to determine whether the Colilert testdetected total coliforms and E. coli as well as the StandardMethodsfor the Examination of Water and Wastewater MF,MPN, and P-A procedures do (5, 8). Although equivalentresults were not always obtained, the authors of the studiesgenerally concluded that the Colilert test was a satisfactoryalternative to the other Standard Methods tests used fordetection of total coliforms. Other studies, in which both theColilert and Coliquik tests were compared with the StandardMethods MF procedures used for detection of totalcoliforms and fecal coliforms, generally revealed good agree-ment among the tests used for detection of total coliforms,but the authors of these studies found that the two commer-cial tests were inferior to the Standard Methods fecalcoliform procedure when colonies growing on mFC mediumwere identified as E. coli (3, 14).

In this study we compared the Ministry of the Environ-ment (MOE) P-A test with the Colilert and Coliquik tests, aswell as the total coliform and fecal coliform MF tests. Inaddition, as the Hach Company has also produced two other

380

on Novem

ber 9, 2020 by guesthttp://aem

.asm.org/

Dow

nloaded from

COMMERCIAL P-A TEST KITS 381

less expensive P-A test kits (the Hach Disposable test kit andthe Hach Vial or Tube test kit), both of these test kits wereincluded in the comparative evaluation. The latter test kitswere similar to the MOE P-A test and contained a lactose-based medium supplemented with bromocresol purple toindicate lactose fermentation by coliform bacteria; the mediain these test kits also contained the 4-methylumbelliferyl-,B-D-glucuronide reagent to determine the presence of 3-glucu-ronidase from E. coli.

MATERIALS AND METHODS

P-A test kits. The four commercial P-A test kits mentionedabove were purchased for analysis of 100-ml samples andwere evaluated over a 6-month period starting at the end ofMay 1990. The Colilert and Coliquik test reagents were inpowder form in screw-cap glass tubes and plastic pillows,respectively. Each reagent was added to a 100-ml sample,which was shaken to dissolve the powder. Each HachDisposable test kit consisted of a plastic bottle containingabout 50 ml of triple-strength medium to which 100 ml of asample was added. Each Hach Vial test kit consisted of asmall plastic vial that held about 25 ml of concentratedmedium, which was added to 100 ml of a sample; thepreparation was then shaken to mix the contents.Because of certain time and space limitations, only two

commercial test kits could be compared with the MOE P-Atest and the MF methods used for detection of totalcoliforms and fecal coliforms at any one time. Four groups oftest evaluations were done in the Central (Toronto) Labora-tory, Etobicoke, Ontario, Canada, and one group of testevaluations was done in the London Regional Laboratory,London, Ontario, Canada.

Test protocol. Although the P-A test was intended prima-rily for examination of samples obtained from distributionsystems, these types of samples rarely produce positive testresults except on certain unpredictable occasions. Thisattribute makes distribution systems poor sources for sam-ples when new tests or media must be evaluated in a shortperiod of time. For this reason, a surface water sample wasused as a source of indicator organisms.A sample for the Toronto laboratory was usually collected

each Monday from a local river, the Humber River, in asterile 300-ml plastic bottle containing sodium thiosulfate.This sample was used for analyses on the day of collectionand then was refrigerated and used the following day foranother set of analyses. Samples for the London RegionalLaboratory were collected weekly from various locationsthat were polluted by sewage or food wastes.The general principles of the multiple-tube fermentation

technique were used to dilute out to extinction the indicatororganisms in each sample (1). Each week, three appropriatedecimal dilutions of the surface water sample were prepared,and 1-ml volumes of each dilution were inoculated into 2599-ml sterile buffered dilution blanks, as shown in Fig. 1.Each dilution blank then became a 100-ml sample for inoc-ulation into one of the P-A test kits or filtration through anMF. The inoculated dilution blanks or samples from each25-bottle dilution series were randomly sorted into sets offive. One set of five samples was inoculated into five P-Abottles used for one of the P-A test kits; the second and thirdsets were treated similarly for the other two P-A test kits.The fourth set of five 100-ml samples was passed throughfive MFs, each of which was placed onto m-Endo agar LES(Difco); the fifth set was treated in a similar manner, and themembranes were placed onto m-TEC agar (Difco).

Surface water sampleI I

10 ml lmlI I0 90ml 04- 99mlI dilution I dilution

imil ml blank lml blankI I Io 0 0o 0 0o 0 0o 0 0

25bottlesIIdlton99 m 1 in 10 1 in 100bilankson dilution dilutionblanks ~ ~~~~~~IIo o 0o o 0o o 0o o 0

Pick sets of 5 bottles at random from each dilutionseries and rearrange in sets of 5 as indicated

00000 00000 0000000000 00000 0000000000 00000 0000000000 00000 0000000000 00000 00000PAPA PAMFMF PAPAPAMFMF PAPAPAMFMF1 2 3 TC FC 1 2 3 TC FC 1 2 3 TCFC

FIG. 1. Diagramatic representation of the dilution procedureused for inoculation of 100-ml volumes into P-A test kits and throughMFs. TC, total coliforms; FC, fecal coliforms.

The whole procedure was repeated for the other twodilution series, so that each P-A test kit and MF parameterconsisted of five bottles or filters that were inoculated with100-ml volumes of a sample over three decimal dilutions.Each of the operations described above was randomized asmuch as possible. The m-Endo agar LES plates were incu-bated at 350C for 20 to 22 h (1), and the m-TEC agar plateswere incubated at 44.50C for 20 to 22 h (7). Any red colonywith a metallic sheen on m-Endo agar LES was considered atotal coliform colony, and any yellow or yellow-browncolony on m-TEC agar was considered a fecal coliformcolony. The P-A test bottles were incubated at 350C andwere examined over a 3-day period for presumptive positivetests (4).-MF test confirmation. Total coliform and fecal coliform

counts were determined to establish the relative number oftarget organisms that were initially inoculated into eachdilution blank, but the plates were also scored on the basis ofthe presence or absence of the target organisms. No colonyconfirmation procedure was performed with the initial set ofcomparative tests, but later any plates containing only one ortwo colonies had their colonies removed for confirmation inECMUG broth; the preparations were incubated at either35 or 44.50C depending on the original isolation medium.EGMUG broth contained 37 g of EC medium (Difco) per literplus 0.075 g of 4-methylumbelliferyl-o-D-glucuronide reagent(Hach) per liter. Following incubation, ECMUG tubes ex-hibiting growth with or without gas were checked for fluo-rescence. However, because limited numbers of fecal coli-form colonies were isolated in ECMUG broth for E. coliconfirmation, this parameter was not included in the statis-tical comparison of MF and P-A results.P-A test confirmation. P-A test bottles were incubated at

350C and examined over a 3-day period for presumptive

VOL. 59, 1993

on Novem

ber 9, 2020 by guesthttp://aem

.asm.org/

Dow

nloaded from

382 CLARK AND EL-SHAARAWI

Positive ECMUG tube

350C 44.50C

Turbidity Gas Gas and Turbidity Gas Gas andonly positive fluorescent only positive fluorescent

positive positive

Total Escherichia ±Fecal icoliform 11 coli coliform 1

Streak MacConkey Streak nutrient gelatinagar for isolated agar for isolated

colonies coloniesI l l l

Red or pink Colourless Gelatin and Gelatin positivelactose positive lactose negative oxidase or negative

colonies colonies positive and oxidasel | negative

Anaerogenic Aeromonas Presumptivecoliform sp. result?

FIG. 2. Procedure and interpretation of results for P-A confir-mation tests.

positive results; the bottles which showed presumptivepositive results were then subjected to confirmation tests (4).The bottles were examined as soon as 18 to 21 h afterinoculation, but they were also examined at 24, 28, 48, and72 h, depending on the test kit being evaluated. Inocula (0.05ml) from each P-A bottle showing evidence of a positiveresult were transferred into two tubes containing ECMUGbroth; one of these tubes was incubated at 35°C, and theother was incubated at 44.5°C. The ECMUG broth tubesinoculated with material from positive 18- to 21-h P-A bottleswere read after 6, 24, and 48 h of incubation. All other tubeswere read after 24 and 48 h. The indicator groups distin-guished by the confirmatory tests included total coliforms,fecal coliforms, E. coli, Aeromonas spp., and anaerogeniccoliforms.

Figure 2 shows how we interpreted the ECMUG brothtube results. Gas production at 35°C was scored as a positiveresult for total coliforms; gas production at 44.5°C wasscored as a positive fecal coliform result. Gas productionwas indicated by the presence of effervescence at 6 h and gasaccumulation in the small inverted tubes after 24 to 48 h. UVfluorescence at either temperature was considered a positiveresult for E. coli on the basis of the results of a previousin-house study; in this study the researchers used Entero-tubes (Hoffman-LaRoche Ltd., Vaudreuil, Quebec, Cana-da), and E. coli was isolated and identified from 97% of 36samples which produced gas and fluorescence in ECMUGbroth at either 35 or 44.5°C or at both 35 and 44.5°C. Tubesexhibiting only turbidity after 48 h of incubation weresubcultured onto MacConkey and nutrient gelatin agarplates to determine the presence of anaerogenic coliforms oraeromonad types of organisms. With some bottles, appro-priate identification of the bacteria was not possible, andthese bottles were labelled presumptive.

Statistical analysis. The results of each of the four P-A testkits were compared with the results of the MOE P-A and MFtests for the total coliform and fecal coliform parametersonly. The E. coli parameter was not included as fecalcoliform colonies were not always identified following MFtests. Also, although results were determined for anaero-genic coliforms, Aeromonas spp., and presumptive tests, nostatistical analysis was performed. These parameters wereconsidered less important because of their lower frequencyof occurrence.

In other studies workers have used a variety of statisticaltechniques to compare the results of MF and P-A tests (2, 3,5, 8, 12, 14-16). In this study, the use of a statistical

procedure which tested whether the proportion of positivesamples from one set of bacteriological test kit results wasconstant for all of the bacteriological test kit results wasconsidered more appropriate. The procedure used to providean evaluation of the performance of the P-A and MF testswas a procedure described by Cox (6). In 1970, Cox pub-lished a treatise on the analysis of binary data (i.e., dataarranged in two classes, either present or absent).The format used in this study involved calculations of a

modified logistic transform associated with the jth ( = 1,2..., k) bacteriological test, that is, Zj = log {(rj + 0.5)/[(nj -rj) + 0.5]}, where rj is the number of positive samples and njis the total number of samples tested. The addition of 0.5 tothe numerator and denominator on the right side of theequation ensures that Zj is defined for all rj values, includingrj = 0 and rj = nj.The data for each test group were arranged in a k x 2

contingency table, with rj and nj - r, being the elements ofthe jth row of such a table. As part of the calculation, theapproximate variance of Zj was determined by using thefollowing equation: Vj = (nj + 1) (nj + 2)/nj (rj + 1) [(nj - rj)+ 11.Before the results of any two bacteriological tests were

compared, an overall statistical test for homogeneity or forthe equality of k bacteriological tests (k > 2) was made, sothat

k k

Z 21Vj - :2 l/Vj, andj = 1

where Z = (I Zj/Vj )/± 1/Vj.j=1 j=

The distribution of D is approximately a chi-square distri-bution with k - 1 degrees of freedom, and the chi-squaretable was used for determining significance. For k = 2, thistest reduced to the case of comparing two of the bacterio-logical test kit results (say i andj), so that Dji = (Zj - Zi)21(V+ vi).

Since Dji has a chi-square distribution with a single de-greeof freedom, the statistic Wii = (Zj - Zi)J/\Vj+Vi wascomputed for every pair (, i) of bacteriological tests, and theresulting data were compared with the quartiles of thestandard normal distribution to determine the significance ofpaired comparisons of the various bacteriological tests foreither total coliforms or fecal coliforms. As noted previ-ously, MF results were determined on a P-A basis for thepurpose of applying the test statistics described above.

Routine drinking water sample analysis. The Hach Vial testkit was chosen in 1991 for an additional study of drinkingwater samples obtained from municipal distribution systems.Each week, several sample submissions from various mu-nicipalities were chosen for parallel P-A analyses. A 100-mlvolume of a sample was analyzed by using the MOE P-A testmedium, and another 100 ml of sample from the same bottlewas analyzed by using the Hach Vial test kit. The results forall of the samples that produced presumptive positive testresults were confirmed by using the procedure describedpreviously (4). The parallel P-A analyses were performed inthe Central (Toronto) Laboratory, the London RegionalLaboratory, and the Thunder Bay Regional Laboratory. TheThunder Bay Regional Laboratory also had some HachDisposable test bottles that were available to perform paral-lel analyses on some drinking water samples.

APPL. ENVIRON. MICROBIOL.

I

on Novem

ber 9, 2020 by guesthttp://aem

.asm.org/

Dow

nloaded from

COMMERCIAL P-A TEST KITS 383

TABLE 1. Comparison of the number of indicator bacterial groups isolated from P-A test kits and in the MF test

No. of the following indicator groups isolated: Homogeneity tests (D)aTest No. of Type ofgroup tests test Total Fecal E. coli Anaerogenic Aeromonas Total Fecal

coliforms coliforms coliforms spp Presumptve coliforms coliforms

A 150 MOE 78 39 36 6 9 14.888b 12.060bP-AColilert 80 35 35 7 2 6Coliquik 53 27 27 7 7 11MF 83 53 c c c

B 300 MOE 143 71 60 8 1 10 0.280 1.236P-AHach 144 63 55 14 3 3DisposableHach 141 62 56 12 7 2VialMF 138 61 4

C 90 MOE 45 24 23 4 2 0.297 0.542P-AColilert 43 21 19 4 10Hach 42 20 20 3 1DisposableMF 45 21 1

D 105 MOE 57 43 41 3 1.687 0.996P-AColilert 49 40 41 4 6Hach 56 37 36 3 1 2VialMF 57 37

E 150 MOE 112 84 94 3 1 5 36.924b 15.795bP-AColilert 112 70 74 6 1 8Coliquik 76 51 60 8 19 14MF 122 77

a See text for explanation.b Significantly different at P < 0.01.c _, additional tests were not done on MF colonies to distinguish these indicator groups.

RESULTSP-A test kit evaluation. The results of the comparative

evaluation of the commercial P-A test kits are shown inTable 1 along with the results of the homogeneity tests doneon the recovery data obtained for total coliforms and fecalcoliforms for each of the five evaluation studies. Although afive-tube, three-dilution procedure was used throughout toachieve extinction of the target organisms (i.e., few or no

total coliforms in the highest dilution), the total number ofMF or P-A positive samples from each set was determinedfor comparative purposes; this was done instead of display-ing the data in an MPN format. For statistical analysis, theproportion of one set of positive samples was compared withthe proportion of another set of positive samples by using a

modified logistic transform procedure. This procedure sim-plified the overall evaluation and was used for comparisonsof levels of target organism recovery in both P-A and MFtests.The first test comparisons performed were comparisons

between the Colilert and Coliquik commercial tests and theMOE P-A and MF tests. The results shown for test group Ain Table 1 are the results obtained after inoculation of 150test bottles for each test procedure or kit over a 10-weekperiod. The MF test for total coliforms gave the highest levelof recovery; 83 MFs had one or more sheen colonies presentfollowing filtration of the inoculated dilution blanks. TheColilert test produced 80 confirmed positive total coliformresults from 95 bottles that gave yellow o-nitrophenyl-3-D-galactopyranoside reactions. The MOE P-A test produced 78

confirmed positive total coliform results from 93 presump-tive positive bottles. The Coliquik test (53 confirmed positivetotal coliform results from 78 presumptive positive bottles)gave a much lower level of recovery than the other P-A testkits; its recovery rates were about 60% for total coliformsand 50% for fecal coliforms when the data were comparedwith the data for the MF technique. The fecal coliformresults exhibited considerable variation; the MF test (53positive results) gave a much higher level of recovery thanthe other P-A test kits. Again, the level of recovery obtainedwith the Coliquik test (only 27 positive results) was muchlower than the levels of recovery obtained with the other twoP-A test kits.When the first group of results was tested for homogene-

ity, significant D values were obtained for test group A(Table 1) for both total coliforms and fecal coliforms. Whenan additional analysis of the test group A results was done ina pair-wise manner (Table 2), the low total coliform recoveryvalue obtained for the Coliquik test (53 positive results) wasvery significantly different (i.e., Wji statistic) from the otherP-A and MF test results. For the fecal coliform pair-wisecomparison, the MF result (53 positive results) was verysignificantly different (P < 0.01) from the Coliquik result (27positive results) and only significantly different (P < 0.05)from the Colilert result (35 positive results). Although thelevel of recovery of fecal coliforms in the MOE P-A test (39positive results) was only slightly higher than the levels ofrecovery obtained with the other two P-A test kits, mathe-

VOL. 59, 1993

on Novem

ber 9, 2020 by guesthttp://aem

.asm.org/

Dow

nloaded from

384 CLARK AND EL-SHAARAWI

TABLE 2. Statistical analysis of the performance of the P-A procedures

Paired comparison value (W[i) for total coliform Paired comparison value (Wji) for fecal coliformrecovery with: recovery with:

Test No. ofgroup tests~ Testlecomparedi Hach Hach Col iet Coliquk Mtet Hach HachColilert Colteqstik MF test Disposable Vial qtettest MFtest Disposable Vial

testtest test test~testtest test test

A 150 MOE P-A -0.230 2.886a -0.577 0.534 1.660 -1.742Colilert 3.110a -0.347 -2.264bColiquik _3.444a -3.338a

B 300 MOE P-A 0.408 -0.082 0.163 0.903 0.783 0.883Hach Disposable 0.490 0.245 0.201 0.100Hach Vial 0.245 0.100

C 90 MOE P-A 0.297 0.000 0.445 0.513 0.513 0.689Colilert -0.297 0.148 0.000 0.177Hach Disposable 0.445 -0.177

D 105 MOE P-A 1.098 0.000 0.138 0.421 0.848 0.848Colilert -1.098 -0.096 0.427 0.427Hach Vial -0.137 0.000

E 150 MOE P-A 0.000 4.227a -1.385 1.610 3.785a 0.807Colilert 4.227a -1.385 2.222b -0.807Coliquik -5.435a -3.009a

a Significantly different at P < 0.01.b Significantly different at P < 0.05.

matically it was not significantly different from the MF testresult.

In the second group of tests, we compared the HachDisposable bottle and Hach Vial tests with the MOE P-A andMF tests. The results for test group B (Table 1) revealed nosignificant differences among the tests for the total coliformrecovery and among the tests for fecal coliform recoverywhen either the homogeneity tests or the pair-wise compar-isons were used (Table 2). As the total coliform resultsobtained with the Hach Disposable and Hach Vial tests wereminimally different from one another, each of these tests wascompared with the Colilert test, which was clearly betterthan the Coliquik test (Table 1, test group A).

In the third group of tests (Table 1, test group C), wecompared the Colilert and Hach Disposable tests with theMOE P-A and MF tests. No significant differences wereobserved among the tests in levels of total coliform or fecalcoliform recovery. In the last series of tests done in theToronto laboratory we compared the Colilert and Hach Vialtests with the MOE P-A and MF tests. The results for testgroup D (Table 1) revealed no significant differences in levelsof total or fecal coliform recovery.Table 1 also shows the results for test group E from a

comparative evaluation done in the London laboratory. Aswe found for test group A (Table 1), the Coliquik test resultsrevealed the lowest levels of recovery of total coliforms andfecal coliforms. The values obtained were significantly lowerthan the values obtained with the other P-A or MF tests fortest group E (Table 2). The higher level of recovery of E. colithan fecal coliforms in the London laboratory resulted fromthe presence of more fluorescence-positive ECMUG brothtubes at 35°C than at 44.5°C. This type of response was notobserved as frequently in the Toronto laboratory and re-sulted in a large difference (20 E. coli confirmations) betweenthe results for the MOE P-A test and the results for theColilert test in the London laboratory.

Indicator group isolation frequencies. The isolation fre-quencies for each indicator group were determined (Table 3)at different initial response times (i.e., color change and/oreffervescence in the appropriate P-A medium within 18 to 28,48, or 72 h). The fecal coliform and E. coli parameters had

the shortest response times since the P-A test kits thatdetected these parameters produced positive results usually98 to 100% of the time with only 1 day of incubation (i.e., 18to 28 h). The MOE P-A test (96 to 98% positive results) wasslightly slower, and the percentages of fecal coliforms and E.coli that were detected within 48 to 72 h were higher with thistest than with the other P-A test kits.For the Colilert and Coliquik tests the level of detection of

total coliform isolates was 95% with response times of 18 to28 h, whereas for the Hach Disposable and Hach Vial teststhe average level of detection was 90% for the same period.The MOE P-A test was slower, having a level of totalcoliform detection for the period from 18 to 28 h of 86%; agreater percentage of total coliform isolates was detectedafter 48 to 72 h.The Coliquik test detected anaerogenic coliforms, Aero-

monas spp., and the presumptive group at higher percent-ages than the other P-A test kits. The anaerogenic coliformsand Aeromonas spp. organisms were detected more fre-quently after 48 and 72 h than after 18 to 28 h with the HachDisposable and Hach Vial tests. All P-A test kits had a smallproportion of their positive test results which could becategorized only as presumptive, and these test resultsoccurred at all of the time periods (i.e., 18 to 28, 48, or 72 h).Although the fluorescent 4-methylumbelliferyl-p-D-glucu-

ronide reaction was not an indicator organism parameter forisolation frequency, the results of this reaction were alsorecorded on the basis of response time for each of the P-Atest kits. The Colilert, MOE P-A, and Hach Vial tests hadthe highest levels of fluorescent reactions (87 to 88%) in thetime period from 18 to 28 h; the levels of response werelower in the Hach Disposable and Coliquik tests (68 to 78%).For each of the P-A test kits fluorescent responses occurredat 48 and 72 h, but less than one-half of these responses wereconfirmed as E. coli responses when the preparations weretransferred to ECMUG broth. Conversely, E. coli fluores-cent reactions occurred in confirmatory ECMUG brothtubes which had been inoculated from presumptive P-A testbottles in which no fluorescent reactions occurred during theinitial incubation period in the P-A test bottles.Response time of ECMUG broth tubes. The results of

APPL. ENVIRON. MICROBIOL.

on Novem

ber 9, 2020 by guesthttp://aem

.asm.org/

Dow

nloaded from

COMMERCIAL P-A TEST KITS 385

TABLE 3. Isolation frequencies and percentages of indicator bacteria determined according to the time required for aninitial response in the P-A test

Isolation frequencies (%) at the following P-AP-A test kit Total no. of Indicator group response times: No.atetests isolates

18 to 28 h 48 h 72 h

MOE P-A 795 Total coliforms 376 (86) 47 (11) 12 (3) 435Fecal coliforms 255 (98) 6 (2) 261E. coli 244 (96) 8 (3) 2 (1) 254Anaerogenic coliforms 13 (62) 8 (38) 21Aeromonas spp. 1 (50) 1 (50) 2Presumptive 10 (34) 13 (45) 6 (21) 29Fluorescent' 194 (87) 21 (9) 8 (4) 223

Colilert 495 Total coliforms 270 (95) 14 (5) 284Fecal coliforms 164 (99) 2 (1) 166E. coli 167 (99) 2 (1) 169Anaerogenic coliforms 13 (62) 7 (33) 1 (5) 21Aeromonas spp. 2 (67) 1 (33) 3Presumptive 16 (55) 7 (24) 6 (21) 29Fluorescenta 143 (88) 10 (6) 9 (6) 162

Coliquik 300 Total coliforms 123 (95) 4 (3) 2 (2) 129Fecal coliforms 78 (100) 78E. coli 85 (98) 2 (2) 87Anaerogenic coliforms 11 (73) 4 (27) 15Aeromonas spp. 22 (85) 4 (15) 26Presumptive 13 (52) 9 (36) 3 (12) 25Fluorescenta 51 (68) 18 (24) 6 (8) 75

Hach Disposable 390 Total coliforms 170 (91) 11 (6) 5 (3) 186Fecal coliforms 83 (100) 83E. coli 75 (100) 75Anaerogenic coliforms 12 (70) 3 (18) 2 (12) 17Aeromonas spp. 2 (67) 1 (33) 3Presumptive 2 (50) 2 (50) 4Fluorescenta 47 (78) 12 (20) 1 (2) 60

Hach Vial 405 Total coliforms 175 (89) 9 (4) 13 (7) 197Fecal coliforms 99 (100) 99E. coli 92 (100) 92Anaerogenic coliforms 9 (60) 1 (7) 5 (33) 15Aeromonas spp. 2 (25) 3 (38) 3 (37) 8Presumptive 3 (75) 1 (25) 4Fluorescenta 72 (87) 11 (13) 83

a Fluorescent reaction in P-A bottle.

inoculation of confirmatory ECMUG broth tubes (Fig. 2)with 0.05-ml inocula from positive P-A bottles are shown inTable 4. The 6-h results were usually recorded between 3:30and 4:30 p.m. and consisted of observations of growth, gas,or effervescence and fluorescence in ECMUG broth tubesthat had been inoculated between 8:30 and 9:30 a.m. on the

same day. Because equivalent numbers of confirmatoryECMUG broth tubes were not inoculated from each of theP-A test kits, the results for the different time periods wereconverted to percentages of total positive tubes.The percentages determined at 6 and 24 h were related to

the numbers of tubes that gave final responses of either gas

TABLE 4. Results of confirmatory ECMUG broth tube tests at different times and with different incubation temperatures followinginoculation from positive P-A bottles

Gas production Fluorescence

Incubation at 35°C Incubation at 44.5°C Incubation at 35°C Incubation at 44.5°C

%oftubes % oftubes ~~~% of tubes % of tubesP-A test kit producing gas No. of tubes producng gas No. of tubes go.of tubes frescenc No. of tubes

after: producing after: producing after: producing after: producinggas gas fluorescence fluorescence

6 24 48 6 24 48 6 24 48 6 24 48h h h h h h h h h h h h

MOE P-A 61 93 100 435 72 97 100 261 82 94 100 247 71 96 100 243Colilert 45 81 100 284 48 97 100 166 77 94 100 161 57 96 100 159Coliquik 28 74 100 129 28 100 100 78 59 94 100 79 43 95 100 74Hach Disposable 48 91 100 186 60 96 100 83 87 100 100 70 73 100 100 74Hach Vial 58 91 100 197 68 99 100 99 82 97 100 89 68 99 100 90

VOL. 59, 1993

on Novem

ber 9, 2020 by guesthttp://aem

.asm.org/

Dow

nloaded from

386 CLARK AND EL-SHAARAWI

TABLE 5. Comparison of the MOE P-A test with the Hach Vial and Hach Disposable bottle tests for analysis of drinking water samples

No. of the following indicator groups recovered:Laboratory No. of tests Type of test Total Fecal Anaerogenic Aeromonas

coliforms coliforms . col coliforms spp. PresumptiveToronto 1602 MOE P-A 38 8 7 3 5 11

Hach Vial 37 8 6 2 6 20London 356 MOE P-A 1 1 2

Hach Vial 1 2 1 29Thunder Bay 150 MOE P-A 3 2 3

Hach Vial 3 4 6Thunder Bay 115 MOE P-A 7 2 1

Hach Disposable 7 1 1

or fluorescence (or both) after 48 h at temperatures of 35 and44.5°C, respectively. For example, Tables 3 and 4 show thatof 795 MOE P-A tests, 435 produced ECMUG broth tubesthat were gas positive at 35°C and 247 produced tubes thatproduced fluorescence; when equivalent numbers of inocu-lated ECMUG broth tubes were incubated at 44.5°C, 261tubes produced gas and 243 tubes produced fluorescence.

Inocula from MOE P-A and Hach Vial bottles resulted inan average level of confirmed gas or effervescence responsesof 60% with the ECMUG broth tubes within 6 h at 35°C(Table 4); inocula from Colilert and Disposable test bottlesresulted in lower levels of confirmed responses (45 and 48%,respectively). At 44.5°C, the MOE P-A, Hach Disposable,and Hach Vial test bottles gave levels of confirmed re-sponses of more than 60%, but for the most part the resultsrepresented cultures that also responded at 35°C. Inoculafrom positive Coliquik tests gave the lowest level of con-firmed responses (28% for the 6-h period).

After 24 h, the MOE P-A, Hach Disposable, and HachVial tests produced confirmed test results in more than 90%of the EC medium tubes at 35°C, and the Colilert andColiquik tests produced confirmed test results in 81 and 74%of the tubes, respectively. EC medium tubes incubated at44.5°C produced confirmed positive results for fecalcoliforms in more than 95% of the tubes by the end of the24-h period for all of the P-A test kits.The fluorescent responses shown in Table 4 appeared

more quickly than the gas responses. More than 80% of theECMUG broth tubes that were incubated at 35°C andinoculated from MOE P-A, Hach Disposable, and Hach Vialtest kits were positive after 6 h. A lower level of fluorescentresponse was observed with the Colilert and Coliquik tests,but even with these two tests, the fluorescent responses (77and 59%, respectively) were better than the gas responses(45 and 28%, respectively). A comparison of the fluorescentresponses at 35°C with the fluorescent responses at 44.5°Cshowed that more fluorescent tubes occurred after 6 h at35°C. By 24 h, 94% or more of the tubes at both 35 and44.5°C had become positive, and the remainder becamefluorescent within 48 h. The numbers of fluorescent tubes at35 and 44.5°C were quite similar to each other and to thenumbers of tubes positive for gas production at 44.5°C, incontrast to the higher number of gas-positive tubes at 350C.

Drinking water sample analyses. A total of 2,108 split-sample, analytical tests were done on municipal drinkingwater samples in the Toronto laboratory and two regionallaboratories by using the Hach Vial test medium and theMOE P-A test medium. The Thunder Bay Regional Labora-tory also performed split-sample analyses by using a set of115 Hach Disposable bottles along with the MOE P-A testmedium. The results of these analyses are shown in Table 5.

No statistical analysis was performed on this data as thevariation in parameter recovery levels for total coliforms,fecal coliforms, and E. coli was very minimal for all of theparallel test analyses. The largest variation occurred with theHach Vial test, which produced more presumptive positivetests; however, none of the usual indicator groups wasisolated or confirmed. This type of result was more prevalentin the London Regional Laboratory than in the other twolaboratories.

DISCUSSION

The primary purpose of this study was to evaluate thevarious commercial P-A test kits which have been marketedrecently to detect the presence of total coliforms and E. coli.A secondary purpose of this study was to determine whetherconfirmed results for total coliforms, fecal coliforms, or E.coli could be obtained within 24 h by the MOE P-A testprocedure to make this test comparable to the Colilert andColiquik test kits, which according to the manufacturers candetect total coliforms and E. coli within 24 h. Our experi-ments were performed over a period of several months. Wecompared two test kits at a time, and the Colilert, Coliquik,Hach Disposable, or Hach Vial test was compared with theroutine MOE P-A procedure and the total coliform MF testby using m-Endo agar LES and with the fecal coliform testby using m-TEC agar.The results of the first group of evaluations (Table 1, test

group A) done in the Toronto laboratory showed that theColiquik test kit was significantly lower than the other testprocedures for recovering total coliforms (Table 2). A similarresult was obtained for test group E when a series of test(Tables 1 and 2) was performed in the London laboratory. Inthe Toronto laboratory both the Colilert and Coliquik testskits gave significantly lower values than the MF test forrecovery of fecal coliforms, but in the London laboratory,only the Coliquik test gave significantly lower levels of fecalcoliform recovery than the other test procedures. In bothlaboratories, the Coliquik test gave higher levels of presump-tive positive unconfirmed tests and Aeromonas spp. recov-ery than the other P-A test kits.Although the data are not shown in Table 1, certain trials

with the Coliquik test kit produced results equivalent to theresults obtained with other test procedures, but in othertrials, the Coliquik test gave lower levels of recovery of totaland fecal coliforms. Other studies have shown that theColiquik test is equivalent to or better than the Colilert test(3, 14), but whether another lot number would have pro-duced better levels of recovery in our laboratory was notdetermined. Because of its low level of recovery, theColiquik test was not included in other test comparisons.

APPL. ENvIRON. MICROBIOL.

on Novem

ber 9, 2020 by guesthttp://aem

.asm.org/

Dow

nloaded from

COMMERCIAL P-A TEST KITS 387

The next group of evaluations (Table 1, test group B)involved comparisons between the Hach Disposable andHach Vial test kits and the MOE P-A and MF tests. Nosignificant differences were observed between the tests fortotal coliform recovery or the tests for fecal coliform recov-ery. Consequently, in the third and fourth evaluation groupswe compared the MOE P-A and Colilert tests with the HachDisposable test kits and then with the Hach Vial test kits(Table 1, test groups C and D, respectively). The levels ofrecovery for both the total coliform and fecal coliformparameters were quite similar for all of the P-A and MF testcomparisons. Overall, when the low Coliquik test resultswere excluded, no significant difference was found betweenthe tests for total coliform recovery. A significant differencewas found with the Colilert test for fecal coliform recovery,but only in test group A in the Toronto laboratory. As aresult, the Hach Disposable, Hach Vial, and Colilert test kitshad levels of recovery that for all practical purposes wereessentially the same as the levels of recovery for the MOEP-A test for the total coliform and fecal coliform parameters.Because provisions were not made early in the study for

confirming that fecal coliform colonies on m-TEC agar wereE. coli colonies, a statistical comparison among the variousP-A test kits was not done. For the analyses performed in theToronto laboratory, most of the differences among the P-Atest kits used for E. coli recovery were minimal; the excep-tion was the Coliquik test kit. However, in the Londonlaboratory large differences between both the Colilert andColiquik test kits and the MOE P-A test were observed.Workers in the London laboratory also reported a higherlevel of recovery of E. coli with all of the P-A test kits as aresult of the fact that more ECMUG broth tubes producedgas and fluorescence at 35°C than at 44.5°C. In fact, nogrowth occurred in most of the tubes incubated at 44.5°C,even though the parallel inoculated tubes incubated at 35°Cgave good gas and fluorescent responses. This unexpectedobservation was not anticipated, and the organisms in the35°C ECMUG broth tubes were not identified more specifi-cally.As workers at the London laboratory collected their

samples mainly from previously chlorinated sewage effluentwater, we considered the possibility that injured E. coli cellswhich were not able to express their characteristic growth at44.5°C were present in these samples. Other studies haveshown that stressed organisms are not recovered efficientlyon the usual growth media, such as m-Endo agar LES, butrequire a less inhibitory medium, such as m-T7 (1). Whetherdifferences in the MOE P-A and Colilert tests were the resultof differences in medium ingredients, the sample source,or inaccurate confirmation of the presence of E. coli inECMUG broth tubes at 35°C will require further study.The isolation frequencies for the indicator bacterial groups

(Table 3) were determined according to the time required toproduce an acid reaction or an acid and foam reaction in theP-A bottle test. Although most of the initial responsesoccurred within 24 h, all P-A test kits produced presumptivepositive tests from which total coliforms and fecal coliformswere recovered after 24 to 28 h. With the test kits in whichthe P-A broth was used, it was not unusual for presumptivepositive tests to show up even after 72 h of incubation.The data in Table 3 demonstrate the need for extended

incubation periods for P-A bottles and the need for the use ofconfirmation tests for all P-A bottles giving a positive coli-form test to increase the level of recovery of E. coli. The E.coli recovery values reported in Table 3 represent the resultsfrom ECMUG broth tube tests, whereas the fluorescent

values represent what the level of E. coli recovery wouldhave been if only P-A bottle fluorescence had been used toindicate the presence of E. coli. As these values show, thefluorescent response in P-A bottles was always less than theEC medium fluorescent response in ECMUG broth tubes.Samples from which fecal coliforms and E. coli were

recovered usually produced presumptive positive P-A testsin 98% or more of the samples within 18 to 24 h. For totalcoliforms, the Colilert and Coliquik tests gave levels ofrecovery of 95% within this same time period, whereas theaverage level of recovery for the Hach Disposable and HachVial tests was 90%. The level of recovery for the MOE P-Atest was the lowest level recorded (86%), even after 28 h ofincubation. Other indicator groups, such as anaerogeniccoliforms, and Aeromonas spp. were for the most partrecovered within 48 h, although with the Hach Disposableand Hach Vial tests on several occasions these organismswere isolated after 72 h of incubation.

Colilert and Coliquik advertisements make the claim thatthese products simultaneously detect, identify, and confirmtotal coliforms and E. coli in the same container within 24 hor less, with a single inoculation and with no additionalconfirmation test. The advertising claims are essentiallycorrect when the results for a large number of samples areconsidered, but as with most biological systems, identifica-tion of organisms by using simplified procedures does notalways result in correct identification. As shown in thisstudy, 5% or more of the P-A test kits took 48 h or more toproduce an initial response, and in a number of samples onlyAeromonas spp. were detected by confirmation tests; inother samples, organisms other than typical coliforms wereresponsible for positive o-nitrophenyl-,-D-galactopyrano-side tests. If ECMUG broth tube tests had not been used, E.coli would not have been detected in a number of samples.

In an attempt to speed up the identification and confirma-tion of total coliforms, fecal coliforms, and E. coli by theMOE P-A test, confirmation tests were performed with apipetting device equipped with sterile disposable tips whichdelivered 0.05-ml inocula from the P-A bottles into ECMUGbroth tubes rather than with just loopfuls of inoculum. P-Atest preparations inoculated on the previous day either in themorning or in the early afternoon were removed from theincubator the following morning, and inocula from presump-tively positive P-A tests were transferred into ECMUGbroth tubes by 9:00 to 9:30 a.m. By 3:00 to 3:30 p.m. of thesame day, tubes that produced fluorescence at either 35 or44.5°C were considered to have E. coli present, and tubesthat produced effervescence at 35°C when they were shakenwere considered to have total coliforms present; tubes thatproduced effervescence at 44.5°C were considered to havefecal coliforms present (Table 4).

In effect, this procedure permitted confirmation of totalcoliforms, fecal coliforms, and E. coli within 24 to 28 h forapproximately 60% of the samples from which total and fecalcoliforms were ultimately isolated, and in the case of E. colithe average level of response was 80% for the test kits inwhich P-A broth was used. In the case of the Colilert andColiquik tests, the confirmation response was slower, pre-sumably because the organisms had to readjust their enzymesystems to working in a more enriched medium.Each of the EC medium tubes that produced effervescence

within 6 h contained a sizable quantity (>10%) of gas in theinverted tube by the following day; if only weak fluorescencewas detected within 6 h, much stronger fluorescence waspresent after 24 h. With 40% or more of the tubes, obviousgas production was not observed until after 24 h of incuba-

VOL. 59, 1993

on Novem

ber 9, 2020 by guesthttp://aem

.asm.org/

Dow

nloaded from

388 CLARK AND EL-SHAARAWI

tion, and with fluorescence about 20% or more of the tubeswere not sufficiently fluorescent to be detected until after 24h. Surprisingly, fluorescence was detected earlier when thepreparations were incubated at 35°C than when they wereincubated at 44.5°C (Table 4).

Indicator organisms were detected and recovered muchless frequently from drinking water samples than from thediluted river and sewage samples which were used in the firstpart of the study. However, except for more frequentpresumptive positive results with the Hach Vial test, bothHach test kits recovered indicator organisms in drinkingwater samples in numbers almost identical to the numbersrecovered by the MOE P-A test procedure. The results forparallel samples were not always identical, but the overalllevels of recovery of indicator organisms in the test proce-dures were similar.Another consideration which should be taken into account

when a P-A test kit is chosen for routine use is the cost pertest. When our evaluation tests were done in 1990, the cost(in Canadian dollars) of the Access Analytical SystemsColilert test was about $8.00 to $9.00 per test depending onthe quantity ordered, and the cost of the Coliquik test wasabout $7.00 per test. The costs of the Hach Disposable andHach Vial tests were about $4.00 and $2.00, respectively.When the MOE P-A test was made up in-house by using P-Abroth medium, the cost was about $1.00 per test. Increasedusage and production of these test kits may lower their costsin the future.Although the Colilert and Coliquik test kits may be easier

to use than the P-A test kits containing a lactose-basedmedium, the former are more expensive and do not have theflexibility for detecting and isolating other water qualityindicator groups. Besides total coliforms and E. coli, thelactose type of P-A tests can detect the potential presence ofother indicator organisms, such as Aeromonas spp., Staph-ylococcus or Micrococcus spp., Clostridium perfringens,and fecal streptococci (4). Aeromonas spp. and Staphylo-coccus or Micrococcus spp. often produce a distinctiveoff-yellow color in presumptive positive P-A bottles. Famil-iarization with this color reaction or the bright yellow colorand foam reaction that occur in the presence of totalcoliforms or E. coli can often assist a technician or operatorto speculate on the indicator group that will be identified byconfirmatory tests.The presence of total coliforms, fecal coliforms, or E. coli

is well recognized as an indication of unsafe or poor waterquality for which corrective measures should be taken.These measures include increased chlorination or watermain flushing or a combination of both of these actions. Lesswell recognized is the possibility that an increased frequencyof presumptive positive tests and/or the isolation of Aero-monas spp., Staphylococcus spp., and other indicator or-ganisms is a sign that the water quality is deteriorating eventhough members of the coliform group have not been de-tected. Although confirmatory tests are required to detectand differentiate these other indicator organisms, these testsare relatively simple and easy to perform (4). The additionalinformation provided by these tests should help operators ofwater works to determine when simple corrective measuresshould be initiated before a serious water quality incidentoccurs. Otherwise, more costly and embarrassing correctivemeasures, such as the measures that accompany the issu-ance of a boil water order and public notification of unsafewater quality, could be necessary.

ACKNOWLEDGMENTS

We thank Colleen Cotter and the staff of the Thunder BayRegional Laboratory and Bill Kutas and the staff of the LondonRegional Laboratory for performing regional laboratory analyses forthis study. We thank Steve Debreceni, Rosa Lee, Ingrid Bon-kowski, and the staff of the Central Laboratory for their assistancewith analytical tests and preparation of the manuscript.

REFERENCES

1. American Public Health Association. 1989. Standard methods forthe examination of water and wastewater, 17th ed. AmericanPublic Health Association, Washington, D.C.

2. Bancroft, K., E. T. Nelson, and G. W. Childers. 1989. Compar-ison of the presence-absence and membrane filter techniques forcoliform detection in small, nonchlorinated water distributionsystems. Appl. Environ. Microbiol. 55:507-510.

3. Clark, D. L., B. B. Milner, M. H. Stewart, R. L. Wolfe, andB. H. Olson. 1991. Comparative study of commercial 4-methyl-umbelliferyl-13-D-glucuronide preparations with the StandardMethods membrane filtration fecal coliform test for the detec-tion of Escherichia coli in samples. Appl. Environ. Microbiol.57:1528-1534.

4. Clark, J. A., C. A. Burger, and L. E. Sabatinos. 1982. Charac-terization of indicator bacteria in municipal raw water, drinkingwater and new main water samples. Can. J. Microbiol. 28:1002-1013.

5. Covert, T. C., L. C. Shadix, E. W. Rice, J. R. Haines, and R. W.Freyberg. 1989. Evaluation of the autoanalysis Colilert test fordetection and enumeration of total coliforms. Appl. Environ.Microbiol. 55:2443-2447.

6. Cox, D. R. 1970. The analysis of binary data. Methuen & Co.Ltd., London.

7. Dufour, A. P., E. R. Strickland, and V. J. Cabelli. 1981.Membrane filter method for enumerating Escherichia coli. Appl.Environ. Microbiol. 41:1152-1158.

8. Edberg, S. C., M. J. Allen, D. B. Smith, and the NationalCollaborative Study. 1989. National field evaluation of a definedsubstrate method for the simultaneous enumeration of totalcoliforms and Escherichia coli from drinking water: comparisonwith presence-absence techniques. Appl. Environ. Microbiol.54:1003-1008.

9. Edberg, S. C., and M. M. Edberg. 1988. A defined substratetechnology for the enumeration of microbial indicators of envi-ronmental pollution. Yale J. Biol. Med. 61:389-399.

10. Federal Register. 1989. Drinking water: national primary drink-ing water regulations; total coliform proposed rule. Fed. Regist.54:27544-27567.

11. Feng, P. C. S., and P. A. Hartman. 1982. Fluorogenic assays forimmediate confirmation of Escherichia coli. Appl. Environ.Microbiol. 43:1320-1329.

12. Jacobs, N. J., W. L. Zeigler, F. C. Reed, T. A. Stukel, and E. W.Rice. 1986. Comparison of membrane filter, multiple-fermenta-tion-tube, and presence-absence techniques for detecting totalcoliforms in small community water systems. Appl. Environ.Microbiol. 51:1007-1012.

13. Lewis, C. M., and J. L. Mak. 1989. Comparison of membranefiltration and autoanalysis Colilert presence-absence techniquesfor analysis of total coliforms and Escherichia coli in drinkingwater samples. Appl. Environ. Microbiol. 55:3091-3094.

14. Olson, B. H., D. L. Clark, B. B. Milner, M. H. Stewart, andR. L. Wolfe. 1991. Total coliform detection in drinking water:comparison of membrane filtration with Colilert and Coliquik.Appl. Environ. Microbiol. 57:1535-1539.

15. Pipes, W. O., H. A. Minnigh, B. Moyer, and M. A. Troy. 1986.Comparison of Clark's presence-absence test and the membranefilter method of coliform detection in potable water samples.Appl. Environ. Microbiol. 52:439-443.

16. Rice, E. W., E. E. Geldreich, and E. J. Read. 1989. Thepresence-absence coliform test for monitoring drinking waterquality. Public Health Rep. 104:54-58.

APPL. ENVIRON. MICROBIOL.

on Novem

ber 9, 2020 by guesthttp://aem

.asm.org/

Dow

nloaded from