Environmental Mutagenesis 4595-603 (1982)

Experimental Design and Statistical Analysis Considerations for In Vitro Mammalian Cell Transformation Assays With BALB13T3 Cells Elbert B. Whorton, Jr., Jonathan B. Ward, Jr., and Debra L. Morris

Division of Environmental Toxicology, Department of Preventive Medicine and Community Health, University of Texas Medical Branch, Galveston

Many mammalian cell assays testing for mutagenic activity have common features which cause statistical estimation and analysis problems. Such assays measure the number of cell alterations occurring in a plate containing an unknown number of cells at risk. The number of cells at risk can be estimated from a parallel cytotoxici- ty study. While the Poisson distribution has been assumed to apply to standardized frequencies, this is questioned. The failure of standardized frequencies to follow a Poisson distribution is attributed to the relatively small and dosage-dependent number of susceptible cells per plate. A minimum number of such cells per plate or random cluster of plates has been determined for each dose so that the measured variable approximates a Poisson distribution. A transformation is suggested to achieve reasonable normality and variance equality, thereby allowing the use of parametric analysis of variance and regression methods and an estimation of re- quired sample sizes.

Key words: mammalian cell transformation assay, distribution, statistical analysis, sample size, sensitivity

INTRODUCTION

A number of test systems using mammalian cells in culture have been developed in order to assess the mutagenic or carcinogenic potential of suspect agents. It has been observed repeatedly that data resulting from the application of these assays do not fol- low those distributional forms which might otherwise have been expected. Consequent- ly, those mathematical transformations commonly used to achieve approximate nor- mality and variance equality have not been effective.

This paper was presented at the 1981 Annual Conference of the Environmental Mutagen Society, San Diego, California; the 1981 Joint Statistical Meetings of the American Statistical Association and Biometric Society, Detroit, Michigan; and the 1981 Conference of Texas Environmental Mutagen Society, Galveston, Texas.

Received December 21, 1981; revised and accepted May 27, 1982.

Address reprint requests to Elbert B. Whorton, Jr., Department of Preventive Medicine and Community Health, 140 Keiller Building, University of Texas Medical Branch, Galveston, TX 77550.

0192-2521/82/0405-0595$03.00 0 1982 Alan R. Liss, Inc.

5% Whorton, Ward, and Moms

Irr and Snee [1979, 19811 concluded that data from mutagenicity in Chinese hamster ovary cells at the hypoxanthine-guanine phosphoribosyl transferase locus (CHO/HGPRT) obtained from several experiments could not be adequately described using the Poisson distribution. Consequently, they derived a data-specific mathemati- cal transformation to achieve reasonable normality and variance equality when applied to the standardized mutation frequency per plate. They noted that while their transfor- mation may not apply in other labs, similar procedures could be used to determine such a transformation for each lab. While nonparametric methods can also be used for statis- tical analyses, such procedures are not always applicable to the more complex designs and questions concerning adequate sample size and experimental sensitivity remain.

In studying data in our lab resulting from in vitro mammalian cell transformation assays, we also observed that the plate to plate variation of standardized or corrected frequencies increased more rapidly than did the mean with larger dosage (Table I); thus, the Poisson assumption again appears untenable. Characteristically, nearly 55 Yo of the plate results in the control dose equaled zero frequency; and this percentage de- creased with larger doses. Cytotoxicity increased with dosage since the proportion of cells treated which survived (SP) decreased with dosage.

In order to gain some understanding of why such data were not distributed ac- cording to the expected Poisson distribution, we first postulated an underlying binomial distribution and subsequently investigated those sampling distribution condi- tions which if met would allow: (1) a Poisson distribution to be used to approximate closely an actual binomial distribution; and (2) the use of a transformation commonly applied to Poisson variates to achieve reasonable normality and equal plate to plate variance.

It was felt that if significant departures from such required conditions could be identified and then met through modifications in experimental design, commonly used parametric statistical methods could perhaps be applied in each laboratory.

The result of this research is the subject of this report. Also, an example illustrating the application of the methodology is given using data obtained from four studies of the neoplastic transformation of BALB/3T3 cells using five doses of 3-methylcholan- threne (3-MCA) and a solvent control.

BINOMIAL DISTRIBUTIONS AND POISSON APPROXIMATIONS

If p is the probability of an alteration, the probability distribution o f f alterations in a sample of n cells at risk has the binomial distribution

f = O,l,. . . . . . . ,n

The mean of this distribution is np and its variance is np(1-p). In considering this binomial distribution it should be noted that n and pare both assumed to be known and constant for each sample (plate).

In many mammalian cell transformation assays the value p tends to be very small. If the number of cells at risk (n) per sample (plate) is very large, equation 1 can be closely approximated by its limiting form, the Poisson distribution given by:

e-np(np)f P ( fnp )= ~- f! f = 0,1,2,. . . . . . .

Mammalian Cell Transformation Assays 597

TABLE 1. Summary of Results Based on Individual Plates*

Dosage Plate results 0 0.1 0.3 1 .O 3.0 10.0

No. of plates (np = lo4) 42 36 40 42 40 36 Plating efficiency (SP) 0.3374 0.2661 0.2040 0.1579 0.1271 0.1087 Cells at risk per plate (n,) 3314 266 1 2040 1579 1271 1087 p, - proportion of n, with foci 0.0002 O.OOO4 O.OOO9 0.0015 0.0026 0.0052 Mean f per plate (f) 0.667 0.946 1.778 2.335 3.251 5.619 Mean fa per plate (fa) 1.98 3.56 8.71 14.79 25.58 51.70 Variance of fa among platesa 6.93 12.70 67.75 156.87 259.59 476.38

*f = observed number of foci; f = rate per n, cells at risk; fa = corrected number of foci; Fa = rate per assumed 10' cells at risk; p, = f/npSP, = f/n,. a Combined within-study variance.

In the more familiar expressions for a Poisson distribution, the value corresponding to np in equation 2 is presented as a single parameter with no explicit mention of n or p. However in the Poisson approximation to the binomial distribution, this constant is ex- plicitly expressed as the product of n and p and is an important consideration in such assays. Under these circumstances the Poisson distribution is such that its mean, np, is larger than, but nearly equal to, its variance, np(1-p). This is so since when p is near zero, the value (1 - p) is near unity.

PLATE TO PLATE VARIATION In the usual case of multiple doses, it is of importance to assess the dose-related

changes in the proportion (p) of cells at risk which become altered. When the values of p indeed change across dose levels, it is necessary that the number of cells at risk (n) re- main constant when the results are to be based solely on observed frequencies. Other- wise the expected value of the observed frequencies (np) will vary, not only because of changes in p but also because of differences in n. The procedure used in current practice to overcome this unfortunate inequality in the cells at risk due to dose-related changes in cytotoxicity is simply by mathematically reexpressing every observed plate frequency (f) as the corresponding corrected frequency (f.& which would be expected to occur if some arbitrary constant number of cells at risk (N) had actually been contained in every plate in every dose. These corrected frequencies for dose (i) can be calculated as

where N is the arbitrary constant, np is the number of cells plated per plate and SPi is the survival proportion or plating efficiency in dose (i). The product niSPi is the estimated number of cells actually at risk per plate in dose (i). While perhaps logical in practice, we find that such mathematical correction procedures have been largely responsible for the unexpected and rapid increases in observed plate to plate variation with increased dosage. That is, it can be shown that when the original variable (0 has

598 Whorton, Ward, and Morris

mean and variance both equal to nipi for dose (i), the new variable fa in equation 3 has mean equal to

[&] "iPi

and variance equal to

[GI' n i P i

(4)

When N > npSPi the increase in the variance is larger than the corresponding in- crease in the mean. Only when N = npSPi will the mean equal the variance in each dose; and, as seen earlier, only when ni = n for every plate and dose will the dosage results expected depend only on pi. Consequently, for the Poisson approximation to maintain its expected equality of mean and variance per dose and to eliminate changes in nipi which are not due to changes in pi, it is necessary that

n P ' S P . = N = n . = n (6)

be the number of cells at risk per dose and plate. Clearly, this condition cannot be met when the number of cells plufed per plate remains the same in each dose unless the sur- vival fractions in each dose are also identical. However, due to cytotoxicity these sur- vival fractions decrease with increased dose. Thus, in order for equation 6 to be true, different numbers of cells must be plated per plate in each dose(i) so that the relation- ship

n "pi = spi (7)

is satisfied. Alternatively and perhaps preferably a number of small plates of constant size np,

can easily be randomly combined to produce plategroups each containing a total of ap- proximately npi plated cells. Analyses then can be easily conducted using the plate group results as illustrated in the example.

DETERMINATION OF THE NUMBER OF CELLS AT RISK

While it is true that the mean and variance are equal in Poisson distributions, it is not true that the commonly used square root transformation stabilizes plate variation regardless of the size of the mean, np [Kendall and Stuart, 1976; Natrella, 1963; Freeman and Tukey, 19501. For np > 10, the square root does stabilize variation near its expected value, 0.25; while when np < 10 the variance can range from 0 to 0.40. However the use of the average square root (ASR) transformation, %(Jf + Jf + 1 ) will result in variances being stable at near the expected 0.25 even when np > 1. If one selects n such that n = 2/p and the ASR transformation is used prior to statistical analysis, nearly equal variances and approximate normality are achieved. The value for p should be estimated using conrrol data and

~

Mammalian Cell hnsformation Assays 599

T p + - npSP

where f is the mean frequency per plate. Convenient tables for using the ASR transfor- mation and for determining the reverse transformations are given by Mosteller and Youtz [1961].

PLATE VALUES OF ZERO FREQUENCY

The proportion of zero value plates in the control dose can be predicted as

(1 - p)"PSP & (1 - p)" (9)

When n = 2/p as desired, equation 9 becomes

(1 - p)2/P t e-2 & 0.135

and therefore only about 13.5% of all control plates or plate groups would be expected to equal zero. The probability of observing a zero plate can be reduced simply by in- creasing n. Since the smallest value of p will normally occur in the control, this group should experience the largest fraction of zero value plates. This can serve as a quick visual check on the adequacy of n in a given experiment.

REQUIRED NUMBER OF PLATES PER DOSE

When n = 2/p per plate in each dose and the ASR transformation is used for analysis, the between plate variance of 0.25 can be applied to procedures described in Snedecor and Cochran [1973] to estimate the number of plates or plate groups per dose necessary to achieve predetermined experimental sensitivity.

For example, when p = 0.0002 and n = 2/p = 1oo00, the mean rate per 10,000 cells at risk is np = 2. Thus, an estimated

plate groups per dose are needed to have probability equal to 0.90 of detecting an in- crease from 2 to 4 (or 1.573 to 2.118 on the transformed scale) at ct = 0.05.

TRANSFORMATION ASSAY The cell line BALB/3T3 clone A31-1-13 was provided by Dr. T. Kakunaga and is

very similar to the well characterized subclone A31-174. Cultures were maintained in Basal Medium Eagle with Earle's salts, 1 mM nonessential amino acids and 10% fetal calf serum. Cells were subcultured twice weekly and maintained at cell densities low enough to minimize cell to cell contact. Individual cryopreserved cultures were used within 10 subcultures after thawing to avoid the accumulation of spontaneous transfor- mants. All cultures were tested for mycoplasma by a fluorescent staining technique [Chen, 19771.

The transformation assays were conducted using focus formation in confluent monolayers as the primary endpoint [Kakunaga, 19731. The basic protocol involved

600 Whorton, Ward, and Moms

plating 10,000 cells in 50 mm-diameter dishes, and treating these cells with 3-methylcholanthrene (3-MCA) at five concentrations plus control 24 hr after the in- itial plating. Dimethyl sulfoxide was used as a solvent with a final concentration of 0.3%. Cultures were then rinsed with saline after a 24-hr treatment and refed with fresh medium. Thereafter, they were refed twice weekly for four weeks. Parallel determina- tions of colony-forming efficiency in plates seeded with 250 cells and treated at the same time as the plates for focus formation were also made. Generally, ten focus for- mation plates and five colony-forming plates were treated at each dose in each assay. At the end of four weeks, the monolayer cultures were fixed with 10% formalin and stained in 5% Giemsa. Foci were examined for three characteristics: (a) fibroblastic cell morphology, (b) random criss-crossed cell arrangement, and (c) migration of cells from the focus over the background monolayer. Only those foci most stringently meeting these criteria were scored as transformations. The data used in this study were from the accumulated results of four replicate experiments.

RESULTS

To illustrate the application of the methodology as it relates to mammalian cell transformation assays, data from the four smaller studies were analyzed. There were 42,36,40,42,40, and 36 focus formation plates used in doses 0,O. 1,0.3, 1.0,3.0, and 10.0 pg/ml of 3-MCA respectively (Table I). Each plate contained loo00 plated cells ("4. Using the plating efficiencies (SP) per dose, the estimated number of cells at risk ranged from 3374 in the control to 1087 in dose 10.0. Since the mean number of foci per plate (7) ranges from 0.667 to 5.619, the corresponding proportions (p) of cells at risk with foci are estimated as f/npSP and range from 0.0002 to 0.00052. When the COT-

rected number of foci (7,) per assumed N = loo00 was derived using plating efficien- cies, these means ranged from 1.98 to 5 1.70 and their corresponding plate variances ranged from 6.93 to 476.38. Therefore, while this rapid increase in variation is characteristic of corrected mammalian cell transformation results, it is not typical of results expected from Poisson distributions.

Since in the control, P = 0.0002, our findings would suggest that at least n = 2/0.0002 = loo00 cells at risk were needed per plate in each dose so that (1) the Poisson assumption would hold true and (2) the suggested transformation would stabilize variances at 0.25. Clearly this condition is not met in these data.

To achieve n = loo00 cells at risk per plate per dose, 29638,37580,49020,6333 1, 78678, and 91996 cells should be plated per plate in doses 0 to 10.0, respectively (equa- tion 7). The existing plates were randomly combined so that the required number of plated cells would be approximately obtained per plate grouping or cluster. Three plates per cluster were needed for the control to obtain the 14 clusters of 3oooO plated cells each (Table 11), while nine plates per cluster were needed to form the 4 clusters of 90000 plated cells in dose 10.0. The expected number of cells at risk per cluster per dose is shown to be reasonably consistent at near loo00 cells. Table 111 shows the detailed results for clusters in the control. It is noted that in the 42 control plates there were 54.8% zeros, while in the 14 random clusters there were only 7.1%. This compares favorably with their expected values of 51% and 13.5%, respectively, based on a postulated binomial distribution. When the cluster results were analyzed, the control mean frequency per n = loo00 cells at risk was 2.00 and the variance was 1.85. Further, the means and variances are seen to be very similar for each dose reaching a mean of

TA

BL

E 11

. Sum

mar

y o

f R

esul

ts B

ased

on

Plat

e G

roup

Clu

ster

s

Dos

age

Clu

ster

res

ults

0

0.1

0.3

1 .o

3.0

10.0

No.

of

plat

es

42

36

40

42

40

36

Plat

es p

er c

lust

er

3 4

5 6

8 9

z C

lust

ers

per

dose

14

9 8

7 5

4 C

ells

pla

ted

per c

lust

er (n

p,)

3oooo

4000

0 5oooo

60000

goo00

9000

0

3 r

Mea

n fa p

er c

lust

er (

f)

2.00

3.78

8.88

14.00

26.00

50.50

I' E

Plat

ing

effic

ienc

y (S

P,)

0.3374

0.2661

0.2040

0.1579

0.1271

0.1087

Cel

ls a

t ris

k pe

r clu

ster

(n,)

10122

1064

10200

9564

10168

9783

Var

ianc

e o

ff b

etw

een

clus

ters

1.85

6.19

10.70

16.33

16.50

45.67

Var

ianc

e of

X b

etw

een

clus

ters

I .80

5.47

10.28

18.19

15.97

47.71

3

Mea

n X

b pe

r cl

uste

r 1.98

3.55

8.77

14.77

25.58

51.62

2 - 9 M

ean Yc

per

clus

ter

1.49

1.93

2.99

3.87

5.09

7.21

3

Var

ianc

e o

f Y

betw

een

clus

ters

0.21

0.34

0.28

0.31

0.16

0.31

z3 e

af =

sum

of i

ndiv

idua

l pla

te fo

ci p

er c

lust

er.

3 s z 3 b

602 Whorion, Ward, and Moms

TABLE 111. Calculations to Obtain Analysis Variable (Y) for Dose 0

Plate group Observed foci f Xa Yb

1 2 3 4 5 6 7 8 9

10 I 1 12 13 14

1 3 1 1 2 2 2 3 1 4 5 0 2 1

0.988 2.964 0.988 0.988 1.976 1.976 1.976 2.964 0.988 3.952 4.940 0 1.976 0.988

1.202 1.856 1.202 1.202 1.565 1.565 1.565 1.856 1.202 2.107 2.330 0.500 1.565 1.202

% Zero 54.8 7.1 Mean 2.00 1.98 1.49 Variance 1.85 1.80 0.21

50.50 and variance of 45.67 at dose 10.0. When the slight correction to n = loo00 cells at risk is used to obtain X values (Table III), the corresponding means and variances re- main essentially unchanged. Applying the average square root transformation the resulting values (Y) have means and variances as shown (Table 11). Each variance is near the expected value of 0.25, and no increasing or decreasing trend is apparent. Based on these findings the cluster data appears to follow an approximate Poisson dis- tribution, the ASR transformation stabilizes the between-cluster variation, and approximate normality may be assumed [Kendall and Stuart, 19761.

Analysis of variance procedures based on transformed data were used to analyze the results (Table IV), and it is seen that the doses produced significantly different ef- fects (P < 0.001) as expected. Further, the overall variance between clusters equals 0.257, nearly identical to its expectation, 0.25.

CONCLUSIONS

While a number of studies have suggested that corrected results from mammalian cell transformation assays do not follow expected Poisson distributions, results of this research have shown that such outcomes arise primarily due to insufficient numbers of cells at risk per plate and because the actual number of cells at risk are not constant per plate in each dosage level. We have suggested procedures to allow one with accurate knowledge of only the control focus formation rate and control plating efficiency to determine the minimum number of cells at risk per plate (or cluster) so that observed re- sults will follow an approximate Poisson distribution, and to ensure that there are few or no focus formation values equal to zero. The procedure is extended to require only the remaining cytotoxicity results to determine the number of plates to combine at random

Mammalian Cell Transformation Assays 603

TABLE IV. Analysis of Variance Results for Variable Y

Source of Degrees of Mean variation freedom square F-ratio P value

Total 46 Between doses 5 28.014 109.004 < .001 Within dosesa 41 0.257

=Overall variance among plate group values of Y.

-

so that the minimum number of cells at risk per cluster is obtained and maintained at near constancy regardless of dose. A particular mathematical transformation is sug- gested so that the observed Poisson variates are converted to variables having equal and known variances and follow an approximate normal distribution. Analysis of variance, regression and other applicable parametric statistical procedures can then be used in these mammalian cell transformation type test systems. Finally the number of clusters which should be included in each dose to ensure a reasonable probability of detecting meaningful differences can be estimated.

ACKNOWLEDGMENTS

This work was supported in part by grants from the U.S. Department of Energy, grant DE-ASO-5-78-EV06023; the U.S. Environmental Protection Agency, grant R806194-01; and the Electric Power Research Institute, grant RP 1638-1.

REFERENCES

Chen TR (1977): In situ detection of mycoplasma contamination in cell cultures by fluorescent Hoechst

Freeman MF, Tukey JW (1950): Transformations related to the angular and the square root. Ann Math

Irr JD, Snee RD (1979): Statistical Evaluation of Mutagenicity in the CHO/HGPRT System. In “Ban-

Kakunaga T (1973): A quantitative system for assay of malignant transformation by chemical carcinogens

Kendall M, Stuart A (1976): “The Advanced Theory of Statistics.” New York: Hafner Press, vol3. Mosteller F, Youtz C (1961): Tables of the Freeman-Turkey transformations for the binomial and Poisson

Natrella MG (1963): “Experimental Statistics.” National Bureau of Standards Handbook 91, Washington,

Snedecor GW, Cochran WG (1973): “Statistical Methods.” 6th ed. Ames, Iowa: Iowa State University

Snee RD, Irr JD (1981): Design of a statistical method for the analysis of mutagenesis at the HGPRT locus

33258 stain. Exp Cell Res 104:255-262.

Stat 22:607-611.

bury Report 2.” New York: Cold Spring Harbor, pp 263-275.

using a clone derived from Balb/3T3. Int J Cancer 12:463-473.

distributions. Biometrika 48:433.

D.C.: U.S. Government Printing Office, Chap 20.

Press, pp 111-116.

of CHO cells. Mutat Res 85:77-93.