STATISTICS IN MEDICINEStatist. Med. 2005; 24:1449–1452Published online in Wiley InterScience (www.interscience.wiley.com).

LETTER TO THE EDITOR

A note on non-parametric ANCOVA for covariate adjustment in randomizedclinical trials

by Emmanuel Lesa�re and Stephen Senn, Statistics in Medicine 2003; 22:3583–3596

From:Xun ChenRY34-A316, Clinical BiostatisticsMerck Research LabRahway, NJ 07065, U.S.A.

Lesa�re and Senn investigated the application of Koch’s sampling-based non-parametriccovariate adjustment method [1] in their recent publication in Statistics in Medicine. Theycontrasted the method to the classical covariate adjustment method ANCOVA and found thatwhen two treatment groups have the same sample size, the estimated vector obtained from theKoch’s method is the same as that obtained from the ANCOVA method, yet the Koch’s teststatistic for the treatment e�ect is never less than and nearly always larger than the test statisticobtained from the ANCOVA. They thus proposed a correction factor to the Koch’s test statis-tic (to make it identical to that of the ANCOVA method) to preserve its probability of typeI error in the cases of small to moderate sample sizes.This letter will revisit the Koch’s sampling-based covariate adjustment method. Our con-

clusions, however, will be somehow di�erent from what Lesa�re and Senn observed in theirrecent publication.As illustrated in Lesa�re and Senn, the Koch’s sampling-based covariate adjustment method

can be formulated as follows. Assume the di�erence of the response (y) between standard(S) and experimental (E) treatments is � in population average. Assume the covariates areall (asymptotically) balanced, which is a reasonable assumption for any randomized clinicaltrials. A linear model (

�ye − �ys�̃xe − �̃xs

)=(10

)�+

(�y�̃x

)

thus follows directly. Let(�yy V ′

yxVyx Vxx

)be a consistent estimate for Var

( �ye− �ys�̃xe− �̃xs

). The Koch’s method

estimates � in a weighted least-square manner, that is,

�̂= ( �ye − �ys)− V ′yxV

−1xx ( �̃xe − �̃xs)

and following a large sample approximation, the Koch’s method estimates the variance of �̂by �yy − V ′

yxV−1xx Vyx. It has been well known that the proposed variance estimator potentially

underestimates the true variance of �̂, especially when sample size is small, as it takes noaccount for the variability of Vyx and Vxx. But does the Koch’s method really derive the same

Copyright ? 2005 John Wiley & Sons, Ltd.

1450 LETTER TO THE EDITOR

estimates as the ANCOVA and consequently, should we adjust the inference of the Koch’smethod based on the ANCOVA standard?To simplify the illustration, we assume that there are m subjects in each treatment group

with 2m= n. We also assume there is only one covariate considered in the adjustment (usuallythe baseline of the response) and the covariate is centered with �xe + �xs=0. The key di�er-ence between our derivation and the Lesa�re and Senn’s derivation is on the estimation ofVar(�xe − �xs), that is, what should Vxx be in the Koch’s procedure?Lesa�re and Senn considered the conventional estimator for the variance of two sample

di�erence:

Vxx=1

m(m− 1)m∑i=1[(xei − �xe)2 + (xsi − �xs)2] (1)

They neglected, however, the assumption of E(xe)=E(xs) in the Koch’s method. Thisassumption should be applied in derivation throughout. Consequently, a more e�cient es-timate for Var(�xe − �xs) in Koch’s procedure should be

Vxx =1

m(m− 1)m∑i=1

[(xei − �xe + �xs

2

)2+(xsi − �xe + �xs

2

)2]

=1

m(m− 1)m∑i=1(x2ei + x

2si) =

Vxx−LS1− � (2)

where Vxx LS represents the Lesa�re and Senn version of Vxx and the parameter

�=2m �x2s∑m

i=1 (x2ei + x2si)

as in Lesa�re and Senn. One can easily verify the above judgment by running any generalizedleast square based statistical procedure, e.g. Proc Mixed in SASJ. Note that the parameter �measures the imbalance of covariates between the two treatment groups and it changes from0 to 1 + the more unbalanced the baseline covariates, the closer � would approach 1; themore balanced the baseline covariates, the closer � would approach 0.Now that the Koch’s estimate for � should be

�̂KOCH = ( �ye − �ys)− (1− �)V ′yxV

−1xx LS(�xe − �xs)

which is di�erent from the ANCOVA estimate

�̂ANCOVA = ( �ye − �ys)− V ′yxV

−1xx LS(�xe − �xs)

It can be shown that �̂KOCH is a consistent estimator for �, i.e. �̂KOCHP−→� if � → 0

as m→ +∞ (which is equivalent to E(xe)=E(xs)). But as most of the generalized least-squares estimates, �̂KOCH is usually not unbiased. Note �̂ANCOVA is both unconditionally andconditionally unbiased. The bias of �̂KOCH can thus be expressed as E[�V ′

yxV−1xx LS(�xe − �xs)]. It

clearly tells us the Koch’s method cannot be applied when randomization has not taken place+ just as Lesa�re and Senn stated in their paper.Similarly, the variance of �̂KOCH should estimated by

V̂ar(�̂KOCH)= �yy − (1− �)V ′yxV

−1xx LSVyx

Copyright ? 2005 John Wiley & Sons, Ltd. Statist. Med. 2005; 24:1449–1452

LETTER TO THE EDITOR 1451

(following the Koch’s procedure) rather than the �yy − V ′yxV

−1xx LSVyx as in Lesa�re and Senn.

In contrast, the variance of �̂ANCOVA is estimated by

V̂ar(�̂ANCOVA)=�yy − V ′

yxV−1xx LSVyx

1− �n− 2

n− p− 2

(p=1 here in our illustration) as shown in Lesa�re and Senn. We can see V̂ar(�̂KOCH) isalways greater than �yy − V ′

yxV−1xx LSVyx unless �=0 (when these two are equal), and

V̂ar(�̂KOCH)− V̂ar(�̂ANCOVA)≈ vyy1− � [((1− �)− (1− �)2�2yx)− (1− �2yx)]

=�vyy1− � (2�

2yx − ��2yx − 1)

with (V 2yx=vyyVxx LS)≈�2yx61. We have known that the Koch’s estimate is only valid when� closes to 0. It follows that V̂ar(�̂KOCH) would be generally less than V̂ar(�̂ANCOVA) if�yx¡0:7, and (when � approaches 0) V̂ar(�̂KOCH) may be slightly greater than V̂ar(�̂ANCOVA)when �yx¿0:75.We have known that the Koch’s procedure underestimates Var(�̂KOCH) especially when

sample size is small. It is thus not very clear for us that V̂ar(�̂KOCH)¡V̂ar(�̂ANCOVA) when�yx¡0:7 is mostly because of the underestimation or because of the e�ciency of the Koch’smethod. But we should note that, the Koch’s method could be more e�cient than the AN-COVA method for the analysis of randomized clinical trial under certain conditions as itimposes some additional assumptions, say, E(xe)=E(xs), throughout its derivation. (On theother hand, of course, this has also limited the application of the Koch’s method beyondrandomized clinical trials.)Since ANCOVA cannot serve as a standard for the Koch’s method, the correction for

V̂ar(�̂KOCH) in small sample cases is not straightforward but not infeasible. One may refer toReference [2] for details.Finally, we want to point out that neither Koch et al. nor Lesa�re and Senn explored

the power performance of the Koch’s method. The Koch’s method was referred as a non-parametric method as it does not require assumptions concerning the distributions of theresponse y and the covariate x and the relationship between them. However, we should notethe rationale of the Koch’s non-parametric method is based on the generalized least squaresprocedure. It is well known that (generalized) least-squares estimates generally perform verypoor (in power) when the distribution of response variable severely departs from normality.We thus question the e�ciency of the Koch’s non-parametric method as compared with othernon-parametric ANCOVA procedures, for example, the rank based method [3] and the M-estimate based method [4]. This of course, could be a topic of future investigation.

REFERENCES

1. Koch GG, Tangen CM, Jung JW, Amara IA. Issues for the covariance analysis of dichotomous and orderedcategorical data from randomized clinical trials and non-parametric strategies for addressing them. Statistics inMedicine 1998; 17:1863–1892.

2. Kenward M, Roger J. Small sample inference for �xed e�ects from restricted maximum likelihood. Biometrics1997; 53:983–997.

Copyright ? 2005 John Wiley & Sons, Ltd. Statist. Med. 2005; 24:1449–1452

1452 LETTER TO THE EDITOR

3. McKean JW, Hettmansperger TP. Test of hypotheses of the general linear model based on ranks. Communicationsin Statistics, Part A: Theory and Methods 1976; 5:571–579.

4. Huber PJ. Robust Statistics. Wiley: New York, 1981.

(DOI: 10.1002/sim.1937)

AUTHORS’ REPLY

Dr Chen’s letter raises some interesting points. Whilst not disagreeing with these, we thinkthat it will be helpful to the reader if we clear up a potential cause of confusion. It shouldbe noted that the original paper by Koch et al. [1] considered two possible estimators for thevariance of the covariate. These are both described in detail in the appendix of that paperand are referred to on p. 1884 using the symbols V0 and Vs. Koch et al. describe the �rstas corresponding to the use of a randomization type argument and the second to a samplingargument. We made it quite clear in our paper [2] that we were only considering the second ofthese. For example, on p. 3584 we said: ‘However, this whole debate is in any case irrelevantto our analysis here, which shows that whatever set of covariates is eventually chosen, then,conditional on the set that is chosen, the sampling-based approach of Koch is anticonservativecompared to ANCOVA.’ (Italics added.) Hence, our argument involves use of Vs which, inthe case where there are equal numbers in the two treatment groups, may be calculated fromthe pooled within treatment groups sums of squares. On the other hand, the estimator that DrChen now examines is based on V0, which recovers a further degree of freedom by using thefact that in a randomized clinical trial there can only be random di�erence between groups forcovariates. This corresponds to the �rst of Koch et al.’s proposals. Whilst it is, indeed, inter-esting to consider the properties of an estimator based on this alternative variance estimate,we did not do so in our paper. Hence, Dr Chen’s note is not directly relevant to our paper.

EMMANUEL LESAFFREBiostatistical Centre

University of Leuven, BelgiumSTEPHEN SENN

Department of StatisticsUniversity of Glasgow, U.K.

REFERENCES

1. Koch GG, Tangen CM, Jung JW, Amara IA. Issues for covariance analysis of dichotomous and ordered categoricaldata from randomized clinical trials and non-parametric strategies for addressing them. Statistics in Medicine1998; 17(15–16):1863–1892.

2. Lesa�re E, Senn S. A note on non-parametric ANCOVA for covariate adjustment in randomized clinical trials.Statistics in Medicine 2003; 22(23):3583–3596.

(DOI: 10.1002/sim.1938)

Copyright ? 2005 John Wiley & Sons, Ltd. Statist. Med. 2005; 24:1449–1452