8/14/2019 Lock_JSM_2010.pptx

1/12

Rerandomization inRandomized Experiments

Kari Lock and Don Rubin

Harvard UniversityJSM 2010

8/14/2019 Lock_JSM_2010.pptx

2/12

The Gold Standard

Why are randomized experiments so good?

They yield unbiased estimates of the treatment effect

They eliminate (?) confounding factors

ON AVERAGE. For any particular experiment,

covariate imbalance is possible (and likely)

8/14/2019 Lock_JSM_2010.pptx

3/12

Rerandomization

Suppose you are doing a randomized experiment andhave covariate information available beforeconducting

the experiment

You randomize to treatment and control, but get a

bad randomization

Can you rerandomize?Yes, but you first need to specify a concrete

definition of bad

8/14/2019 Lock_JSM_2010.pptx

4/12

Randomize subjects totreated and control

Collect covariate dataSpecify a criteria determining whena randomization is unacceptable;

based on covariate balance

(Re)randomize subjectsto treated and control

Check covariate balance

1)

2)

Conduct experiment

unacceptable acceptable

Analyze results with aFisher randomization test

3)

4)

8/14/2019 Lock_JSM_2010.pptx

5/12

Unbiased

To maintain an unbiased estimate of the treatmenteffect, the decision to rerandomize or not must be

automatic and specified in advance

blind to which group is treated

Theorem: If the treated and control groups are the

same size, and if for every unacceptable randomization

the exact opposite randomization is also unacceptable,

then rerandomization yields an unbiased estimate of

the treatment effect.

8/14/2019 Lock_JSM_2010.pptx

6/12

Mahalanobis Distance

Define overall covariate distance by

M = Dr-1D

2Under adequate sample sizes and pure randomization: ~ kM

Dj : Standardized difference between treated andcontrol covariate means for covariatej

k = number of covariates

D= (D1, , Dk)

r= covariate correlation matrix = cov(D)

Choose aand rerandomize when M > a

8/14/2019 Lock_JSM_2010.pptx

7/12

Rerandomization Based on M

Since M follows a known distribution, easy tospecify the proportion of rejected randomizations

M is affinely invariant

Correlations between covariates are maintained

The variance reduction on each covariateis the

same (and known)

The variance reduction for any linear combinationof the covariates is known

8/14/2019 Lock_JSM_2010.pptx

8/12

Rerandomization

Theorem:

If nT= nCand rerandomizationoccurs when M> a, then

| ,

cov co| v

T C

T C T C a

E M a

M a v

X X 0

X X X X

and

1,2 2 2

, is the incomplete gamma function

,2 2

a

k a

vk ak

2va

| 0,

| 1 (1 ) var .r

T C

T C T C a

E M a

M

Y Y

Y Ya vY R Y

8/14/2019 Lock_JSM_2010.pptx

9/12

Differencein CovariateMeans

Difference in Outcome Means

8/14/2019 Lock_JSM_2010.pptx

10/12

Pure Randomization

Re-Randomization

Standardized Differences in Covariate Means

-4 -2 0 2 4

male

age

collgpaa

actcomp

preflit

likelit

likemath

numbmath 0.14

0.15

0.17

0.14

0.16

0.16

0.16

0.15

, ,

, ,

var( |

var(

)

)

j T j C

j T j C

X X aT

X X

(theoretical va= .16)

8/14/2019 Lock_JSM_2010.pptx

11/12

Pure Randomization

Re-Randomization

var( | .57var(

))

T C

T C

Y Y aY Y

T

(theory = .58)

-1.0 -0.5 0.0 0.5 1.0

Equivalent to

increasing the

sample size by a

factor of 1.7

Difference in Outcome Means Under Null

8/14/2019 Lock_JSM_2010.pptx

12/12

Conclusion

Rerandomization improves covariate balancebetween the treated and control means, and

increases precision in estimating the treatment effect

if the covariates are correlated with the response

Rerandomization gives the researcher more power

to detect a significant result, and more faith that an

observed effect is really due to the treatment

lock@stat.harvard.edu