This article was downloaded by: [Southern Methodist University]On: 18 November 2014, At: 22:12Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK

Journal of the American Statistical AssociationPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/uasa20

Space-Filling Fractional Factorial DesignsYong-Dao Zhou & Hongquan XuAccepted author version posted online: 06 Jan 2014.Published online: 02 Oct 2014.

To cite this article: Yong-Dao Zhou & Hongquan Xu (2014) Space-Filling Fractional Factorial Designs, Journal of the AmericanStatistical Association, 109:507, 1134-1144, DOI: 10.1080/01621459.2013.873367

To link to this article: http://dx.doi.org/10.1080/01621459.2013.873367

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information. Taylor and Francis shall not be liable forany losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use ofthe Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Supplementary materials for this article are available online. Please go to www.tandfonline.com/r/JASA

Space-Filling Fractional Factorial DesignsYong-Dao ZHOU and Hongquan XU

Fractional factorial designs are widely used in various scientific investigations and industrial applications. Level permutation of factorscould alter their geometrical structures and statistical properties. This article studies space-filling properties of fractional factorial designsunder two commonly used space-filling measures, discrepancy and maximin distance. When all possible level permutations are considered,the average discrepancy is expressed as a linear combination of generalized word length pattern for fractional factorial designs with anynumber of levels and any discrepancy defined by a reproducing kernel. Generalized minimum aberration designs are shown to have goodspace-filling properties on average in terms of both discrepancy and distance. Several novel relationships between distance distribution andgeneralized word length pattern are derived. It is also shown that level permutations can improve space-filling properties for many existingsaturated designs. A two-step construction procedure is proposed and three-, four-, and five-level space-filling fractional factorial designsare obtained. These new designs have better space-filling properties, such as larger distance and lower discrepancy, than existing ones, andare recommended for use in practice. Supplementary materials for this article are available online.

KEY WORDS: Discrepancy; Generalized minimum aberration; Maximin distance design; Orthogonal array; Space-filling design; Uniformdesign.

1. INTRODUCTION

Fractional factorial designs are widely used in various scien-tific investigations and industrial applications. They are oftenchosen by the minimum aberration (MA) criterion (Fries andHunter 1980) and its extension, generalized minimum aberra-tion (GMA) criterion (Tang and Deng 1999; Xu and Wu 2001).GMA designs have many desirable properties and are robustagainst model uncertainty (Tang 2001; Xu and Wu 2001; Cheng,Deng, and Tang 2002).

The current research is motivated by the need of good designsfor several ongoing projects on combinatory drug experiments.Combinatory drugs have been broadly applied to treat variousdiseases since they often have higher efficacy and lower toxicitycompared to individual drugs. Fractional factorial designs, en-abling researchers to build good statistical models with a smallnumber of runs, were recently reported to study the effects of sixantiviral drugs on Herpes simplex virus type 1 (Ding et al. 2013;Jaynes et al. 2013). It is challenging to quantify drug contribu-tions and drug interactions for multiple drugs due to the inherentcomplexity of underlying biological systems (Al-Shyoukh et al.2011). Various models have been used to analyze drug combi-nation experiments. Nonlinear models such as Hill models arepopular for one or two drugs (Straetemans et al. 2005; Chou2010) whereas polynomial models and neural networks are alsoused for multiple drugs (Al-Shyoukh et al. 2011). With the un-certainty of models, space-filling designs are ideal for multipledrug combination experiments and enable the researchers toexplore various models. Latin hypercube designs are popularspace-filling designs for computer experiments; however, theyare not suitable for such biological experiments where the num-

Yong-Dao Zhou is Associate Professor, College of Mathematics, SichuanUniversity, Chengdu, Sichuan 610064, China (E-mail: ydzhou@scu.edu.cn).Hongquan Xu is Professor and Graduate Vice Chair, Department of Statis-tics, University of California, Los Angeles, California 90095 (E-mail:hqxu@stat.ucla.edu). This research was done when the first author was vis-iting Department of Statistics at University of California, Los Angeles, withsupport from Sichuan University. This work was partially supported by NNSFof China with grant No. 11001186 for Zhou and by National Science Foun-dation Grant DMS-1106854 for Xu. The authors thank an associate editor andtwo referees for their valuable comments which led to an improvement of thepresentation.

ber of levels or dosages for each drug should be small in prac-tice. Uniform designs are another class of space-filling designsfor physical and computer experiments (Fang and Wang 1994;Fang, Li, and Sudjianto 2006a). They could be potentially usedfor these biological experiments; however, as we will show later,many existing uniform designs do not have good space-fillingproperties. Therefore, we propose new methods to constructspace-filling fractional factorial designs.

As a measure of uniformity, discrepancy plays a key rolein uniform design. Hickernell (1998) used the tool of repro-ducing kernel Hilbert spaces to define several discrepancies,such as the centered L2-discrepancy (CD) and the wrap-aroundL2-discrepancy (WD). Recently, Zhou, Fang, and Ning (2013)proposed a new discrepancy, mixture discrepancy, and arguedthat mixture discrepancy could be a better measure of unifor-mity than CD or WD. There is a close relationship betweenGMA and discrepancy for two or three-level designs. Fang andMukerjee (2000) derived an expression of CD for two-levelregular designs in terms of word length pattern; Ma and Fang(2001) extended their result to nonregular fractional designs andalso showed that WD can be expressed by the generalized wordlength pattern (GWP) for two- or three-level designs. Zhou,Ning, and Song (2008) showed that the Lee discrepancy fortwo or three-level designs can also be expressed by GWP. Fordesigns with more than two levels, level permutation of one ormore factors can alter their geometrical structures and statisticalproperties (Cheng and Ye 2004). Considering all possible levelpermutations, Tang, Xu, and Lin (2012) recently derived an ex-pression of average CD in terms of GWP for three-level designsand Tang and Xu (2013) extended their result to multilevel de-signs. Inspired by their work, in Section 3, we further studythe connection between aberration and uniformity under levelpermutations. We show that the average discrepancy is a linearcombination of GWP for fractional factorial designs with anynumber of levels and any discrepancy defined by a reproducingkernel. Therefore, GMA designs tend to have good uniformity

© 2014 American Statistical AssociationJournal of the American Statistical Association

September 2014, Vol. 109, No. 507, Theory and MethodsDOI: 10.1080/01621459.2013.873367

1134

Dow

nloa

ded

by [

Sout

hern

Met

hodi

st U

nive

rsity

] at

22:

12 1

8 N

ovem

ber

2014

Zhou and Xu: Space-Filling Fractional Factorial Designs 1135

under any aforementioned discrepancy. Our result unifies manyexisting results in the literature.

Maximin distance designs are another important class ofspace-filling designs (Johnson, Moore, and Ylvisaker 1990).The maximin distance criterion seeks to scatter design pointsover an experimental domain so that the separation distance(i.e., the minimal distance between pairs of points) is maxi-mized. The maximin distance criterion is conceptually simplerthan discrepancy, but harder to deal with theoretically. Johnson,Moore, and Ylvisaker (1990) showed that a maximin distancedesign is asymptotically D-optimal when the correlation func-tion decreases rapidly as the distance increases. John et al.(1995) considered the minimax distance designs in two-levelfactorial experiments. More discussion between fractional facto-rial designs and space-filling designs can be seen in Kerr (2001)and Ba and Joseph (2011). Unlike these authors, we study thespace-filling properties of fractional factorial designs under levelpermutations in Section 3. We obtain several novel relationshipsbetween distance and GWP. We show that GMA designs tendto have good space-filling properties under maximin distance.We further show that level permutations can increase the min-imal distance for many existing saturated designs. Based onthe theory, in Section 5, we propose a two-step procedure forconstructing space-filling fractional factorial designs. We ob-tain many three-, four-, and five-level designs, which often havelarger distance and lower discrepancy than existing designs.

The article is organized as follows. In Section 2, we establish akey lemma on level permutations for some general space-fillingmeasure. In Section 3, we study level permutations under anydiscrepancy defined by a reproducing kernel. In Section 4, weconsider level permutations under a distance measure. Section5 considers the construction of space-filling fractional factorialdesigns and Section 6 gives some conclusions. For clarity, werefer all proofs to the Appendix.

2. A LEMMA ON LEVEL PERMUTATIONS

For an N-point design D = {x1, . . . , xN } and a nonnegativefunction F (xi , xj ) ≥ 0, define

φ(D) = 1

N2

N∑i,j=1

F (xi , xj ). (1)

Call φ(D) as a space-filling measure of D with respect to F.Here we only require that the function F (xi , xj ) is nonnegativeto keep its generality. We can add conditions later to get specialspace-filling measures. Different choices of F (xi , xj ) naturallylead to different design criteria. If F (xi , xj ) is the absolute cor-relation function between points xi and xj , then φ(D) measuresthe average correlation among design points. If F (xi , xj ) is adecreasing function of distance between xi and xj , φ(D) is aspace-filling measure based on distance. As we will see in thefollowing sections, both maximin distance and discrepancy de-fined by a reproducing kernel can be obtained from the generalexpression (1).

Let Zs = {0, 1, . . . , s − 1}. A design with N runs, n factors,and s levels, denoted by (N, sn), is a set of N points over Zn

s oran N × n matrix over Zs , where each point or row representsa run, and each column represents a factor.

Let H = Zs × · · · × Zs and wt(u) be the number of nonzeroelements of the vector u ∈ H . For an (N, sn)-design D, con-sider an ANOVA model Y = X0α0 + X1α1 + · · · + Xnαn + ε,

where Y is the vector of N observations, ε is the ran-dom error, α0 is the intercept and X0 is an N × 1 vec-tor of 1’s, αj is the vector of all j-factor interactions andXj is the matrix of orthonormal contrast coefficients for αj .More precisely, Xj = [χu (x)]x∈D, wt(u)=j , where χu (x) =∏n

i=1 χui

(xi) for u = (u1, . . . , un) ∈ H and x = (x1, . . . , xn) ∈H , and {χ

ui(xi), ui ∈ Zs} are normalized orthogonal contrasts

for the ith factor such that∑s−1

xi=0 χui

(xi)2 = s. Define Aj (D) =N−2||XT

0 Xj ||2, j = 0, . . . , n, where ||X||2 = ∑i,j x2

ij for amatrix X = (xij ). It is obvious that A0(D) = 1. The vec-tor (A1(D), A2(D), . . . , An(D)) is called the generalized wordlength pattern (GWP) of design D.

The GMA criterion (Xu and Wu 2001) is to sequentiallyminimize A1(D), A2(D), . . . , An(D).

An orthogonal array of N runs, n columns, s levels andstrength t, denoted by OA(N, sn, t), is an N × n matrix in whichall st level combinations appear equally often in every N × t

submatrix. Xu and Wu (2001) showed that D is an OA(N, sn, t)if and only if A1(D) = · · · = At (D) = 0.

For multilevel designs, level permutations lead to differentgeometrical structures and statistical properties and one mayimprove design properties by permuting levels of some factors.For an (N, sn)-design D, when considering all s! possible levelpermutations for every factor, we obtain (s!)n combinatoriallyisomorphic designs. Denote the set of these designs as P(D).Because reordering the rows or columns does not change thegeometrical structure and statistical properties of a design, thereis no need to consider row or column permutations. All designsin P(D) share the same GWP, but may have different φ(D). Wecan compute φ(D) for each design, as well as the average value,denoted by φ̄(D), of all designs in P(D). More precisely, define

φ̄(D) = 1

(s!)n∑

D′∈P(D)

φ(D′). (2)

The following result shows that the average value φ̄(D) in (2)can be expressed as a linear combination of GWP for a wideclass of space-filling measures.

Lemma 1. Suppose F (xi , xj ) = ∏nk=1 f (xik, xjk) and f (·, ·)

satisfies

f (x, y) ≥ 0 and f (x, x) + f (y, y) > f (x, y) + f (y, x)

for any x �= y, x, y ∈ [0, 1]. (3)

For an (N, sn)-design D,

φ̄(D) =(

c1(c2 + s − 1)

s2(s − 1)

)n n∑i=0

(c2 − 1

c2 + s − 1

)i

Ai(D), (4)

where c1 = ∑s−1k=0

∑l �=k f (k, l) and c2 = (s − 1)

∑s−1k=0

f (k, k)/c1.

The conditions of F (·, ·) in Lemma 1 are nonrestrictive andsatisfied by many commonly used discrepancies and other mea-sures. The requirement (3) makes c2 > 1 so that the coefficientof Ai(D) in (4) decreases geometrically as i increases. As a

Dow

nloa

ded

by [

Sout

hern

Met

hodi

st U

nive

rsity

] at

22:

12 1

8 N

ovem

ber

2014

1136 Journal of the American Statistical Association, September 2014

result, when all level permutations are considered, φ̄(D) tendsto agree with the GMA criterion.

3. DISCREPANCY AND LEVEL PERMUTATIONS

Tang, Xu, and Lin (2012) and Tang and Xu (2013) showedthat the average CD value is a linear function of the GWP, whenall level permutations are considered. Here we generalize theirresults for any discrepancy defined by a reproducing kernel.

Let X be an experimental domain. A reproducing ker-nel K(x, y) defined on X 2 = X × X satisfies two prop-erties: (i) K(x, y) = K( y, x) for all x, y ∈ X and (ii)∑N

i,j=1 ciK(xi , xj )cj ≥ 0 for all xi , xj ∈ X and ci, cj ∈ R. Foran N-point design D = {x1, . . . , xN } over X , the L2-type dis-crepancy for a given kernel K(x, y) is defined as (see Hickernell1998)

Disc(D,K) =∫X 2

K(x, y)dFu(x)dFu( y)

− 2

N

N∑i=1

∫XK(xi , y)dFu( y)

+ 1

N2

N∑i,j=1

K(xi , xj ), (5)

where Fu(·) is the uniform distribution in the experimental do-main X . Different kernel functions K(·, ·) induce different dis-crepancies. Note that we can express the discrepancy in (5) as aspace-filling measure (1) with

F (xi , xj ) = K(xi , xj ) − K1(xi) − K1(xj ) + K2, (6)

where K1(x) = ∫X K(x, y)dFu( y) and K2 = ∫

X 2 K(x, y)dFu(x)dFu( y) is a constant. In other words, any discrepancy de-fined by the reproducing kernel method is a special space-fillingmeasure.

Commonly used reproducing kernels for discrepancies in theliterature are defined on X = [0, 1]n and have a multiplicativeform

K(x, y) =n∏

k=1

f (xk, yk), (7)

where f (x, y) is defined on [0, 1]2. Then the correspondingdiscrepancy in (5) can be expressed by

Disc(D,K) = K2 − 2

N

N∑i=1

n∏k=1

f1(xik)

+ 1

N2

N∑i,j=1

n∏k=1

f (xik, xjk), (8)

where f1(x) = ∫ 10 f (x, y)dy. The various kernel functions are

as follows:

(i) for CD, f (x, y) = 1 + (|x − 0.5| + |y − 0.5| − |x −y|)/2;

(ii) for WD, f (x, y) = 1.5 − |x − y| + |x − y|2;(iii) for mixture discrepancy, f (x, y) = 15/8 − |x −

0.5|/4 − |y − 0.5|/4 − 3|x − y|/4 + |x − y|2/2;(iv) for Lee discrepancy, f (x, y) = 1 − min{|x − y|, 1 −

|x − y|}.

Specially, for an (N, sn)-design D = (xik)N×n, its (squared)CD and (squared) WD are defined as follows:

CD(D) =(

13

12

)n

− 2

N

N∑i=1

n∏k=1

(1 + 1

2

∣∣∣∣uik − 1

2

∣∣∣∣− 1

2

∣∣∣∣uik − 1

2

∣∣∣∣2)

+ 1

N2

N∑i=1

N∑j=1

n∏k=1

(1 + 1

2

∣∣∣∣ uik

− 1

2

∣∣∣∣+ 1

2

∣∣∣∣ujk − 1

2

∣∣∣∣− 1

2

∣∣uik − ujk

∣∣) ,

WD(D) = −(

4

3

)n

+ 1

N2

N∑i=1

N∑j=1

n∏k=1

×[

3

2− |uik − ujk| + |uik − ujk|2

], (9)

where uik = (xik + 0.5)/s. Note that 0 ≤ xik ≤ s − 1 and 0 <

uik < 1.Now we consider level permutations of any given fractional

factorial design D and calculate the average discrepancy of allpermuted designs, denoted by Disc(D,K). For any row xi of D,when one considers all level permutations, each n−tuple in Zn

s

occurs ((s − 1)!)n times. Then,

∑D′∈P(D)

N∑i=1

n∏k=1

f1(xik) =N∑

i=1

∑D′∈P(D)

n∏k=1

f1(xik)

= N

((s − 1)!

s−1∑k=0

f1(k)

)n

, (10)

which is a constant. From Lemmas 1, (8), and (10), we have thefollowing result.

Theorem 1. Suppose that K(x, y) = ∏nk=1 f (xk, yk) and

f (·, ·) satisfies (3). For an (N, sn)-design D, when all levelpermutations of D are considered,

Disc(D,K) = K0 +(

c1(c2 + s − 1)

s2(s − 1)

)n

×n∑

i=0

(c2 − 1

c2 + s − 1

)i

Ai(D), (11)

where K0 = K2 − 2(∑s−1

k=0 f1((k + 0.5)/s)/s)n is a constant,K2 and f1(·) are, respectively, defined in (6) and (8),c1 = ∑s−1

k=0

∑l �=k f ((k + 0.5)/s, (l + 0.5)/s), and c2 = (s −

1)∑s−1

k=0 f ((k + 0.5)/s, (k + 0.5)/s)/c1.

From Theorem 1, for any discrepancy defined by a re-producing kernel satisfying (3), the average discrepancy isa linear combination of GWP under all level permutations,and the commonly used discrepancies such as WD andCD satisfy the condition (3). For example, from the ker-nel of WD, we have K2 = (4/3)n,

∑s−1k=0 f1((k + 0.5)/s)/s =

4/3, c1 = ∑s−1k=0

∑l �=k(1.5 − |k − l|/s + |k − l|2/s2) = (s −

1)(8s − 1)/6, and c2 = 9s/(8s − 1) > 1. Then for the averageWD value, we have

WD(D) = −(

4

3

)n

+(

8s2 + 1

6s2

)n n∑i=0

(s + 1

8s2 + 1

)i

Ai(D).

(12)

Dow

nloa

ded

by [

Sout

hern

Met

hodi

st U

nive

rsity

] at

22:

12 1

8 N

ovem

ber

2014

Zhou and Xu: Space-Filling Fractional Factorial Designs 1137

Especially, when s = 2, all level permuted designs have thesame WD value since WD is invariant under coordinate rotation;therefore, Equation (12) shows the exact relationship betweenGWP and WD for two-level designs. When s = 3, Tang, Xu,and Lin (2012) showed that we need only to consider linearlevel permutations when computing φ̄(D), then based on theexpression of WD in (9), any linear level permutation does notchange the WD value, which means that Equation (12) alsoshows the exact relationship between GWP and WD for three-level designs. In other words, Equation (12) includes the resultof WD for two and three-level designs in Ma and Fang (2001).Similarly, applying Theorem 1 to CD, we obtain the relationshipbetween average CD and GWP, which was reported by Fangand Mukerjee (2000), Ma and Fang (2001), Tang, Xu, and Lin(2012), and Tang and Xu (2013) for two-, three-, and multileveldesigns, respectively. Moreover, Theorem 1 includes the resultson Lee discrepancy for two- and three-level designs in Zhou,Ning, and Song (2008) and the result on mixture discrepancy fortwo-level designs in Zhou, Fang, and Ning (2013). In summary,Theorem 1 gives a unified result for any type of discrepancydefined by a reproducing kernel.

The relationship between average discrepancy and GWP inTheorem 1 is useful in the construction of low discrepancydesigns. Here is an example.

Example 1. Consider the four-level nonregular de-sign OA(32, 49, 2) from Sloane’s website, http://neilsloane.com/oadir/. It can be verified that its first n (n ≤ 9) columnsform a GMA design. For n = 6 − 9 , from (12), the average WDvalues of these GMA designs are 0.276046, 0.437511, 0.681677,and 1.054606, respectively. These average discrepancy valuesare lower than the discrepancy values of the correspondingdesigns on the UD website, http://www.uic.edu.hk/isci/, whichare 0.276559, 0.443409, 0.698060, and 1.084390, respectively.Clearly, we can generate designs with discrepancy values lowerthan the averages by permuting these GMA designs.

4. MAXIMIN DISTANCE DESIGNS AND LEVELPERMUTATIONS

For any two rows x = (x1, . . . , xn) and y = (y1, . . . , yn) ofan (N, sn)-design D, define dp(x, y) = ∑n

i=1 |xi − yi |p, p ≥ 1,as the Lp-distance of rows x and y. Our definition here makesthe Lp-distance an integer for integer p and is different from theconventional one. More importantly, under our definition, theLp-distance is additive, that is, dp(x, y) = ∑n

i=1 dp(xi, yi).The Hamming distance between x and y, denoted by dH (x, y),is the number of positions where they differ, that is, the numberof i’s such that xi �= yi . The Hamming distance is also addi-tive. Obviously, dp(x, y) ≥ dH (x, y) for p ≥ 1. Define the Lp-distance of D to be dp(D) = min{dp(x, y) : x, y ∈ D, x �= y},that is, dp(D) is the smallest distance among all design points inthe sense of Lp−norm. Similarly, dH (D) denotes the minimumHamming distance among all design points.

4.1 Distance Distribution

The maximin distance criterion, proposed by Johnson, Moore,and Ylvisaker (1990), chooses designs that maximize the min-

imum distance among design points and further minimize thenumber of pairs that have minimum distance.

Suppose d(x, y) is a distance measure which takes integervalues 0, 1, . . . , m for points x and y in Zn

s . Let Bj (D) =N−1�{(x, y) : x, y ∈ D and d(x, y) = j} for j = 0, 1, . . . , m,where �{·} denotes the cardinality of a set. The vector

(B0(D), B1(D), . . . , Bm(D)) (13)

is called the distance distribution of D. When the design hasno repeat points, B0(D) = 1 and the minimum distance isthe smallest positive integer j such that Bj (D) > 0. Mim-icking the GMA criterion, Morris and Mitchell (1995) ex-tended the maximin distance criterion to sequentially minimizeB0(D), B1(D), . . . , Bm(D). For clarity, the distance distribu-tion under the Hamming distance or Lp-distance is referredto Hamming distance distribution or Lp-distance distribution,respectively.

Under the Hamming distance, the maximum possible valuem = n. It is well known that the Hamming distance distributionhas a close relationship with GWP. Xu and Wu (2001) showedthat GWP is equivalent to the MacWilliams transform of thedistance distribution, that is,

Aj (D) = N−1n∑

i=0

Pj (i; n, s)Bi(D), j = 0, 1, . . . , n, (14)

where Pj (x; n, s) = ∑j

i=0(−1)i(s − 1)j−i( x

i)( n−x

j−i) are the

Krawtchouk polynomials. In other words, GWP is determinedby the Hamming distance distribution. Moreover, by the orthog-onality of the Krawtchouk polynomials,

Bj (D) = Ns−n

n∑i=0

Pj (i; n, s)Ai(D), j = 0, 1, . . . , n. (15)

Under the Lp-distance, the maximum possible value m =n(s − 1)p. Usually, p = 1 or p = 2 is chosen for the maximincriterion. Especially, when p = 1, dp(x, y) is the rectangulardistance between x and y. For clarity, we focus on the L1-distance in this article.

In this section we consider F (xi , xj ) in (1) as a decreasingfunction of distance d(xi , xj ). Specially, consider

F (xi , xj ) = ρd(x i ,xj ), ρ ∈ (0, 1). (16)

Then φ(D) = N−2∑Ni,j=1 F (xi , xj ) = N−1 ∑m

k=0 ρkBk(D).When ρ → 0, minimizing φ(D) is equivalent to sequentiallyminimize B0(D), B1(D), . . . , Bm(D) in (13). Therefore, thecriterion of minimizing φ(D) with F (xi , xj ) defined in (16)can be viewed as an extension of the maximin criterion.

When d(xi , xj ) is an additive distance measure, F (xi , xj ) =ρd(x i ,xj ) = ∏n

k=1 ρd(xik ,xjk ) satisfies the condition in Lemma 1with c1 = ∑s−1

k=0

∑l �=k ρd(k,l) and c2 = (s − 1)s/c1. Note that

c2 > 1 for any 0 < ρ < 1. Then from Lemma 1, we directlyhave the following result regarding additive distance measures.

Theorem 2. Suppose that d(xi , xj ) is an additive distancemeasure and F (xi , xj ) = ρd(x i ,xj ), where 0 < ρ < 1. Foran (N, sn)-design D, when all level permutations of D are

Dow

nloa

ded

by [

Sout

hern

Met

hodi

st U

nive

rsity

] at

22:

12 1

8 N

ovem

ber

2014

1138 Journal of the American Statistical Association, September 2014

considered,

φ̄(D) =(

c2 + s − 1

c2s

)n n∑i=0

(c2 − 1

c2 + s − 1

)i

Ai(D), (17)

where c2 = (s − 1)s/c1 and c1 = ∑s−1k=0

∑l �=k ρd(k,l).

Under the rectangular distance, d(k, l) = |k − l|, c1 =∑s−1k=0

∑l �=k ρd(k,l) = 2

∑s−1k=1(s − k)ρk , then we have

φ̄(D) = 1

s2n

n∑i=0

(s + 2

s−1∑k=1

(s − k)ρk

)n−i

×(

1

s − 1

(s2 − s − 2

s−1∑k=1

(s − k)ρk

))i

Ai(D).

(18)

When s = 3, the above expression simplifies to φ̄(D) =9−n

∑ni=0(3 − ρ2 − 2ρ)i(3 + 2ρ2 + 4ρ)n−iAi(D).

On the other hand, we know φ(D) = N−1 ∑mk=0 Bk(D)ρk .

When considering all level permutations of D, we have

φ̄(D) = 1

N

m∑k=0

B̄k(D)ρk, (19)

where B̄k(D) is the average value of Bk(D). Moreover, thecoefficients of ρk in (18) and (19) must be equal to each other,so we obtain an interesting relationship between B̄k(D) andGWP. For illustration, when s = 3, we have the following resultfor the rectangular distance.

Theorem 3. For an (N, 3n)-design D and rectangular dis-tance,

B̄k(D) = 2kN

3n+k

n∑i=0

⎛⎝ n∑

j=0

(3

4

)j (k − j

j

)Pk−j (i; n, 3)

⎞⎠Ai(D),

k = 0, 1, . . . , 2n,

where Pj (x; n, s) are the Krawtchouk polynomials.

Theorem 3 can be proved via some tedious calculations andthe proof is given in the supplementary materials of this arti-cle. Theorem 3 shows that B̄k(D) of a three-level design is alinear combination of GWP under the L1−distance. We can ob-tain similar results for higher-level designs and other distancemeasures in the same manner.

Example 2. Consider the MA 310−7 design D given byXu (2005). It is straightforward to verify that the L1-distance distribution is B0(D) = 1, B1(D) = · · · = B5(D) =0, B6(D) = 18/27, etc. The GWP is (A1, . . . , A10) =(0, 0, 42, 144, 270, 480, 630, 378, 206, 36). Using Theorem 3,we can calculate the average L1-distance distribution for all levelpermutations from the GWP as B̄0(D) = 1, B̄1(D) = · · · =B̄5(D) = 0, B̄6(D) = 128/243, etc. Since B̄6(D) < B6(D), weconclude that there exists some level permutations that will im-prove the space-filling of the design. Indeed, we can find levelpermutations to increase the minimum distance from 6 to 7.

Furthermore, differentiating (18) and (19) r times with respectto ρ and then letting ρ = 1, we obtain another relationshipbetween B̄k(D) and Ai(D).

Theorem 4. For an (N, sn)-design D and rectangular distance,

1

N

m∑k=1

kB̄k(D) = n(s2 − 1)

3s− s + 1

3sA1(D), (20)

1

N

m∑k=1

k2B̄k(D) = C2,0 + C2,1A1(D) + 2

(s + 1

3s

)2

A2(D),

(21)

where m = (s − 1)n, C2,0 = n(s2 − 1)(s2 + 2ns2 − 2n +2)/(18s2), C2,1 = −(s + 1)(3s2 + 4(n − 1)(s2 − 1))/(18s2).Moreover, for r ≥ 3,

1

N

m∑k=1

kr B̄k(D) =r∑

j=0

Cr,jAj (D), (22)

where Cr,j are constants and Cr,r = (−1)r r!(s + 1)r/(3s)r .

Note that N−1∑mk=1 kB̄k(D) and N−1 ∑m

k=1 k2B̄k(D) mea-sure the average and variation of distance among design pointswhen all level permutations are considered, respectively. Sincethe GMA criterion sequentially minimizes A1(D), A2(D), . . .,from Theorem 4, we have the following result.

Corollary 1. When all level permutations are considered,GMA designs maximize the average distance and minimizethe variation of the distance among design points under therectangular distance.

In other words, GMA designs tend to have good space-fillingproperties under L1-distance.

Example 3. Consider L1-distance and all possible 9-column designs from OA(36, 313, 2) on Sloane’s website,http://neilsloane.com/oadir/. Among the total

(139

) = 715 de-signs, there are 16 GMA designs and 699 non-GMA designs;some of them may be combinatorially isomorphic. Without levelpermutations, all 16 GMA designs have maximin distance 5while 207 non-GMA designs have maximin distance 5 and 492non-GMA designs have maximin distance 4. We conduct allpossible level permutations for each design and record theirbest and worse maximin distance. All 16 GMA designs havebest maximin distance 6 and worst maximin distance 5. Amongthe 207 non-GMA designs with maximin distance 5, 204 de-signs have both best and worst maximin distance 5 while other3 designs have best maximin distance 5 and worst maximin dis-tance 4. The remaining 492 non-GMA designs have both bestand worst maximin distance 4. We should mention that the num-bers of pairs with maximin distance are different under differentlevel permutations even though the maximin distance is thesame.

4.2 Saturated Designs

An (N, sn)-design is saturated if N − 1 = n(s − 1). Saturateddesigns are important in practice because they can accommodatethe maximum number of factors.

Mukerjee and Wu (1995) showed that the Hamming distancebetween any pair of rows of a saturated OA(N, sn, 2) is equal

Dow

nloa

ded

by [

Sout

hern

Met

hodi

st U

nive

rsity

] at

22:

12 1

8 N

ovem

ber

2014

Zhou and Xu: Space-Filling Fractional Factorial Designs 1139

to N/s. Then the L1-distance of any saturated design should benot less than N/s. For the average L1-distance distribution of asaturated design, we have the following result.

Theorem 5.

(i) For a saturated OA(N, sn, 2) design D and rectangulardistance,

B̄0(D) = 1, B̄k(D) = 0 (k = 1, . . . , N/s − 1),

B̄N/s(D) = (N − 1)

(2

s

)N/s

,

B̄N/s+1(D) = N (N − 1)(s − 2)2N/s

(s − 1)sN/s+1.

(ii) For a saturated OA(N, sn, 2) design D with s = 3 andrectangular distance,

B̄k(D) = (N − 1)22N/3−k

3N/3

(N/3

k − N/3

),

for k = N/3, . . . , 2n.

We do not list B̄k(D) in Theorem 5(i) when k > N/s + 1because it has a complicated form. When the L1−distance of asaturated OA(N, sn, 2) is equal to N/s, there exists at least onepair of rows separated by that distance, that is, BN/s(D) ≥ 2/N .From Theorem 5, when B̄N/s(D) < 2/N , there must exist somelevel permutations which increase the L1-distance. Furthermore,when the value B̄N/s(D) + B̄N/s+1(D) in Theorem 5 is alsosmaller than 2/N , then there also exist some level permutationssuch that the L1-distance increases by at least two. Generalizingthis, we have the following result.

Proposition 1. For an (N, sn)-design D with B0(D) = 1,when

∑kj=1 B̄j (D) < 2/N , there exists a level permuted design

with L1-distance ≥ k + 1.

Combining Theorem 5 and Proposition 1, we can show thatwhen the run size is large enough, there exist some level permu-tations that increase the L1-distance for many existing saturateddesigns.

Example 4. For any saturated OA(81, 340, 2), its Hammingdistance is dH (D) = 27. It is straightforward to compute thatB̄27(D) + B̄28(D) ≈ 0.0014 + 0.0190 = 0.0204 < 2/81. Fol-lowing Proposition 1, there exists a permuted design with rect-angular design at least 29. As another example, for any satu-rated OA(243, 3121, 2), its Hamming distance is dH (D) = 81.It is straightforward to compute that

∑91j=1 B̄j (D) ≈ 0.0082 <

2/81. Following Proposition 1, there exists a permuted designwith rectangular design at least 92.

An important class of saturated designs is regular designsconstructed by the Rao-Hamming method (Hedayat, Sloane,and Stufken 1999). The standard construction method worksas follows. Let x1, . . . , xn−k be n − k indeterminate variablestaking values from GF(s), the finite field of s elements, where sis a prime or prime power. A saturated regular sn−k design hasn = (sn−k − 1)/(s − 1) columns and its columns can be labeledas c1x1 + c2x2 + · · · + cn−kxn−k , where ci ∈ GF(s), at least oneof the ci �= 0, and the first nonzero ci = 1. For example, the

columns of a saturated regular design with s2 runs are

x1, x2, x1 + x2, x1 + 2x2, . . . , x1 + (s − 1)x2. (23)

When x1, . . . , xn−k are evaluated over GF(s), we obtain a stan-dard sn−k design which is saturated. Nonsaturated regular de-signs are often constructed as a subset of the saturated design.Notice that by the conventional choice of the ci , the saturateddesign has a row whose elements are 0 and 1 only. The sat-urated design also has a row of zeros. As a result, the L1-distance of the saturated design is equal to its Hamming dis-tance, N/s = sn−k−1. From Theorem 5 and Proposition 1, when(N − 1)(2/s)N/s < 2/N , we can always find some level permu-tations to increase the L1−distance of the saturated regular sn−k

design. We summarize the results as a corollary.

Corollary 2.

(i) A saturated regular sn−k design constructed under thestandard Rao-Hamming method has L1-distance N/s =sn−k−1.

(ii) When (N − 1)(2/s)N/s < 2/N , there exist level permu-tations to increase the L1-distance of the design in (i).

The condition in Corollary 2(ii) is easy to be fulfilled whenN is large; see Example 4 for instance.

5. CONSTRUCTION OF SPACE-FILLINGFRACTIONAL FACTORIAL DESIGNS

We have shown that GMA designs tend to have good space-filling properties and level permutations can improve space-filling properties of many standard designs. Based on these the-oretical results, we propose the following two-step procedurefor constructing space-filling fractional factorial designs:

1. Use the GMA criterion to select the best design from someexisting orthogonal arrays.

2. For the resulting GMA design in Step 1, conduct levelpermutations to improve its space-filling property.

As Example 3 shows, the first step is effective in screening outpoor designs. The construction of GMA designs is a challengingproblem itself so we recommend to search the best design fromsome existing orthogonal arrays. In some cases, it is possible toshow that the resulting designs indeed have GMA among all pos-sible designs; see Xu (2003) and Butler (2005). Although levelpermutations alter geometrical structures, all designs evaluatedin Step 2 are combinatorially isomorphic to and have the sameGWP as the GMA design from Step 1. This restriction helpsspeed up the search in Step 2 tremendously, but sometimes mayprevent our algorithm finding the best design.

In Step 2, we can use either discrepancy or maximin distancecriterion to select the best level permutation. For any (N, sn)-design, there are total (s!)n level permutations. Considering thesymmetry of geometrical structure, we only need to evaluate(s!/2)n permuted designs. For a regular 3n−k design it is suf-ficient to consider permuting the k dependent columns, whichleads to 3k permuted designs (Tang, Xu, and Lin 2012). Forsmall designs, one can conduct all level permutations and findthe best design under either discrepancy or maximin distancecriterion. For larger designs, it may be infeasible to perform allpossible level permutations. In such circumstances, heuristic or

Dow

nloa

ded

by [

Sout

hern

Met

hodi

st U

nive

rsity

] at

22:

12 1

8 N

ovem

ber

2014

1140 Journal of the American Statistical Association, September 2014

Algorithm 1 Pseudocode for prototype local search heuristic.Initialize τ (number of iterations)Initialize δ (impairment threshold) and u0 (acceptance probability)Input starting design Dc and let Dmin := Dcfor i = 1 to τ do

Generate Dnew ∈ N (Dc) (neighbor to current solution)Compute ∇ = ϕ(Dnew) − ϕ(Dc) and generate a random number u fromuniform [0, 1]if (∇ < 0) or (∇ < δ and u < u0) then let Dc := Dnewif ϕ(Dc) < ϕ(Dmin) then let Dmin := Dc

end for

stochastic search algorithms can be used to find optimal or nearoptimal level permutations. Tang and Xu (2013) recently pro-posed a local search heuristic algorithm for searching best levelpermutations in the construction of uniform designs. We ini-tially conduct all level permutations whenever possible, whichmay take up to hours or days on a laptop in some cases. Laterwe adopt the Tang and Xu (2013) algorithm by replacing theCD criterion with the maximin distance criterion.

The pseudocode for the local search heuristic algorithm isshown in Algorithm 1. The starting design is the GMA designfrom Step 1. As in Tang and Xu (2013), we define the neighborN (Dc) by exchanging all elements of two distinct levels within asingle column of the current design Dc so that the combinatorialstructure of the design is unchanged. For the criterion ϕ(D),Tang and Xu (2013) used the CD value of design D, while we useϕ(D) = −dp(D) + f (D), where dp(D) is the Lp-distance ofdesign D, f (D) = pair/(N(N − 1)/2), and pair is the number ofpairs with distance dp(D). We choose the impairment thresholdδ around 0.1 and the acceptance probability u0 between 0.1 and0.3. The modified Tang and Xu (2013) algorithm is efficient infinding the best level permutation within a few seconds, with upto τ = 10,000 iterations, in many cases.

For illustration, we select four orthogonal arrays, OA(16, 45),OA(18, 37), OA(32, 49), OA(36, 313), from Sloane’s website;they are “oa.16.5.4.2.txt,” “oa.18.7.3.2.txt,” “oa.32.9.4.2.a.txt,”and “oa.36.13.3.2.txt,” respectively. We further constructOA(25, 56) via the standard Rao-Hamming method as in (23).For the 16-, 18-, 25-, and 32-run designs, the first n columnsform the GMA design. We also consider regular 27-run designsand choose MA designs according to Xu (2005). Table 1 showsthe GMA designs for 18, 27, and 36 runs.

We use the maximin L1-distance as the space-filling measurein Step 2. If we use CD as the space-filling measure, the resultsare similar and therefore omitted. We conduct all level permu-tations except the cases of 32-run design with nine factors and25-run designs with five and six factors, which are found usingthe modified Tang and Xu (2013) algorithm. For 18-, 27-, and36-run designs, we give the level permutation vectors in Table 1since only linear permutations are necessary for three-level de-signs. For 16-, 25-, and 32-run designs, we explicitly give thebest designs in the supplementary materials of this article.

Table 2 compares the maximin distance and CD values forstandard GMA designs from Step 1 and permuted GMA designsfrom Step 2. The permuted GMA designs often have larger L1-distance or smaller number of pairs separated by the minimumdistance. In more than half of the cases, the permuted GMAdesigns also have smaller CD values than the standard designs,

Table 1. GMA designs and level permutations for 18, 27, and 36 runs

N s n Columns Level permutation

18 3 3 1 ∼ 3 0 0 118 3 4 1 ∼ 4 0 0 0 218 3 5 1 ∼ 5 0 0 0 2 218 3 6 1 ∼ 6 0 0 0 2 2 218 3 7 1 ∼ 7 0 0 1 0 1 1 127 3 4 1 2 5 8 0 0 0 027 3 5 1 2 5 8 4 0 0 0 2 127 3 6 1 2 5 8 4 12 0 0 0 2 1 127 3 7 1 2 5 8 4 12 6 0 0 0 2 1 1 127 3 8 1 2 5 8 4 12 6 11 0 0 0 2 1 1 2 027 3 9 1 2 5 8 4 12 6 11 13 0 0 0 2 1 1 2 0 027 3 10 1 2 5 8 4 12 6 11 13 3 0 0 0 2 1 1 2 0 0 227 3 11 1 2 5 8 4 12 6 11 13 3 7 0 0 0 2 1 1 1 1 1 2 027 3 12 1 2 5 8 4 12 6 11 13 3 7 10 0 0 0 0 1 2 0 2 1 1 0 027 3 13 1 2 5 8 4 12 6 11 13 3 7 10 9 0 0 0 2 2 0 0 1 2 1 2 1 136 3 4 1 ∼ 3, 6 2 1 0 236 3 5 1 ∼ 3, 9, 10 2 2 0 2 136 3 6 1 ∼ 3, 9 ∼ 11 0 1 0 2 0 036 3 7 1 ∼ 7 1 0 2 0 1 2 036 3 8 1 ∼ 3, 6 ∼ 9, 12 2 2 2 1 0 0 0 036 3 9 1 ∼ 8, 10 2 2 0 1 0 0 2 1 236 3 10 1 ∼ 10 2 2 1 0 0 1 2 1 1 136 3 11 1 ∼ 11 2 1 2 0 1 2 1 0 2 0 136 3 12 1 ∼ 12 2 0 2 2 1 2 0 0 1 1 1 236 3 13 1 ∼ 13 0 2 1 0 0 0 2 0 2 2 2 1 2

NOTE: The 18- and 36-run designs are from Sloane’s website, http://neilsloane.com/oadir/;the 27-run designs are from Xu (2005). The level permutation {c1, . . . , cn} can be used togenerate a new design by changing each column of the original design (xij ) to (xij + cj )(mod 3) for j = 1, . . . , n and i = 1, . . . , N.

although the uniformity is not used in the selection of levelpermutations.

We further compare permuted GMA designs with existinguniform designs. The existing uniform designs on the UD web-site were constructed via local search heuristic threshold accept-ing algorithms and the search was restricted within the class ofU-type designs; see Winker and Fang (1997), Fang, Lu, andWinker (2003) and Fang et al. (2006b). For an (N, sn)-design,there are total (N !/((N/s)!)s)n U-type designs since each levelappears N/s times for any factor. Unfortunately, none of exist-ing one-step optimization algorithms were effective in search-ing over this huge space for even moderate N. In contrast, oursecond step only searches maximum (s!)n designs that are com-binatorially isomorphic to the initial GMA design; therefore,the two-step procedure has much smaller candidate space and isfaster than a one-step procedure. The two-step procedure is effi-cient in finding good space-filling or uniform designs, especiallyfor large N.

Table 2 shows that majority of permuted GMA designs havebetter space-filling properties in terms of both distance and uni-formity than the existing uniform designs. The L1-distances ofall permuted GMA designs are larger than or equal to that of ex-isting uniform designs. Note that for (N, n) = (18, 3), (36, 4), or(36, 5), the existing uniform design has a pair of repeated pointsand so its L1-distance is 0. Moreover, permuted GMA designsalso have smaller CD values than existing uniform designs inmore than half of the cases. When the number of factors or runs

Dow

nloa

ded

by [

Sout

hern

Met

hodi

st U

nive

rsity

] at

22:

12 1

8 N

ovem

ber

2014

Zhou and Xu: Space-Filling Fractional Factorial Designs 1141

Table 2. Comparison of designs under maximin distance and CD

Standard GMA designs Permuted GMA designs Existing uniform designs

N s n d(pair) CD d(pair) CD d(pair) CD A2

16 4 3 2 (12) 0.0189 2 (12) 0.0189 2 (14) 0.0189 016 4 4 4 (57) 0.0285 4 (56) 0.02843 3 (8) 0.02844 016 4 5 4 (9) 0.0428 4 (4) 0.0421 4 (5) 0.0417 018 3 3 1 (18) 0.03254 1 (18) 0.03250 0 (1) 0.03250 018 3 4 2 (24) 0.04745 2 (21) 0.04739 2 (36) 0.04739 018 3 5 3 (30) 0.06534 3 (24) 0.06528 2 (4) 0.06528 018 3 6 4 (45) 0.0870 4 (18) 0.0871 3 (2) 0.0869 018 3 7 4 (9) 0.1157 4 (2) 0.1151 4 (5) 0.1136 025 5 3 2 (16) 0.0119 2 (16) 0.0119 2 (18) 0.0118 025 5 4 3 (12) 0.0181 4 (47) 0.017691 3 (6) 0.017692 025 5 5 4 (8) 0.0262 5 (4) 0.02519 4 (2) 0.02521 025 5 6 5 (4) 0.0368 6 (5) 0.03544 5 (1) 0.03541 027 3 4 2 (72) 0.0465 2 (72) 0.0465 2 (72) 0.0465 027 3 5 2 (12) 0.0639 2 (12) 0.0637 1 (1) 0.0635 027 3 6 3 (18) 0.0839 3 (12) 0.0835 3 (12) 0.0835 027 3 7 4 (27) 0.1089 4 (23) 0.1083 2 (1) 0.1087 0.1027 3 8 5 (32) 0.1382 5 (24) 0.1373 4 (1) 0.1387 0.3527 3 9 6 (35) 0.1736 6 (18) 0.1718 5 (5) 0.1753 0.6927 3 10 6 (9) 0.2202 7 (48) 0.2176 6 (3) 0.2191 1.3627 3 11 7 (14) 0.2761 8 (56) 0.2715 7 (6) 0.2724 227 3 12 8 (20) 0.3431 8 (4) 0.3354 8 (9) 0.3364 2.3227 3 13 9 (29) 0.4256 9 (3) 0.4145 8 (1) 0.4148 3.5332 4 3 1 (24) 0.0187 2 (156) 0.0187 1 (12) 0.0187 032 4 4 2 (40) 0.0276 2 (8) 0.0276 2 (10) 0.0274 032 4 5 3 (12) 0.0384 4 (106) 0.0383 3 (9) 0.0384 0.2532 4 6 4 (4) 0.0516 5 (58) 0.0520 4 (8) 0.0521 0.1932 4 7 6 (28) 0.0681 6 (28) 0.0680 4 (1) 0.0702 1.0932 4 8 8 (129) 0.0888 8 (128) 0.0885 6 (6) 0.0925 2.3832 4 9 8 (33) 0.1213 8 (6) 0.1200 8 (9) 0.1214 3.9136 3 4 1 (24) 0.0467 1 (24) 0.0467 0 (1) 0.0467 036 3 5 1 (2) 0.0634 1 (2) 0.0634 0 (1) 0.0633 036 3 6 2 (3) 0.0828 2 (3) 0.0828 2 (4) 0.0827 036 3 7 3 (9) 0.1062 3 (3) 0.1065 3 (6) 0.1060 0.0636 3 8 4 (18) 0.1336 4 (5) 0.1336 3 (3) 0.1337 0.1336 3 9 5 (21) 0.1661 6 (114) 0.1667 4 (1) 0.1671 0.3336 3 10 6 (27) 0.2050 6 (6) 0.2060 5 (2) 0.2067 0.6436 3 11 7 (36) 0.2511 7 (9) 0.2526 6 (4) 0.2544 1.1836 3 12 8 (54) 0.3063 8 (11) 0.3101 7 (5) 0.3111 1.6736 3 13 8 (18) 0.3811 9 (23) 0.3847 8 (3) 0.3779 2.01

NOTE: A boldfaced value indicates that it is better than the value of corresponding existing uniform design. The uniform designs are from the UD website, http://www.uic.edu.hk/isci/.

is small, several existing uniform designs have less discrepancythan the permuted GMA designs; however, this is not surprisingbecause we limit our search to designs that are combinatori-ally isomorphic to the initial design and we use the maximindistance criterion. If we select designs using the discrepancy,we can further improve the CD values in several cases, whichcan be seen from Table 2 where a few standard GMA designshave smaller CD values than permuted GMA designs. Whenthe number of factors or runs becomes larger, permuted GMAdesigns often have larger L1-distance and lower CD value thanexisting uniform designs. Furthermore, all of permuted GMAdesigns are orthogonal arrays of strength two (i.e., A2 = 0)because level permutations do not change orthogonality. In con-trast, many existing uniform designs with 27, 32, and 36 runsare not orthogonal arrays (i.e., A2 > 0); see the last column of

Table 2. In conclusion, with the exception of a few cases, per-muted GMA designs constructed here have better space-fillingproperties than existing uniform designs and are recommendedfor use in practice.

To investigate the relationship between the maximin distancecriterion and uniformity, we further conduct all level permuta-tions for MA 81-run designs from Xu (2005) for n = 5–18 andobtain both maximin distance design and minimum CD designfor each n. Table 3 compares their ranks and percentages of theranks under CD and maximin criterion. As explained earlier, weonly need to evaluate total 3n−4 permuted designs. From Table3, except for n = 5, the standard MA designs from Xu (2005)are all ranked at or near the bottom under both criteria. Forn = 5–12, maximin distance designs coincide with minimumCD designs so both have rank one under the other criterion.

Dow

nloa

ded

by [

Sout

hern

Met

hodi

st U

nive

rsity

] at

22:

12 1

8 N

ovem

ber

2014

1142 Journal of the American Statistical Association, September 2014

Table 3. Ranks of 81-run designs under maximin distance and CD

Rank (percentage) under CD Rank (percentage) under maximin

Maximin Standard Minimum Standardn Total (3n−4) distance design MA design CD design MA design

5 3 1 (*) 1 (*) 1 (*) 1 (*)6 9 1 (*) 9 (100%) 1 (*) 9 (100%)7 27 1 (*) 27 (100%) 1 (*) 27 (100%)8 81 1 (*) 68 (84%) 1 (*) 53 (65%)9 243 1 (*) 223 (92%) 1 (*) 225 (93%)

10 729 1 (*) 475 (65%) 1 (*) 445 (61%)11 2187 1 (*) 2184 (99.9%) 1 (*) 1795 (82%)12 6561 1 (*) 6539 (99.7%) 1 (*) 5137 (78%)13 19683 235 (1.2%) 18760 (95%) 421 (2%) 18067 (92%)14 59049 401 (0.7%) 57955 (98%) 35929 (61%) 58730 (99%)15 177147 2201 (1.2%) 175637 (99%) 84709 (48%) 174076 (98%)16 531441 19993 (3.8%) 528749 (99%) 476425 (90%) 530482 (99.8%)17 1594323 14993 (0.9%) 1593573 (99.95%) 1173691 (74%) 1593604 (99.95%)18 4782969 19521 (0.4%) 4781307 (99.97%) 4674673 (98%) 4734298 (99%)

NOTE: A star (*) means that the design is the best among all permuted designs.

When n > 12, maximin distance designs differ from minimumCD designs. The maximin distance designs are ranked withintop 4% under CD whereas the minimum CD designs are ranked48% or above under maximin distance for n > 13. This indi-cates that maximin distance designs have better space-fillingproperties than minimum CD designs.

6. CONCLUSIONS

We establish a general relationship between GWP and a wideclass of space-filling measures for designs with any number oflevels. We show that GMA designs tend to have good space-filing properties under various discrepancies and distance mea-sures. We obtain several novel relationships between distancedistribution and GWP under the L1-distance and show thatthere exist some level permutations to increase the L1-distanceof many existing saturated designs. These results can be ex-tended easily to Lp-distance for p > 1. We propose a two-stepconstruction procedure and obtain many space-filling fractionalfactorial designs with 3, 4, and 5 levels. These new designs havebetter space-filling properties, such as larger distance and lowerdiscrepancy, than standard GMA designs and existing uniformdesigns, and are recommended for use in practice.

APPENDIX: PROOFS

To prove Lemma 1, we need the following lemma from Tang, Xu,and Lin (2012).

Lemma A.1. For two rows xi and xj of an (N, sn)-design D, denoteδij be the number of places where they take the same value. Then forany real number z > 1, we have

N∑i,j=1

zδij = N 2

(z + s − 1

s

)n n∑i=0

(z − 1

z + s − 1

)i

Ai(D). (A.1)

Proof of Lemma 1. For any two rows xi , xj ∈ D with Hammingdistance dH (xi , xj ) = n − δij , when all level permutations of D are

considered, each identical pair (l, l) occurs (s − 1)! times in the δij

positions of coincidence and each distinct pair (k, l) occurs (s − 2)!times in the other dH (xi , xj ) positions. Then

N 2∑

D′∈P(D)

φ(D′)

=N∑

i,j=1

∑D′∈P(D)

⎛⎝ ∏

k:xik=xjk

f (xik, xjk)

⎞⎠⎛⎝ ∏

k:xik �=xjk

f (xik, xjk)

⎞⎠

=N∑

i,j=1

⎛⎜⎜⎝((s − 1)!)δij

∑y=(y1,...,yδij

)∈Zδijs

δij∏k=1

f (yk, yk)

⎞⎟⎟⎠

×

⎛⎜⎝((s − 2)!)n−δij

∑y,z∈Z

n−δijs ,dH ( y,z)=n−δij

n−δij∏k=1

f (yk, zk)

⎞⎟⎠

=N∑

i,j=1

((s − 1)!

s−1∑k=0

f (k, k)

)δij⎛⎝(s − 2)!

s−1∑k=0

∑l �=k

f (k, l)

⎞⎠

n−δij

= (s!)n(

c1

s(s − 1)

)n N∑i,j=1

cδij

2

where c1 and c2 are defined in Lemma 1. Notice that c2 > 1 undercondition (3). Then the result in (4) follows from Lemma A.1. �

Proof of Theorem 4. Denote G1 = s + 2∑s−1

k=1(s − k)ρk and G2 =s2 − G1. Then Equation (18) can be rewritten as

φ̄(D) = 1

s2n

n∑i=0

1

(s − 1)iGn−i

1 Gi2Ai(D). (A.2)

Let G3 = ∂G1/∂ρ = 2∑s−1

k=1 k(s − k)ρk−1. When ρ = 1, G1 = s2,G2 = 0 and G3 = (s3 − s)/3. Differentiating (A.2) with respect to ρ,we have

∂φ̄(D)

∂ρ= 1

s2n

n∑i=0

Ai(D)

(s − 1)i

× ((n − i)Gn−i−1

1 G3Gi2 + iGn−i

1 Gi−12 (−G3)

). (A.3)

Dow

nloa

ded

by [

Sout

hern

Met

hodi

st U

nive

rsity

] at

22:

12 1

8 N

ovem

ber

2014

Zhou and Xu: Space-Filling Fractional Factorial Designs 1143

We obtain (20) from (A.3) by letting ρ = 1. Denote G4 = ∂G3/∂ρ =2∑s−1

k=1 k(k − 1)(s − k)ρk−2. When ρ = 1, G4 = (s − 2)(s − 1)s(s +1)/6. Differentiating (A.3) with respect to ρ again, we have

∂2φ̄(D)

∂ρ2= 1

s2n

n∑i=0

Ai(D)

(s − 1)i(G4

((n − i)Gn−i−1

1 Gi2 − iGn−i

1 Gi−12

)+ G2

3

((n − i)(n − i − 1)Gn−i−2

1 Gi2

− 2i(n − i)Gn−i−11 Gi−1

2 + i(i − 1)Gn−i1 Gi−2

2

)). (A.4)

Letting ρ = 1 in (A.4) and combining the second derivative of (19),we have

1

N

m∑k=1

k(k − 1)B̄k(D) = C0 + C1A1(D) + 2

(s + 1

3s

)2

A2(D),

(A.5)

where C0 = n(s2 − 1)(3s(s − 2) + 2(s2 − 1)(n − 1))/(18s2), C1 =−(s + 1)(3s(s − 2) + 4(n − 1)(s2 − 1))/(9s2). Combining (20) and(A.5), we obtain (21). In general, when we differentiate (A.2) r timeswith respect to ρ and then let ρ = 1, the coefficient of Ai(D) is 0 fori > r because G2 = 0 when ρ = 1, and the coefficient of Ar (D) isequal to s−2n(−1)r r!Gr

3Gn−r1 /(s − 1)r = (−1)r r!(s + 1)r/(3s)r when

ρ = 1. We obtain the result in (22) by induction. This completes theproof. �

To prove Theorem 5, we need the following two lemmas. Mukerjeeand Wu (1995) showed the Hamming distance between any pair ofrows of a saturated OA(N, sn, 2) is equal to N/s, so the Hammingdistance distribution is B0(D) = 1, BN/s = N − 1 and Bi(D) = 0 fori �= 0 or N/s. Then from the MacWilliams transform in (14), we easilyhave the following lemma.

Lemma A.2. The GWP of a saturated OA(N, sn, 2) is

Ai(D) = 1

N(Pi(0; n, s) + (N − 1)Pi(N/s; n, s)) , i = 0, 1, . . . , n,

(A.6)

where Pi(x; n, s) are the Krawtchouk polynomials.

The other lemma, from MacWilliams and Sloane (1977), is on aproperty of Krawtchouk polynomials.

Lemma A.3. For nonnegative integers n, x, and s with n ≥ x, s ≥ 2and 0 < z < 1,

n∑i=0

Pi(x; n, s)zi = (1 + (s − 1)z)n−x(1 − z)x . (A.7)

Proof of Theorem 5.

(i) Since D is a saturated design, substituting (A.6) into (17), wehave

φ̄(D) = 1

N

(c2 + s − 1

c2s

)n n∑i=0

(c2 − 1

c2 + s − 1

)i

× (Pi(0; n, s) + (N − 1)Pi(N/s; n, s)) .

Denote z = (c2 − 1)/(c2 + s − 1). By Lemma A.3, we have

φ̄(D) = 1

N

(c2 + s − 1

c2s

)n

((1 + (s − 1)z)n + (N − 1)

× (1 + (s − 1)z)n−N/s (1 − z)N/s),

which can be simplified as φ̄(D) = N−1(1 + (N − 1) (c2)−N/s).Since c2 = s(s − 1)/c1 and c1 = ∑s−1

k=0

∑l �=k ρd(k,l) =

2∑s−1

k=1(s − k)ρk for L1−distance, we have

φ̄(D) = 1

N

⎛⎝1 + (N − 1)

(2

s(s − 1)

s−1∑k=1

(s − k)ρk

)N/s⎞⎠ .

(A.8)

Comparing the coefficients of ρk in (A.8) with that in (19), weobtain the result of Theorem 5(i).

(ii) When s = 3, Equation (A.8) simplifies to

φ̄(D) = 1

N

(1 + (N − 1)

(2ρ + ρ2

3

)N/3)

= 1

N

⎛⎜⎝1 + (N − 1)

(4

3

)N/3

×2N/3∑

k=N/3

(N/3

k − N/3

)(ρ

2

)k

⎞⎟⎠.

We obtain Theorem 5(ii) by comparing the coefficients of ρk

again. �

SUPPLEMENTARY MATERIALS

The online supplementary materials contain (i) the proof ofTheorem 3, (ii) some space-filling fractional factorial designswith 16, 25, and 32 runs, and (iii) Matlab code for conductinglevel permutations.

[Received December 2012. Revised August 2013.]

REFERENCES

Al-Shyoukh, I., Yu, F., Feng, J., Yan, K., Dubinett, S., Ho, C. M., Shamma, J. S.,and Sun, R. (2011), “Systematic Quantitative Characterization of CellularResponses Induced by Multiple Signals,” BMC Systems Biology, 5, 88.[1134]

Ba, S., and Joseph, V. R. (2011), “Multi-Layer Designs for Computer Experi-ments,” Journal of the American Statistical Association, 106, 1139–1149.[1135]

Butler, N. A. (2005), “Generalised Minimum Aberration Construction Resultsfor Symmetrical Orthogonal Arrays,” Biometrika, 92, 485–491. [1139]

Cheng, C. S., Deng, L. Y., and Tang, B. (2002), “Generalized Minimum Aber-ration and Design Efficiency for Nonregular Fractional Factorial Designs,”Statistica Sinica, 12, 991–1000. [1134]

Cheng, S. W., and Ye, K. Q. (2004), “Geometric Isomorphism and MinimumAberration for Factorial Designs With Quantitative Factors,” The Annals ofStatistics, 32, 2168–2185. [1134]

Chou, T. C. (2010), “Drug Combination Studies and Their Synergy Quantifi-cation Using the Chou-Talalay Method,” American Association for CancerResearch, 70, 440–445. [1134]

Ding, X., Xu, H., Hopper, C., Yang, J., and Ho, C. M. (2013), “Use of Frac-tional Factorial Designs in Antiviral Drug Studies,” Quality and ReliabilityEngineering International, 29, 299–304. [1134]

Fang, K. T., Li, R. Z., and Sudjianto, A. (2006a), Design and Modeling forComputer Experiments, Boca Raton: Chapman and Hall/CRC. [1134]

Fang, K. T., Lu, X., and Winker, P. (2003), “Lower Bounds for Centered andWrap-Around L2-Discrepancies and Construction of Uniform Design byThreshold Accepting,” Journal of Complexity, 20, 268–272. [1140]

Fang, K. T., Maringer, D., Tang, Y., and Winker, P. (2006b), “Lower Boundsand Stochastic Optimization Algorithms for Uniform Designs With Threeor Four Levels,” Mathematics of Computation, 75, 859–878. [1140]

Fang, K. T., and Mukerjee, R. (2000), “A Connection Between Uniformity andAberration in Regular Fractions of Two-Level Factorials,” Biometrika, 87,193–198. [1134,1137]

Dow

nloa

ded

by [

Sout

hern

Met

hodi

st U

nive

rsity

] at

22:

12 1

8 N

ovem

ber

2014

1144 Journal of the American Statistical Association, September 2014

Fang, K. T., and Wang, Y. (1994), Number-Theoretic Methods in Statistics,London: Chapman and Hall. [1134]

Fries, A., and Hunter, W. G. (1980), “Minimum Aberration 2k−p Designs,”Technometrics, 22, 601–608. [1134]

Hedayat, A. S., Sloane, N. J. A., and Stufken, J. (1999), Orthogonal Arrays:Theory and Applications, New York: Springer. [1139]

Hickernell, F. J. (1998), “A Generalised Discrepancy and Quadrature ErrorBound,” Mathematics of Computation, 67, 299–322. [1134,1136]

Jaynes, J., Ding, X., Xu, H., Wong, W. K., and Ho, C. M. (2013), “Applicationof Fractional Factorial Designs to Study Drug Combinations,” Statistics inMedicine, 32, 307–318. [1134]

John, P. W. M., Johnson, M. E., Moore, L. M., and Ylvisaker, D. (1995),“Minimax Distance Designs in Two-Level Factorial Experiments,” Journalof Statistical Planning and Inference, 44, 249–263. [1135]

Johnson, M. E., Moore, L. M., and Ylvisaker, D. (1990), “Minimax and Max-imin Distance Design,” Journal of Statistical Planning and Inference, 26,131–148. [1135,1137]

Kerr, M. K. (2001), “Bayesian Optimal Fractional Factorials,” Statistica Sinica,11, 605–630. [1135]

Ma, C. X., and Fang, K. T. (2001), “A Note on Generalized Aberration inFactorial Designs,” Metrika, 53, 85–93. [1134,1137]

MacWilliams, F. J., and Sloane, N. J. A. (1977), The Theory of Error-CorrectingCodes, Amsterdam: North-Holland. [1143]

Morris, M. D., and Mitchell, T. J. (1995), “Exploratory Designs for Compu-tational Experiments,” Journal of Statistical Planning and Inference, 43,381–402. [1137]

Mukerjee, R., and Wu, C. F. J. (1995), “On the Existence of Saturated and NearlySaturated Asymmetrical Orthogonal Arrays,” The Annals of Statistics, 23,2102–2115. [1138,1143]

Straetemans, R., O’Brien, T., Wouters, L., Dun, J. V., Janicot, M., Bijnens,L., Burzykowski, T., and Aerts, M. (2005), “Design and Analysis of DrugCombination Experiments,” Biometrical Journal, 47, 299–308. [1134]

Tang, B. (2001), “Theory of J-Characteristics for Fractional Factorial Designsand Projection Justification of Minimum G2-Aberration,” Biometrika, 88,401–407. [1134]

Tang, B., and Deng, L. Y. (1999), “Minimum G2-Aberration for Non-RegularFractional Factorial Designs,” The Annals of Statistics, 27, 1914–1926.[1134]

Tang, Y., and Xu, H. (2013), “An Effective Construction Method for Multi-level Uniform Designs,” Journal of Statistical Planning and Inference, 143,1583–1589. [1134,1136,1140]

Tang, Y., Xu, H., and Lin, D. K. J. (2012), “Uniform Fractional Factorial De-signs,” The Annals of Statistics, 40, 891–907. [1134,1136,1139,1142]

Winker, P., and Fang, K. T. (1997), “Application of Threshold-Accepting tothe Evaluation of the Discrepancy of a Set of Points,” SIAM Journal onNumerical Analysis, 34, 2028–2042. [1140]

Xu, H. (2003), “Minimum Moment Aberration for Nonregular Designs andSupersaturated Designs,” Statistica Sinica, 13, 691–708. [1139]

——— (2005), “A Catalogue of Three-Level Regular Fractional Factorial De-signs,” Metrika, 62, 259–281. [1138,1140,1141]

Xu, H., and Wu, C. F. J. (2001), “Generalized Minimum Aberration forAsymmetrical Fractional Factorial Designs,” The Annals of Statistics, 29,1066–1077. [1134,1135,1137]

Zhou, Y. D., Fang, K. T., and Ning, J. H. (2013), “Mixture Discrepancy for Quasi-Random Point Sets,” Journal of Complexity, 29, 283–301. [1134,1137]

Zhou, Y. D., Ning, J. H., and Song, X. B. (2008), “Lee Discrepancy and ItsApplications in Experimental Designs,” Statistics & Probability Letters, 78,1933–1942. [1134,1137]

Dow

nloa

ded

by [

Sout

hern

Met

hodi

st U

nive

rsity

] at

22:

12 1

8 N

ovem

ber

2014