sequence covering arrays - clemson universitygoddard/mini/2012/colbourn.pdfcovering arrays charles...

SequenceCovering Arrays

Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm

A ProductConstruction

Conclusion

Sequence Covering Arrays

Charles J. Colbourn1

1School of Computing, Informatics, and Decision Systems EngineeringArizona State University

(with Yeow Meng Chee, Daniel Horsley, Junling Zhou)

Clemson Mini-ConferenceClemson University, November 2012


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Testing Event Sequences

I In some processes, for example in manufacturing, aset of tasks must be carried out in a certainsequence.

I But people are not good at following instructions, andsometimes do the steps (or events) in the wrongorder.

I If certain subsets are done in the wrong order, thismay cause the wrong behaviour.

I So we want to test the system to make sure thatwhen users do some steps in the wrong order, eitherthe process still works or the user gets an errormessage.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion


I Suppose that there are v steps in the process.I Suppose that errors can be caused by performing

some subset of t or fewer steps in a certain order.I We want to ensure that for every subset of t or fewer

steps, we perform the steps in each of the t! ordersat least once – note that in doing this, we still performall v steps in some order.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion


I This problem was introduced by Kuhn, Higdon,Kacker, Lawrence, and Lei in April 2012 at theWorkshop on Combinatorial Testing.

I They give a basic lower bound on the number oftests needed, and a heuristic algorithm forconstructing tests.

I Since then, two constraint programming methodshave been published.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Permutation t-Coverings

I A t-subpermutation of {0, . . . , v − 1} is a t-tuple(x1, . . . , xt ) with xi ∈ {0, . . . , v − 1} for 1 ≤ i ≤ t , andxi 6= xj when i 6= j .

I A permutation π of {0, . . . , v − 1} covers thet-subpermutation (x1, . . . , xt ) if π−1(xi) < π−1(xj)whenever i < j .

I For example, with v = 5 and t = 3, (4,0,3) is a3-subpermutation that is covered by the permutation4 2 0 3 1.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Permutation t-Coverings

I A permutation covering of order v and strength t is aset Π = {π1, . . . , πN} where πi is a permutation of{0, . . . , v − 1}, and every t-subpermutation of{0, . . . , v − 1} is covered by at least one of thepermutations {π1, . . . , πN}.

I Call one a PermC(N; t , v).


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Permutation t-CoveringExample

PermC(8;3,5) – t = 3, v = 5, N = 8

PermC SeqCA4 2 0 3 1 2 4 1 3 01 4 3 0 2 3 0 4 2 13 1 2 0 4 3 1 2 0 40 2 4 1 3 0 3 1 4 22 1 3 4 0 4 1 0 2 30 3 4 1 2 0 3 4 1 23 0 2 1 4 1 3 2 0 44 1 2 0 3 3 1 2 4 0


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Sequence Covering Arrays

I A sequence covering array, SeqCA(N; t , v) is anN × v array A = (aij) for which

I every row forms a permutation of the v symbols, andI in every set of t columns c1, . . . , ct , and for every

permutation ψ of {1, . . . , t}, there is a row ρ such thataρcψ(i < aρcψ(i+1 for 1 ≤ i < t .

I (in other words, in every set of t columns, every‘pattern’ appears on these t columns in at least onerow)

I This is equivalent to a PermC(N; t , v) – justinterchange the roles of columns and symbols.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Sequence Covering ArraysThe Existence Question

I Given t and v , what is the smallest N for which aSeqCA(N; t , v) exists?

I Call this number SeqCAN(t , v).I SeqCAN(t , v) ≥ t!I SeqCAN(t , v) ≤

(vt

)t!

I SeqCAN(2, v) = 2 for all v ≥ 2 – Just take theidentity permutation and its reversal!


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Sequence Covering ArraysThe Case of Equality

I For t ≥ 3, when is SeqCAN(t , v) = t!?I Levenshtein (1991) showed that it always is when

v ≤ t + 1, and conjectured that it never is whenv > t + 1.

I Mathon and Tran Van Trung (1999) showed thatSeqCAN(4,6) = 4!, and showed thatSeqCAN(t , v) > t! when v exceeds a certainquadratic function of t .

I We have shown that SeqCAN(t , v) > t! when v ≥ 2t .I We need a way to produce lower bounds.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Covering Array. Definition

I Let N, k , t , and v be positive integers.I Let C be an N × k array with entries from an alphabet

Σ of size v ; we typically take Σ = {0, . . . , v − 1}.I When (ν1, . . . , νt ) is a t-tuple with νi ∈ Σ for 1 ≤ i ≤ t ,

(c1, . . . , ct ) is a tuple of t column indices(ci ∈ {1, . . . , k}), and ci 6= cj whenever νi 6= νj , thet-tuple {(ci , νi) : 1 ≤ i ≤ t} is a t-way interaction.

I The array covers the t-way interaction{(ci , νi) : 1 ≤ i ≤ t} if, in at least one row ρ of C, theentry in row ρ and column ci is νi for 1 ≤ i ≤ t .

I Array C is a covering array CA(N; t , k , v) of strength twhen every t-way interaction is covered.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Covering ArrayCA(13;3,10,2)

0 0 0 0 0 0 0 0 0 01 1 1 1 1 1 1 1 1 11 1 1 0 1 0 0 0 0 11 0 1 1 0 1 0 1 0 01 0 0 0 1 1 1 0 0 00 1 1 0 0 1 0 0 1 00 0 1 0 1 0 1 1 1 01 1 0 1 0 0 1 0 1 00 0 0 1 1 1 0 0 1 10 0 1 1 0 0 1 0 0 10 1 0 1 1 0 0 1 0 01 0 0 0 0 0 0 1 1 10 1 0 0 0 1 1 1 0 1


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Covering Array

I CAN(t , k , v) is the minimum N for which aCA(N; t , k , v) exists.

I The basic goal is to minimize CAN(t , k , v).I It is easy to establish that, when t and v are both

fixed, CAN(t , k , v) is O(log k).


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Lower Bounds for Sequence Covering Arrays

TheoremLet k and t be integers satisfying k ≥ t ≥ 3. Whenever0 ≤ a < t , the size of a sequence covering array for kevents with strength t is at least

a!CAN(t − a, k − a,a + 1)


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion


I Choose any a events.I For each ordering (e1, . . . ,ea) of the a events, select

the permutations of the permutation covering inwhich the a selected events appear in the chosenorder; suppose that there are n such permutations.

I Form an n × (k − a) array A whose columns areindexed by the remaining events, and whose rowsare indexed by the permutations selected.

I In the row indexed by π and the column indexed byevent e,

I place 0 if π(e) < π(e1)I place a if π(e) > π(ea)I otherwise, place i when π(ei ) < π(e) < π(ei+1).


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Lower BoundExample

SeqCA(9;3,7) – t = 3, v = 7, N = 9take a = 1 and use symbol 6

0 5 6 4 3 2 12 1 6 5 0 3 43 4 5 1 2 0 66 1 2 4 3 0 50 1 4 3 6 2 55 2 3 4 6 0 13 6 1 5 0 2 44 0 1 2 5 6 36 2 5 1 3 4 0

0 1 1 1 1 01 0 0 1 1 10 0 0 0 0 01 1 1 1 1 10 0 1 0 0 11 1 0 0 0 01 1 1 0 1 10 0 0 1 0 01 1 1 1 1 1


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion


I The theorem can give better estimates.I For t = 4, we obtain the lower bounds

I SeqCAN(4, v) ≥ CAN(4, v ,1) by taking a = 0,I SeqCAN(4, v) ≥ CAN(3, v − 1,2) by taking a = 1,I SeqCAN(4, v) ≥ 2CAN(2, v − 2,3) by taking a = 2,I SeqCAN(4, v) ≥ 6CAN(1, v − 3,4) = 24 by taking

a = 3.I Taking a = 0 always gives a trivial bound.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Upper Bounds for Sequence Covering ArraysRandom Arrays

I Suppose that you pick a permutation of{0, . . . , v − 1} uniformly at random.

I Any specific t-subpermutation is covered withprobability 1

t! .I And it fails to be covered with probability t!−1

t! .I So if you pick N permutations of {0, . . . , v − 1}

uniformly at random and independently, any specifict-subpermutation is covered with probability1−

( t!−1t!

)N.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion


I There are v !(v−t)! t-subpermutations.

I When N permutations are chosen, eachsubpermutation is not covered with probability( t!−1

t!

)N.

I So the probability that at least one subpermutation isnot covered is at most v !

(v−t)!

( t!−1t!

)N.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion


I When v !(v−t)!

( t!−1t!

)N< 1, a SeqCA(N; t , v) must

exist!I So when t is fixed, this gives an O(log v) upper

bound on the number of permutations needed.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Upper Bounds for Sequence Covering ArraysGreedy Random Arrays

I Instead consider generating the set of permutationsone permutation at a time.

I Idea: Always pick the next permutation so that itcovers the largest possible number ofas-yet-uncovered t-subpermutations.

I Suppose that after i permutations have already beenchosen, there remain Ri t-subpermutations to becovered.

I Then R0 = v !(v−t)! .


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion


I Consider selecting the (i + 1)st permutation.I If we choose it uniformly at random, then as before

every (as-yet-uncovered) t-subpermutation iscovered by the chosen permutation with probability1t! .

I So using the linearity of expectations (“the sum of theexpectations is the expectation of the sum”), theexpected number of t-subpermutations covered forthe first time by the chosen permutation is Ri

t! .


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion


I But then some permutation covers at least Rit!

t-subpermutations for the first time, and hence

Ri+1 ≤ Ri −Ri

t!.

I Combine with R0 = v !(v−t)! to get

Ri ≤(

t!− 1t!

)i v !

(v − t)!.

I When RN < 1, a SeqCA(N; t , v) exists – and thegreedy method guarantees to find one!


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Stein-Lovász-Johnson

I The greedy strategy has proved remarkablysuccessful for a variety of combinatorial coveringproblems.

I Most surprising is that in many cases it has beenshown that we can select the next row efficiently (intime polynomial in the number of columns when thestrength t is fixed).

I Such efficient methods have been found forI covering arrays (Bryce and Colbourn (2007, 2009))I perfect hash families (Colbourn (2008))I separating, distributing, strengthening, scattering, . . .

hash families (Colbourn (2011); Colbourn, Horsley,and Syrotiuk (2012))


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Stein-Lovász-Johnson

I The key observations in these efficient methods areI It is enough to choose the next “row” so that it covers

at least the average number of as-yet-uncovered“entities”

I We can fill in the row one entry at a time so that theexpected number covered by any completion of therow never decreases, and what we need to do is tofind an entry – efficiently – that does not decreasethis expected number.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Strength Three

I Using a notion of “signing” a sequence coveringarray, we have developed a direct productconstruction.

I Whenever (signed) SeqCA(N; 3, v) andSeqCA(M; 3,w) both exist, there is a (signed)SeqCA(N + M; 3, vw).

I This improves on all of the known computationalmethods when v ≥ 40, but only applies for strengththree.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Some Computation

Events t = 3v EX U UR O DR D BTI ER REC5 7 17 16 8 10 8 7

10 9 30 28 14 14 12 9 1115 10 37 34 18 16 14 10 1120 42 38 22 18 16 12 19 1230 49 46 26 22 19 15 23 1540 54 50 32 24 21 17 27 1650 58 52 34 26 23 19 31 1660 61 56 38 26 24 21 34 1770 64 58 40 28 25 22 36 1780 66 60 42 30 26 24 38 1890 68 62 44 30 27 18


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Open Problems # 1

I Can we explain why including the reversal of apermutation seems like a “good idea” but does notnecessarily lead to the best result?

I Can we determine when SeqCAN(t , v) = t!? This isrelated to Directed Steiner t-designs.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Open Problems # 2

I Find combinatorial direct constructions for sequencecovering arrays.

I Find combinatorial recursive constructions forsequence covering arrays with t ≥ 4.


Charles J.Colbourn


Covering Arrays

Lower Bounds

A GreedyAlgorithm


Conclusion

Open Problems # 3

I Can SeqCAN(3, v) be determined exactly?I Can we construct a sequence covering array from a

covering array? What conditions are necessary?sufficient?

sequence covering arrays - clemson universitygoddard/mini/2012/colbourn.pdfcovering arrays charles...

Documents