On Asymptotic Complexity of Linear Ordering the Paley Tournament Graphs:

First Prototypes with Self-Avoiding Walks

Franc Brglez and Yang Ho and Johnny NguyenComputer Science, NC State University

Raleigh, NC 27695, USAEmail: brglez@ncsu.edu

Abstract—The linear ordering problem (LOP) arises in anumber of divers domains. The notoriously hard class of LOP in-stances, even for sizes L < 50, is represented by Paley tournamentgraphs. Instances of these graphs, along with best-known-values(BKVs), some of them proven optimum, are now readily availableon the Internet. There is no published record of asymptoticcomplexity to solve instances of Paley tournament graphs, alsoknown as the pal instances.

In this paper we propose a new stochastic solver basedon a variation of a self-avoiding walk. The solver we proposehas the platform-independent asymptotic walkLength complexityof 0.09336 ⇤ 1.7727L (in number of steps). We demonstrateclose correlation of walkLength with the asymptotic runtimecomplexity on different platforms as well under two rapidprototype implementations: one in Tcl, one in Python. Mostimportantly, the paper presents two significant improvements inBKVs: pal(31, 300) improves on pal(31, 285) and pal(43, 597)improves on pal(43, 543).

I. INTRODUCTION

The linear ordering problem (LOP) arises in a number ofdivers domains, ranging from applications in triangulation ofinput-output matrices in economics, chronological orderingof artifacts in archeology, job-shop scheduling, schedulingof tournaments, query processing of wireless data broadcast,ordering of mutations as well as amino acid replacements inpolypeptide, learning by ordering for better language trans-lation, mathematical psychology, among others. For a com-prehensive overview of the linear ordering problem, and thesolutions provided by exact and heuristic methods, see [1].

While there is a prolific number of heuristic solvers, only afew exact methods and LOP solvers are known to exist. Therelatively short list of solvers that implement exact algorithmsincludes [2], [3], and [4]. The importance of finding an exactsolution for given LOP cannot be overstated: clearly they areimportant in a number of practical applications where runtimeconstraints are not critical and we can afford to wait for severaldays to get an exact solution. However, they also are importantas the initial source of best-known-solutions (BKVs). Thesesolutions provide a foundation for an experimental statisticalmodel that reliably predicts the asymptotic complexity of aheuristic linear ordering solver – when running a well-definedclass of LOP instances of increasing size L.

The notoriously hard set of LOP instances, even for sizesL < 50, is represented by Paley tournament graphs [5].These and many other instances are introduced in the surveypaper [6] and the appendix [7]. To access all instances and theExcel file lolib method exp.xls with BKVs, see [8]. Names offiles representing matrices of Paley graph tournaments containthe string pal which we also use in Eq. 1 below.

The asymptotic experiments with the LOP solver introducedin this paper return two new BKVs for L = 31 and L = 43. Theimproved BKVs are significant, see the paragraph followingEq. 1 for details.

BKV(L, pal) = {pal(11, 35), pal(13, 55), pal(19, 107), (1)pal(23, 161), pal(27, 252), pal(31, 300)⇤,

pal(43, 597)⇤, pal(55, 1084)⇤}

In the Excel file lolib method exp.xls under [8] we find best-known-values for pal instances, but no runtimes. The solverslisted are {TS, MA, VNS, SA, SS, GRASP}. The condensedsummary is below:

• pal(31, 285) as the best value by some but not all solvers• pal(43, 543) as the best value by some but not all solvers• pal(55, 1045) as the best value by some but not all solvers

The same Excel file also lists pal(31, 300)⇤, pal(43, 597)⇤,pal(55, 1084)⇤ as the Upper Bound Norm. In view of resultswith our solver, we suggest to use the values in Eq. 1 as the setof target BKVs when designing asymptotic experiments withpal instances for any LOP solvers in the future.

The number of heuristic methods and solvers is indeedprolific, one more reason to categorize them [1]. For a briefchronological perspective, we list some of the representativeapproaches: [9], [10], [11], [12], [13], [14], [15], [16],[17], [18], [19]. The only paper that includes experimentswith pal instances is [17]. None of these experiments reachthe BKVs posted in the appendix [7]. What they report is ameasure of distances from these BKVs.Uncensored Experiments. Our experiments with combina-torial problems in three different domain have demonstratedintrinsic merits of statistical models that can predict theasymptotic runtime complexity of any solver that returns BKVsassociated a well-defined instance set in the problem domainof choice: see Figure 7 in [20], a number of plots in [21],and Figure 5 in [22]. We expect no less for the LOP solveroutlined in this paper, given the instance set defined in Eq. 1.

Fig. 1: A hypothetical tablet version of the linear ordering puzzle; a 4x4 arrangement 16 balls; each ball labeled with a number.The player can select any pair of columns to swap (corresponding rows) will track, then click for the swap to be completed.

We follow the same four principles in all of these experiments:(1) Select a set of at least 3 instances of increasing size from a well-defined problem class such as shown in Eq. 1.(2) Select N random initial seeds and invoke N solver instancesunder the conservative runtime limit tlmt on the same instance class;consider either N runs on a single processor or N runs on the gridof N processors.(3) Continue the runs until at least Nu � 100 runs reach the targetBKV with the first-passage-time < tlmt. We say that any such runis uncensored. In other words, the mean runtime value we measureis based strictly on at least 100 uncensored observations from theexperiment.(4) Increase the instance size and repeat the experiment with newinitial seeds and an increased runtime limit tlmt.

The larger the set of instances in the experiment, the morereliable is the asymptotic prediction model which is based onthe least-squares fit with respect to the observed uncensoredruntime mean values.

Paper outline. Section 2 introduces notation and definitions,Section 3 outlines the LOP algorithm and the LOP solver in-strumentation, Section 4 summarizes asymptotic experimentsto date. Section 5 concludes with plans for future work.

Typical papers on solutions of the linear ordering problemare more formal than the content of Figures 1, 2, and 3 in thispaper. The tutorial presentation style has been conditioned bythe undergraduate 3-credit-hour research project undertakenby two senior students who desired hands-on experiencewith topics in combinatorial optimization: solving the linearordering as an asymptotic progression of puzzles formulatedwith permutation coordinates as described in Figure 1, andsolving the lights-out problem described as an asymptoticprogression of puzzles formulated with binary coordinates,to be presented elsewhere. Rather than studying pseudo codeonly, students learned about new concepts and algorithms fromthe executable prototypes in Tcl, prepared by the instructor.Students also learned to rapidly prototype, test, and presentthe functionally equivalent solvers using Python.

II. NOTATION AND DEFINITIONS

Due to page limitation, we ask the reader to also consult therelated work [21] for clarifications about the notation.The LOP puzzle. An example of the LOP puzzle is shown inFigure 1. Each configuration is expressed with a permutationcoordinate & of length L. For L = 4, two permutationcoordinates are 1,2,3,4 and 3,2,1,4. The cost of arrangement⇥(&) under any permutation is defined as the negative sum ofall numbers in the matrix above the diagonal. The objective isto simultaneously swap pairs of rows and columns such thatthe negative sum above the diagonal is minimum.

The puzzle is played as follows. The player selects thecolumn pair such as 2,3 (a pair of bold green lines abovecolomn). By clicking on one of the pairs, columns 2,3 androws 2,3 are swapped and shown as the Step=1, decreasingthe negative sum of numbers above the diagonal to -9. Thispuzzle is solved in 4 steps. To solve this puzzle from anystarting configuration, in the least number of steps, requiresan efficient probing strategy to select the best coordinate foreach step as well as a good memory of the steps taken.

Formally, given a L ⇤ L matrix M of numbers with an all-zero diagonal, the cost function ⇥(&), to be minimized in termsof its permutation coordinate, is expressed with Eq. 2:

⇥(&) = (�1)LX

i=1

LX

j=i+1

M(&i, &j) (2)

See Figure 2 for an illustration of how Eq. 2 is evaluated.

Coordinate rank and rankMax. Both variables are computedfrom the standard inversion number of a permutation.

rank(&i) = inversionNumber(&i) (3)rankMax = L ⇤ (L� 1)/2 (4)

For simplicity of notation, we shall use rank and rankMax.

Coordinate distance. The distance between two permutationcoordinates a and b is defined as the absolute differencebetween the two ranks of each coordinate:

d(a, b) = |rank(a)� rank(b)| (5)

Fig. 2: Canonic views of the LOP puzzle instance in Figure 1. The permutation coordinate length L = 4 induces 4! = 24 statesor permutations, each evaluated by Equation 2. The graph bidirectional edges in the landscape graph next to the table areinduced by computing unit distances between the permutation coordinates. The vertex labels are state-weight pairs which canbe expanded to coordinate-weight pairs & : ⇥(&). On x-axis, vertices are ordered by the coordinate rank; on y-axis, verticesare ordered by the vertex weight. The two walks are examples of self-avoiding walks, both reaching the absorbing state Z1.

The landscape graph. We maintain several views of the land-scape graph, a data structure underlying the implementationof each of our stochastic combinatorial solvers; introductoryexamples for a variety of problem-specific structures areexamined in [23] and [24]. The landscape graph example inFigure 2 is an extension of the puzzle instance in Figure 1.

The principal columns in the table, coordinate and weight,are generated by exhaustive enumeration of the Eq. 2. Thecolumn state is a list of symbolic names for each coordinate,values under the column rank correspond to distances of eachcoordinate from the reference coordinate 1,2,3,...,L-1,L. Sym-bolic names rather than specific coordinates are typically usedto represent configurations or states when annotating Markovchains. The graph bidirectional edges in the landscape graphnext to the table are induced by computing unit distancesbetween the binary coordinates; we associate no self-loopswith this graph. The vertex labels are state-weight pairs whichcan be expanded to coordinate-weight pairs & : ⇥(&) whenneeded for clarity.

Formally, the landscape graph is represented with aweighted graph adjacency matrix and just like in the statetransition probability graph, we can define simple walks onthe landscape graph. A single state-weight pairs, Z1:�18is associated with the minimum weight and is denoted asabsorbing state. Since the vertex is declared absorbing, it hasincoming edges only. In the context of stochastic combinatorialoptimization, the concept of absorbing states is replaced withsolution coordinates, reaching the target value or BKV.Contiguous walks and pivot coordinates. Let the coordinate&0 be the initial coordinate from which the walk takes the firststep. Then the sequence

{&0, &1, &2, . . . , &j , . . . , &!} (6)

is called a walk list or a walk of length !, the coordinates&j are denoted as pivot coordinates and ⇥(&j) are denoted aspivot values. Given an instance of size L and its best upperbound ⇥ub

L , we say that the walk reaches its target value (andstops) when ⇥(&!) ⇥ub

L .We say that the walk is contiguous if the distance between

adjacent pivots is 1; i.e., given Eq. 5, we find

d(&j , &j�1) = 1, j = 1, 2, ...,! (7)

Distance=1 neighborhood. The distance=1 neighborhood ofa pivot coordinate &j is a set of permutation coordinates:

N (&j) = { &ij | d(&j , &ij) = 1, i = 1, 2, . . . , L� 1 } (8)

Self-avoiding walks (SAWs). We say that the walk is self-avoiding if all pivots in Eq. 6 are unique. We say that thewalk is composed of two or more walk segments if the initialpivot of each walk segment has been induced by a heuristicsuch as random restarts.

III. SAW.WANDERING: SOLVER AND INSTRUMENTATION

We view each walk or walk segment as a sequence of stepsthat chain a set of pivot coordinates:• at each step we compute the set of all adjacent coordinatesN (&j), also known as the local neighborhood of the pivotcoordinate &j• for each adjacent coordinate &ij , we probe or evaluate thecost function such as Eq. 2 for its value.

There is a large family of self-avoiding walks; some aremore efficient than others, depending on the problem. Whilestill new to LOP, we implemented the wandering walk first.

Wandering Walk. We describe the wandering walk in itssimplest form; formal details can be found in [23] and [24].

Fig. 3: An extended view of the LOP puzzle instance in Figure 2. Consider M:-11 as the pivot vertex with three adjacentvertices: E:-6, Q:-15, and I:-14. The cost of probing each vertex for its value directly with Equation 2 represents the simple ornaive probing approach. The tableau-based probing returns values for each of the adjacent vertices in constant time, leveragingthe values already computed by the pivot. For details about the complexity of probing the pivot and its entire neighborhoodof constant size L� 1, see text in the paper.

1: procedure SAW.wander()2: while true do3: (1) select a random coordinate and mark it as the ’initial

pivot’.4: (2) probe all unmarked adjacent coordinates, then select

and mark the coordinate with the ’best value’ as the newpivot. If multiple coordinates return the same value, arandom choice is made to avoid biasing the walk.

5: (3) continue the walk until either the ’best value’ ‘targetvalue’ or the walk is being trapped by adjacent coordinatesthat are already pivots;

6: (4) if the walk is trapped, restart the walk from a randomlyselected ‘new initial pivot’;

7: (5) manage the memory constraints with an efficient datastructure such as a hash table.

8: end while9: end procedure

For illustrations of two ‘wandering walks’ that find the min-imum of the LOP example in relatively few steps, see thelandscape graph in Figure 2.

Trapped pivots have been observed only rarely, and only forvery small instances. Issues that that arise when running thisalgorithm on LOP instances include:• with large instances, the length of the walk may increasethe number of hash collision and impact the runtime of thesolver; a random restart clears the hash.• as the size of the problem increases, the computational cost

of probing each adjacent coordinate of the pivot coordinatebefore making the decision on the next step of the walkcan become an important factor. For any LOP instance, thecomplexity of probing the pivot and its entire neighborhoodunder what we denote as simple or naive approach is

O(L2 + (L� 1) ⇤ L2) (9)

In [23] we illustrate efficient data structures to significantlyreduce the complexity of probing the entire distance=1 neigh-borhood of the pivot coordinate – under binary, permutation,and other coordinate encodings. In contrast to simple ornaive probing, we denote the more efficient approach as thetableau-based probing. We illustrate the differences betweenthe two approaches with the LOP puzzle instance in Figure 3.We see by inspection why the tableau-based probing of thepivot neighborhood is more efficient: rather than performingindependent evaluation of each neighborhood coordinate, weleverage the already computed data about the pivot to find, inconstant time, the cost at each neighborhood coordinate. Thecomplexity of probing the pivot and its entire neighborhoodunder the tableau-based probing is thus

O(L2 + L� 1) (10)

Solver Instrumentation. The wandering walk implementationof the current tcl/python solvers P.lopT/P.lopP is based

10 12 14 16 18 20 22

1050

500

5000

50000

instanceSize = 'L' (under instanceClass = 'pal')

wal

kLen

gth

[ste

ps]

P.lopT,S ... observed walkLength means P.lopT,T ... observed walkLength means P.lopT,S ... 0.23336326*1.7033^LP.lopT,T ... 0.21534923*1.7131^L

** sampleSize = 100 **P.lopT,S ... tcl, option SP.lopT,T ... tcl, option T

(a)

10 12 14 16 18 20 22

1050

500

5000

50000

instanceSize = 'L' (under instanceClass = 'pal')

wal

kLen

gth

[ste

ps]

P.lopT,S ... observed walkLength means P.lopP,S ... observed walkLength means P.lopT,S ... 0.23336326*1.7033^LP.lopP,S ... 0.14762501*1.7446^L

** sampleSize = 100 **P.lopT,S ... tcl, option SP.lopP,S ... python, option S

(b)

10 12 14 16 18 20 22

1050

500

5000

50000

instanceSize = 'L' (under instanceClass = 'pal')

wal

kLen

gth

[ste

ps]

P.lopT,T ... observed walkLength means P.lopP,T ... observed walkLength means P.lopP,T ... 0.21534923*1.7131^LP.lopP,T ... 0.18462781*1.7303^L

** sampleSize = 100 **P.lopT,T ... tcl, option TP.lopP,T ... python, option T

(c)

10 12 14 16 18 20 22

1050

500

5000

50000

instanceSize = 'L' (under instanceClass = 'pal')

wal

kLen

gth

[ste

ps]

P.lopP,S ... observed walkLength means P.lopP,T ... observed walkLength means P.lopP,S ... 0.14762501*1.7446^LP.lopP,T ... 0.18462781*1.7303^L

** sampleSize = 100 **P.lopP,S ... python, option SP.lopP,T ... python, option T

(d)

Fig. 4: Mean values of walkLength (in steps), based on observations and the asymptotic model of the tcl and the python ‘lop’solvers, each under two options: ‘S’ (isSimple) and ‘T’ (tableau). Under 100 randomly selected initial seeds for each instancein class ‘pal’, both solvers return 100 solutions with the known optimum value (’valueTarget’).

on the pseudo-code outlined in this section. Both solvers areinstrumented to measure runtime (in seconds) and stop eitherafter recording the first-passage-time on reaching BKV or afterthe reaching runtime limit ttmt. The solvers also maintaincounters such as cntProbe, cntTrapped, cntRestart, walkLengthand decision variables such as isCensored, targetReached. Foreach run under different initial seeds, results are returnedin standardized output table for statistical evaluation aftercompletion of N runs.

These counters are of critical importance: cntProbe andwalkLength provide a way to test and compare platform-independent equivalence of two solver implementation, suchas tcl/python solvers P.lopT/P.lopP. Before we pair any

two solver for runtime comparisons, we should ask whether theuncensored asymptotic models for cntProbe and walkLengthare equivalent.

IV. SUMMARY OF EXPERIMENTS

Our experiments follow the methodology that rigorously estab-lished the asymptotic performance model of the state-of-the-artstochastic solver for the low autocorrelation binary sequenceproblem [21].

All computations rely on individual processors accessedremotely. Each processor requires reservations and is alsosubject to expiring tokens in 24 hours. As a consequence, wecould not avoid censored experiments for the largest instances.

10 12 14 16 18 20 22

1e-03

1e-01

1e+01

instanceSize = 'L' (under instanceClass = 'pal')

runt

ime

[sec

onds

]

P.lopT,S ... observed runtime means P.lopT,T ... observed runtime means P.lopT,S ... 0.00002377*1.9921^LP.lopT,T ... 0.00001444*1.8886^L

** sampleSize = 100 **P.lopT,S ... tcl, option SP.lopT,T ... tcl, option T

(a)

10 12 14 16 18 20 22

1e-03

1e-01

1e+01

instanceSize = 'L' (under instanceClass = 'pal')

runt

ime

[sec

onds

]

P.lopT,S ... observed runtime means P.lopP,S ... observed runtime means P.lopT,S ... 0.00002377*1.9921^LP.lopP,S ... 0.00000619*2.0517^L

** sampleSize = 100 **P.lopT,S ... tcl, option SP.lopP,S ... python, option S

(b)

10 12 14 16 18 20 22

1e-03

1e-01

1e+01

instanceSize = 'L' (under instanceClass = 'pal')

runt

ime

[sec

onds

]

P.lopT,T ... observed runtime means P.lopP,T ... observed runtime means P.lopP,T ... 0.00001444*1.8886^LP.lopP,T ... 0.00000846*1.8676^L

** sampleSize = 100 **P.lopT,T ... tcl, option TP.lopP,T ... python, option T

(c)

10 12 14 16 18 20 22

1e-03

1e-01

1e+01

instanceSize = 'L' (under instanceClass = 'pal')

runt

ime

[sec

onds

]

P.lopP,S ... observed runtime means P.lopP,T ... observed runtime means P.lopP,S ... 0.00000619*2.0517^LP.lopP,T ... 0.00000846*1.8676^L

** sampleSize = 100 **P.lopP,S ... python, option SP.lopP,T ... python, option T

(d)

Fig. 5: Mean values of runtime (in seconds), based on observations and the asymptotic model of the tcl and the python ‘lop’solvers, each under two options: ‘S’ (isSimple) and ‘T’ (tableau). Under 100 randomly selected initial seeds for each instancein class ‘pal’, both solvers return 100 solutions with the known optimum value (’valueTarget’).

A typical processor is an Intel(R) Xeon(R) CPU E5645 @2.40GHz, 4 GB virtual memory [25].

All asymptotic experiments are summarized in three figures.Figures 4 and 5 compare asymptotic performance of two solverprototypes: one implemented in Tcl, the other in Python. Theseexperiments were completed as ‘uncensored’ and were partof the project course with students. The instances includedpal(11,35), pal(13,55), pal(19,107), and pal(23,161). Figure 6was completed after the course with all instances from Eq. 1.

Figure 4. This figure presents four asymptotic models forwalkLength: (a) with and without the tableau-probing underTcl. (b) with simple-probing only, but with two solvers: Tcl

and Python. (c) with tableau-probing only, but with twosolvers: Tcl and Python. (d) with and without the tableau-probing under Python. Not surprisingly, in this series ofexperiments, we observe cntProbe and walkLength and ideally,there is little if any observable difference between simple-probing and tableau-probing. Similary, we observe negligibledifferences in observed values from either solver. As expected,these observations are platform-independent and the two solverimplementations are indeed functionally equivalent.Figure 5. This figure summarizes the same 4-factor exper-iments discussed in Figure 4 except that now, we observeruntime: (a) with and without the tableau-probing under Tcl;with tableau-probing the Tcl solver cleary much faster and the

10 20 30 40 50 60

1e−0

21e

+01

1e+0

41e

+07

1e+1

0

L ('pal' instance sizes 11, 13, 19, 23, 27, 31, 43, 55), plot A

runt

ime

[sec

onds

]

●

●

●

●

●

?

?

?● solver P.lopT: 0.00000917*1.9349^L

instances 31, 43, 55 are censoredat 7200 seconds except for

2 instances at 31, new BKV=3001 instances at 43, new BKV=597

10 20 30 40 50 60

1e+0

11e

+04

1e+0

71e

+10

1e+1

3

L ('pal' instance sizes 11, 13, 19, 23, 27, 31, 43, 55), plot B

walk

Leng

th [

step

s]

●

●

●

●

●

?

?

?● solver P.lopT: 0.09336*1.7727^L

instances 31, 43, 55 are censored

at 7200 seconds except for

2 instances at 31, new BKV=300

1 instances at 43, new BKV=597

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●

●●●●●●●●

●●●●●

●

●

●

●●

●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●

●●●●●●

●●●●

●●●●

●

●

●●●●●●●●●●●●

●

●

●●

●●●●●●●

●

●●

●●●●●●●●

●●●

●●

●●●

●

●●

●

●●

●●●●●●●●●●●

●

●

●●

●

●

●

●●

●

●●●●

●

●●

●●●●

●●

●●

●

●●●

●●●●

●●

●●

●●●

●

●●●●●

●●

●●●

●

●●●

●●

●●●●●●

●

●●●

●

●

●

●●●●●●

●●

●

●

●●●●●

●●●●●●●●●●●

●●

●●●●

●●●

●

●

●

●●●●

●●●●●●

●●

●●●

●

●

●●

●●●

●●●●●

●

●

●●●●●●●

●

●

●●●

●●●●●●●

●

●●

●●●

●●●●●●●●●

●●●

●●●●●●●●●●

●

●●●●●●●

●●

●

●●●●●●●●●●●●

●●●●

●

●●●●

●

●●●●●●●

●●●●●

●●●

●●●●●●

●

●●●

●●●●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●●

●

●

●

●●

●

●

●●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

0e+00 1e+06 2e+06 3e+06 4e+06

050

010

0015

0020

0025

0030

0035

00

walkLength ('pal' instances of size 11, 13, 19, 23, 27), plot C

runt

ime

[sec

onds

]

data collected on two processors... one of the processors is faster

10 20 30 40 50 60

1e+0

01e

+02

1e+0

41e

+06

1e+0

8

L ('pal' instance sizes 11, 13, 19, 23, 27, 31, 43, 55), plot D

walk

Leng

thN

orm

alize

d [s

teps

]

●

●

●

●

●

?

?

?● solver P.lopT: 0.005548*1.5786^L

given that rankMax = L*(L−1)/2

we create

walkLengthNormalized = walkLength/rankMax

Fig. 6: A summary of the currently most accurate model of asymptotic complexity for pal instances, achieved with the currentLOP solver prototype P.lopT: walkLengthModel = 0.09336 ⇤ 1.7727L, runtimeModel = 0.00000917 ⇤ 1.9349L (in seconds).

gap should increase as the instance size increases. (b) withsimple-probing only, but with two solvers: Tcl and Python.Here, Python is the faster solver by about factor of 2. (c)with tableau-probing only, but with two solvers: Tcl andPython. Again, Python solver is faster by about factor of 2.(d) with and without the tableau-probing under Python; withtableau-probing the Python solver cleary much faster and thegap should increase as the instance size increases. Since allexperiments have been performed on similar processors, thesecomparisons are fair.Figure 6. This figure extends the range of experiments that are

summarized in the preceding two figures. With a runtime limitof 3600 seconds, we can now report on Nu > 100 uncensoredruns with pal(27, 252) which represent BKV for this instance.Significantly, while the current computational environment isnot adequate to handle larger pal instance over a long run,we were fortunate: a few uncensored runs returned BKV thatsignificantly exceed the ones posted to date.

• a new BKV of pal(31, 300) improves on pal(31, 285)• a new BKV of pal(43, 597) improves on pal(43, 543).

All this with only a self-avoiding walk running under a simple’wandering heuristic’ as a rapid solver prototype in Tcl.

V. CONCLUSIONS AND FUTURE WORK

One of our observations is that currently, we have no referencepoint how our model of platform-independent asymptoticwalkLength complexity of 0.09336⇤1.7727L compares to anyother solver when applied to the class of pal instances inEq. 1. However, it should serve us well when we considerexpanding our heuristic to other than wandering self-avoidingwalk.

We definitely plan to move to a grid-based computer plat-form such as [26] to better harness advantages of massivelyparallel computation. Concurrently, we expect to acceleratethe solver prototype P.lopT by a factor of at least 5000 justby recoding it in C++. With the current solver under C++, theexpected average uncensored runtime to solve pal(55,1084)would then be about 0.00000917 ⇤ 1.934955/(24 ⇤ 3600 ⇤5000) = 124 days.Herein is the motivation to devise a fasteralgorithm!

Finally, we argue that testing the true performance of twoalgorithms can most reliably be performed by designing aseries of asymptotic experiments on well-defined classes ofinstances that increase in size. The example for the palclass as expressed in Eq. 1 is just a start. Only by runningan asymptotic experiment with two solvers can we observewhether the asymptotic lines run in parallel or whether theycross, and at what instance size they cross. Only by observingthe increasing gap between one solver runtime or walkLengthwill we know for sure which is the dominating solver on thegiven instance class.

ACKNOWLEDGMENTS

We gratefully acknowledge Dr. Sarah Heckman for creatingand supporting a stimulating environment where undergradu-ate students could sign-up for research projects and for thesupport from the College of Engineering with student grantsto Yang Ho and Johnny Nguyen to upgrade the source codeto Python 3.x after completion of the project course.

This paper would not have started without the ready accessto LOP resources compiled under http://www.optsicom.es/lolib/ by Rafael Martı, Gerhard Reinelt, and Abraham Duarte.The instances and lists of BKVs in general, and pal instancesin particular, continue to provide a valuable resource for thisresearch.

SOURCE CODE RELEASE

Repositories of the Tcl and Python code released after com-pletion of the project course can be accessed at https://github.com/fbrglez/gitBed

REFERENCES

[1] R. Martı and G. Reinelt, The Linear Ordering Problem: Exact andHeuristic Methods in Combinatorial Optimization. Springer, 2011.

[2] M. Grotschel, M. Junger, and G. Reinelt, “A cutting plane algorithm forthe linear ordering problem,” Operations Research, vol. 32, no. 6, pp.1195–1220, 1984.

[3] J. E. Mitchell and B. Borchers, “Solving real-world linear orderingproblems using a primal-dual interior point cutting plane method,”Annals of Operations Research, vol. 62, no. 1-4, pp. 253 – 276, 1996.

[4] I. Charon and O. Hudry, “A branch-and-bound algorithm to solve thelinear ordering problem for weighted tournaments,” Discrete AppliedMathematics, vol. 154, no. 15, pp. 2097–2116, 2006.

[5] M. Goemans and L. Hall, “The Strongest Facets of the Acyclic SubgraphPolytope are Unknown,” in Fifth Integer Programming and Combinato-rial Optimization Conference, 1996, pp. 415–429.

[6] R. Martı, G. Reinelt, and A. Duarte, “A benchmark library and acomparison of heuristic methods for the linear ordering problem,”Computational Optimization and Applications, vol. 51, no. 3, pp. 1297–1317, 2012.

[7] ——, “Appendix to : A benchmark library and a comparison of heuristicmethods for the linear ordering problem,” Computational Optimizationand Applications, vol. 51, no. 3, pp. 1297–1317, 2012.

[8] ——, “Linear Ordering Problem: Instances, Computational Experiences,Best Methods, LOLIB ,” Optiscom Project, accessible from http://www.optsicom.es/lolib/, 2009.

[9] M. Laguna, R. Marti, and V. Campos, “Intensification and diversificationwith elite tabu search solutions for the linear ordering problem,”Comput. Oper. Res., vol. 26, no. 12, pp. 1217–1230, 1999. [Online].Available: http://dx.doi.org/10.1016/S0305-0548(98)00104-X

[10] T. Schiavinotto and T. Stutzle, “The linear ordering problem: Instances,search space analysis and algorithms,” Journal of Mathematical Mod-elling and Algorithms, vol. 3, no. 4, pp. 367–402, 2004.

[11] R. Martı, M. Laguna, and V. Campos, Scatter Search vs. GeneticAlgorithms. Springer US, 2005.

[12] R. Martı, “Scatter Search – Wellsprings and Challenges,” EuropeanJournal of Operational Research, pp. 351–358, 2006.

[13] C. G. Garcia, D. Perez-Brito, V. Campos, and R. Martı, “Variableneighborhood search for the linear ordering problem,” Computers andOperations Research, vol. 33, no. 12, pp. 3549 – 3565, 2006.

[14] R. Tromble and J. Eisner, “Learning Linear Ordering Problems for BetterTranslation,” in Proc. 2009 Conference on Empirical Methods in NaturalLanguage Processing: Volume 2. Association for ComputationalLinguistics, Stroudsburg, PA, USA, 2009, pp. 1007–1016.

[15] R. Tromble, “Search and Learning for the Linear Ordering Problemwith an Application to Machine Translation,” Ph.D. dissertation, TheJohn Hopkins University, 2009.

[16] W. A. Chaovalitwongse, C. A. S. Oliveira, B. Chiarini, P. M. Pardalos,and M. G. C. Resende, “Revised GRASP with path-relinking forthe linear ordering problem,” Journal of Combinatorial Optimization,vol. 22, no. 4, pp. 572–593, 2011.

[17] P. Hungerlander and F. Rendl, “Semidefinite relaxations of orderingproblems,” Mathematical Programming, vol. 140, no. 1, pp. 77–97,2013.

[18] T. Ye, K. Zhou, Z. Lu, and J.-K. Hao, “A Memetic Algorithm for theLinear Ordering Problem with Cumulative Costs,” https://arxiv.org/abs/1405.4510, 2014.

[19] B. Correal and P. Galinier, On the Complexity of Searchingthe Linear Ordering Problem Neighborhoods. Cham: SpringerInternational Publishing, 2015, pp. 150–159. [Online]. Available:http://dx.doi.org/10.1007/978-3-319-16468-7 13

[20] F. Brglez, “Self-Avoiding Walks across n-Dimensional Dice and Com-binatorial Optimization: An Introduction,” Informacije MIDEM, 44 (1)(2014), pp. 53-68, English Edition http://www.midem-drustvo.si/journal/; also at http://arxiv.org/abs/1309.7508 ; An invited talk at 2013-MIDEM, Sept. 2013, Kranjska Gora, Slovenia , vol. 44, no. 1, pp. 53–68,2014.

[21] B. Boskovic, F. Brglez, and J. Brest, “Low-Autocorrelation BinarySequences: On Improved Merit Factors and Runtime Predictions toAchieve Them,” https://arxiv.org/abs/1406.5301, under review, 2016.

[22] F. Brglez, B. Boskovic, and J. Brest, “On Asymptotic Complexity of theOptimum Golomb Ruler Problem: From Established Stochastic Methodsto Self-Avoiding Walks,” Submitted for review to CEC-2017, 2017.

[23] F. Brglez, “On Markov Chain Structures, Self-Avoiding Walks, andCover Time in Very Large Sparse Graphs,” A draft manuscript availableon request from the author by mid-2017, 2017.

[24] ——, “On Stochastic Combinatorial Optimization and Markov Chains:from Walks with Self-Loops to Self-Avoiding Walks,” A draftmanuscript available on request from the author by mid-2017, 2017.

[25] NCSU, “Virtual Computing Laboratory,” For information, click on https://vcl.ncsu.edu/, 2017.

[26] ——, “High Performance Computing Services,” For information, clickon https://projects.ncsu.edu/hpc/main.php/, 2017.

�1./P.lopT ../benchm/lop/pal/pal19.lop$ ./P.lopT# .. sourced all tcl libraries defined under the sandbox# Mon Feb 06 11:35:54 EST 2017

USAGE: ./P.lopT <instanceDef> [optional_arguments]

EXAMPLES: ./P.lopT ../benchm/lop/pal/pal19.lop -seedInit 1914 ./P.lopT ../benchm/lop/pal/pal19.lop -walkSegmCoef 128 -runtimeLmt 5

DESCRIPTION: P.lopT takes one REQUIRED argument instanceDef (here, a filePath with an extension .lop or NO extension)

and a number of OPTIONAL arguments in any order. Here is a list of pairs 'name defaultValue', with short in-line descriptions:

-runtimeLmt 30 Stop if the solver exceeds these many seconds. -seedInit NA If NA, a random positive integer is created to initialize a random number generator. -coordInit NA If NA, a random permutation coordinate is generated internally -- unless user enters an initial coordinate -walkSegmLmt NA Inactive unless assigned an integer or if walkSegmCoef > 0 -walkSegmCoef NA A coefficient that determines walkSegmLmt as walkSegmCoef*instanceDim. -isSimple FALSE If asserted, simpler-to-code but an inefficient procedure to probe neighborhood coordinates is invoked.

DETAILS:This solver reads an instance of the 'linear ordering problem' in a matrix formatand returns a column/row ordering that minimizes the negative sum of matrixelements above the diagonal. The example below shows an instance of such a matrixwith sum = -8 under its 'natural order', and an instance under an optimal permutation of 3,1,4,2 with a sum of -13. For this matrix, there are two more such optimal permutations: 2,3,1,4 and 4,2,3,1. natural order under permutation 1,2,3,4 3,1,4,2 sum = -8 sum = -13------------ ------------4 4 0 0 0 5 0 4 1 1 1 0 2 0 0 0 5 0 4 1 0 1 1 3 0 2 3 2 1 0 2 1 0 0

# Mon Feb 06 11:35:54 EST 2017 .. completed P.lopT # ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++$

$ ./P.lopT ../benchm/lop/pal/pal19.lop# commandLine = ./P.lopT ../benchm/lop/pal/pal19.lop # dateLine = Mon Feb 06 11:57:59 EST 2017 # timeStamp = 20170206165757 #instanceDef ../xBenchm/lop/pal/pal19.lopsolverId P.lopTcoordInit 1,9,16,8,10,13,7,18,15,17,11,5,3,4,14,6,12,2,19coordBest 17,5,14,2,11,9,18,6,15,3,12,19,7,16,4,13,1,10,8instanceDim 19rankMax 171walkLength 4150walkSegmLmt NAwalkSegmCoef NAcntRestart 0cntProbe 78851cntTrapped 0runtimeLmt 30runtime 1.64isSimple FALSEhostId brglez@belair-Darwin-15.6.0compiler tcl-8.5.9seedFirst 749353919seedInit 749353919valueInit -89valueBest -107valueTarget -107targetReached 1isCensored 0

# Mon Feb 06 11:57:59 EST 2017 .. completed P.lopT # ++++++++++++++++++++++++++++++++++++++++++++++++++++$