integer and combinatorial optimization: algorithm...

Integer and Combinatorial Optimization:Algorithm Complexity

John E. Mitchell

Department of Mathematical SciencesRPI, Troy, NY 12180 USA

January 2019

Mitchell Algorithm Complexity 1 / 23

How run-time grows as problem size grows

Outline

1 How run-time grows as problem size grows

2 Polynomial time algorithms and the class PExamples of problems in P



Big-O notation

DefinitionLet f and g be functions from the set of integers or the set of reals to

the set of real numbers. The function f (k) is O(g(k)) whenever there

exist positive constants C and k 0 such that

|f (k)| C|g(k)| 8 k � k0.


106×4 i s 01×4) 106×3 e 10-6×4 i s

01×4)


Size of an instance

The size of an instance is the number of binary characters required tostore the problem.For example, given a positive integer x , we set p = blog2(x)c. Then

2p x < 2p+1 and x =pX

i=0

�i2i for some binary �i .

For example, 42 = 0⇥ 20 + 1⇥ 21 + 0⇥ 22 + 1⇥ 23 + 0⇥ 24 + 1⇥ 25.So we need O(log(x)) bits to store x .

We don’t need to specify the base for the logarithm here, because

if x > 0 and y = loga(x), z = logb(x) then y = z loga(b).


4 2 = 2 + 8 + 3 2


Data representation

The major classification of algorithm efficiency is insensitive to thechoice of data representation, for the most part.

There are two restrictions:1 The alphabet used to represent the data must contain at least two

symbols. So the space to store a positive integer x is O(log(x)),instead of x with a unary alphabet.

2 The “data” can’t contain information that requires a lot ofcalculation. For example, can’t store the length of all tours as partof the representation of a traveling salesman problem.


4 1 - # # # # # # # # 1 1


Data representation

The major classification of algorithm efficiency is insensitive to thechoice of data representation, for the most part.

There are two restrictions:1 The alphabet used to represent the data must contain at least two

symbols. So the space to store a positive integer x is O(log(x)),instead of x with a unary alphabet.

2 The “data” can’t contain information that requires a lot ofcalculation. For example, can’t store the length of all tours as partof the representation of a traveling salesman problem.



Computation time

The time required for an elementary operation (addition, subtraction,multiplication, division, comparison) is counted as unit time.

(Need to be a little careful if multiply very large numbers together.)



Instances

Perfect matching, linear programming, and node packing are allexamples of optimization problems.

An optimization problem X consists of an infinite number of instancesd1, d2, . . . where the data for instance di is given by a binary string oflength l(di).

For example, an instance of a perfect matching problem is given bylists of the vertices and edges of the graph G = (V ,E).



Worst case running time

Let A be an algorithm the solves every instance of X in finite time.

Let gA(di) be the running time of A on instance di .

Measure performance of A on X using the worst-case running time, foreach problem size k :

fA(k) := max{gA(di) : l(di) = k}.

Advantages of using worst-case performance measure:Gives a guarantee of performance of the algorithmIndependent of the probability distribution of the instances.Appears to be the easiest to analyze.



Klee-MintyFor example, the simplex algorithm solves any linear optimizationproblem in finite time (provided we use an anti-cycling rule).

In the worst case, it may require O(2n) iterations to solve a problem inn nonnegative variables with n inequality constraints (namely, theKlee-Minty cube).

x1

x2

x3

(0, 0, 0)(0, 100, 0)

(0, 0, 10000)

(1, 0, 0)

(1, 80, 0)

(0, 100, 8000)

(1, 80, 8200)(1, 0, 9800)


Polynomial time algorithms and the class P

Outline





Feasibility problems

For example, an instance of the perfect matching feasibility problem

consists of a graph G = (V ,E) and asks the question “Does there exista perfect matching on this graph”? The answer is either Yes or No.

DefinitionA feasibility problem X consists of a set of instances D and a set of

feasible instances F ✓ D. Given an instance d 2 D, determine

whether d 2 F. The answer is either Yes or No.


÷⇒ .


Binary searchMany instances of optimization problems can be solved using binarysearch on a sequence of instances of feasibility problems.

For example, can solve the 0� 1 linear optimization problem

minx

{cT

x : Ax � b, x 2 Bn}

by repeatedly solving the 0� 1 linear feasibility problem

Does there exist x 2 Bn with Ax � b, cT

x z?

At each iteration, we have upper and lower bounds on the optimalvalue z⇤. The initial bounds on z⇤ can be taken to be

zUB =

X

j:cj>0

cj , zLB =

X

j:cj<0

cj

since x is binary. We solve the feasibility problem withz = 1

2(zLB + zUB). We assume all the data is integral, so the optimal

value is integral, so we can maintain integral values for zLB, zUB, and z.Mitchell Algorithm Complexity 12 / 23

m m Tx , - T x , t 7x , -6×4 = : z

s t . A x > b

x binary.

Then 2 3 - 9 (assuming feasible)

a n d z e 1 2 (achievedi f onlyfealibl. solutioni s

x = (491,0).)

- Doe,Ixe lB"Initial guns: 2 = 1 f L'¥9))). w i n Axes,I f N O : love bound→ 2 . exez?i f Y E S : upperbound→ I .


Binary search algorithm

1 Initialize zUB, zLB.2 If zLB = zUB, STOP and return the optimal value

z⇤ = zLB = zUB.3 Let z =

⌅12(z

LB + zUB)⇧.

4 Solve the 0� 1 linear feasibility problem

Does there exist x 2 Bn with Ax � b, cT

x z?

5 If answer is YES: update zUB z, return to Step 2.6 If answer is NO: update zLB z + 1, return to Step 2.

Let z0 be the initial value of zUB � zLB. The number of iterations of thealgorithm is O(log2(z

0)), since the difference between the bounds ishalved at each iteration.


I f c a n solve fearibility problem i npolynomial t ime f(di) forinsta.. d i :

Then time t o solve ophinication problem

i s f (di) log(initialgap)

i n i npolynomial polynomial

i n storage i n storage

i npolynomial i n storage.


The Class P

DefinitionAlgorithm A is a polynomial time algorithm for the feasibility problem

X if fA(k) is O(kp) for some fixed p.

P denotes the class of feasibility problems that can be solved in

polynomial time.

Problem X 2 P if and only if there is a polynomial time algorithm for X.


µk : storage.

Polynomial time algorithms and the class P Examples of problems in P

Outline





Minimum spanning tree with upper boundGiven a graph G = (V ,E), edge weights we for e 2 E , and an integerz, does there exist a spanning tree with total weight no greater than z?

Use a greedy algorithm.

1

2

3

4

5

6

7

8

2

351 5

4

67

4

6

7

64

8

3 5

Weight 25?

Solution:

1

2

3

4

5

6

7

8

2

31

4

6

4

3

YES (weight 23)



Shortest path with upper boundGiven a graph G = (V ,E), edge weights we for e 2 E , source node s

and sink node t , and an integer z, does there exist a path from s to t

with total length no greater than z?Use Dijkstra’s algorithm if edge weights are nonnegative.

s

2

3

4

tcs2 = 2, xs2 = 1

cs3 = 6

c24 = 1, x24 = 1

c32 =5

c3t = 3, x3t = 1

c43 = 2, x43 = 1

c4t= 6

Path 6? NO



Perfect matching feasibility problemGiven a graph G = (V ,E), does there exist a perfect matching?Use Edmonds blossom (alternating path) algorithm.

1

2

3

4

5

6

No alternating path between unmatched vertices 2 and 6



Linear programming with bound

Given m ⇥ n matrix A, vectors b 2 IRm and c 2 IRn, and a scalar z,does there exist a nonnegative x 2 IRn satisfying Ax = b and cT x z?

Use the ellipsoid algorithm or some interior point algorithms.



Exponential functions

DefinitionThe function f (k) is exponential if there exist positive constants

c1, c2 > 0, constants d1, d2 > 1, and a positive constant k 0 such that

c1dk

1 f (k) c2dk

2 8 k � k0.

For example, let b be the largest integer for an instance of anoptimization problem X , so this requires storage O(log2(b)).

Assume the total storage is also O(log2(b)).

If algorithm A runs in time O(b) then it requires exponential time forthis instance, since

b = 2log2(b).



The knapsack problem

The knapsack problem max{cT x : aT x b, x binary} can be solvedin O(b) iterations using dynamic programming.

However, the storage requirement is only O(log(b)), assuming b is thelargest integer in the problem definition.

So the runtime is exponential in the storage requirement.


b - 24¥....,..f:::::"?"")

runtime


Max Flow

Consider a max flow problem on the following graph. We seek tomaximize flow from node 1 to node 4. The arc capacities are indicated.

1

2

3

4M

M

1M

M



Solve with augmenting path method

1

2

3

4t t

t

1

2

3

4

(M, 1)

(M, 0)

(1, 1)

(M, 0)

(M, 1)

take t = 1

(ue, xe)

Next iteration:

1

2

3

4t

�t

t

1

2

3

4

(M, 1)

(M, 1)

(1, 0)

(M, 1)

(M, 1)

take t = 1

(ue, xe)

Need 2M iterations to get to optimal solution, which is exponential inthe storage requirement. To get polynomial algorithm, need to selectaugmenting path with fewest edges.


integer and combinatorial optimization: algorithm...

Documents