Chapter 5: Decrease and Conquer

The Design and Analysis of Algorithms

2

Chapter 5. Chapter 5. Decrease and Conquer Algorithms Basic Idea Decrease by a Constant (usually one)

Insertion sort, graph search Permutations, subsets

Decrease by a Constant Factor Fake-coin problem, Multiplication a la Russe

Variable Size Decrease Conclusion

3

Basic Idea Reduce problem instance to smaller

instance of the same problem and extend solution

Solve smaller instance

Extend solution of smaller instance to obtain solution to original problem

4

Basic Idea

5

Examples of Decrease-and-Conquer Algorithms

Decrease by one: Insertion sort Graph search algorithms:

DFS BFS

Topological sorting Algorithms for generating permutations,

subsets

6

Examples of Decrease-and-Conquer Algorithms

Decrease by one: Insertion sort Graph search algorithms:

DFSBFS

Topological sorting Algorithms for generating permutations, subsets

7

Examples of Decrease-and-Conquer Algorithms

Decrease by a constant factorBinary search Fake-coin problemsMultiplication à la russeJosephus problem

Variable-size decreaseEuclid’s algorithmSelection by partition

8

Decrease by One: Insertion Sort

Insertion SortAlgorithm to sort n elements:

Sort n-1 elements of the array Insert the n-th element

Complexity: (n2) in the worst and the average case, and (n) on almost sorted arrays.

9

Decrease by One: Graph Search

Complexity: Θ(V2) if represented by an adjacency table, Θ(|V| + |E|) if represented by adjacency lists

Depth-first search algorithm:Depth-first search algorithm:dfs(v)

process(v) mark v as visited for all vertices i adjacent to v not visited

dfs(i)

10

Graph Search: DFS

11

Graph Search: BFS

Breadth-first search algorithmBreadth-first search algorithm:procedure bfs(v) q := make_queue() enqueue(q,v) mark v as visited while q is not empty v = dequeue(q) process v for all unvisited vertices v' adjacent to v mark v' as visited enqueue(q,v')

12

Graph Search: BFS

13

Graph Search

Complexity of DFS and BFS: Θ(V2) if represented by an adjacency table, Θ(|V| + |E|) if represented by adjacency lists.

Various applications in AI problems and graph problems (e.g. finding cycles, articulation points)

14

Permutations of Size n

find all permutations of size n-1 of elements a1, a2, .., a n-1

construct permutations of n elements as: append an to each permutation

of size n-1 for each permutation of size n-1

for k from 1 to n-1

insert an in front of ak

Generating permutations of size n

15

Johnson-Trotter Algorithm

The straight-forward implementation is very inefficient since it obtains all lower-level permutations (permutations of size less than n).

Trotter (1962) and Johnson (1963): an algorithm to obtain all permutations of size n without going through shorter permutations.

Directed integers.

Mobile integer: greater than the integer it points to

16

Johnson-Trotter Algorithm

Initialize the first permutation with <1 <2 ... <n

while the last permutation has a mobile integer do

find the largest mobile integer k

swap k and the adjacent integer it is looking at

reverse the direction of all integers larger than k

17

Johnson-Trotter AlgorithmExample:

<1 <2 <3 <4 largest mobile element is 4<1 <2 <4 <3 swap 3 and 4; largest mobile element is 4<1 <4 <2 <3 swap 2 and 4; largest mobile element is 4<4 <1 <2 <3 swap 1 and 4; largest mobile element is 34> <1 <3 <2 swap 2 and 3; change direction of all greater than 3; largest mobile element is 4<1 4> <3 <2 swap 1 and 4; largest mobile element is 4<1 <3 4> <2 swap 3 and 4; largest mobile element is 4<1 <3 <2 4> swap 2 and 4; largest mobile element is 3. . . . . . . . <2 <1 3> 4>.

“minimal-change” algorithm; runs in (N!) time

18

Subsets and Gray Codes

Let Sn-1 be the set of all subsets of n-1 elements, Sn-1 = {A1, A2, … Am}, m = 2n-1

Sn = {A1, A2, … Am, A1 an , A2 an , … Am an }

No need to generate all power sets of smaller sets.

Frank Gray (1953): a minimal-change algorithm for generating all binary sequences of length n -

“binary reflected Gray code”.

19

Subsets and Gray Codes

base reflect prepend reflect prepend0 0 00 00 0001 1 01 01 001

1 11 11 011 0 10 10 010

10 110 11 111 01 101 00 100

The complexity is (2n)

20

Decrease by a Constant Factor

The Fake-coin problem Multiplication a la Russe

Usually logarithmic in complexity

21

The Fake Coin Problem

You have 27 coins among which one is fake and it is lighter than the others.

You have also a balance scale and you can compare the weight of any two sets of coins.

How can you find the fake coin with three measurements only?

How many measurements if the number of coins is N?

22

Multiplications a la Russe

We can multiply two positive integers using only addition and division by 2.

The algorithm is based on the observation that N*M = (N/2) * (M * 2) if N is even, andN*M = ((N-1)/2) * (M*2) + M if N is odd

The base case is N = 1: 1*M = M

23

Multiplications a la Russe

24

Variable Size Decrease

The reduction pattern varies from one iteration of the algorithm to another.

Examples are Euclid’s algorithm for computing the greatest common divisor , Search and Insertion in binary search trees, and others.

25

Conclusion

Very efficient solutions. The algorithms exploit a relation

between the instance of the problem and a smaller instance of the same problem

Recursive in nature Implementation: recursion or iteration