cosc 3100 decrease and conquer

COSC 3100Decrease and

ConquerInstructor: Tanvir

Decrease and Conquer• Exploit the relationship between a

solution to a given instance of a problem and a solution to its smaller instance

• Top-down: recursive• Bottom-up: iterative• 3 major types:

– Decrease by a constant– Decrease by a constant factor– Variable size decrease

Decrease and Conquer (contd.)

• Decrease by a constant– Compute an where a ≠ 0 and n is a

nonnegative– an = an-1 × a– Top down: recursive

• f(n) =

– Bottom up: iterative• Multiply 1 by a, n times

f(n-1) × a if n > 0

1 if n = 0

We shall see moreinteresting ones inthis chapter!

Can you improve an computation ?


• Decrease by a constant factor (usually 2)

• an = (𝒂𝒏 /𝟐)𝟐

(𝒂 (𝒏−𝟏 )/𝟐 )𝟐 ∙𝒂𝟏

If n is even and positive

If n is odd

If n = 0

529 =?

Decrease and Conquer (contd.), Decr. By const.

factorALGORITHM Exponentiate(a, n)if n = 0

return 1tmp <- Exponentiate(a, n >> 1)if (n & 1) = 0 // n is even

return tmp*tmpelse

return tmp*tmp*a

What’s the time complexity ?Θ(lgn)

(𝒂𝒏 /𝟐)𝟐

(𝒂 (𝒏−𝟏 )/𝟐 )𝟐 ∙𝒂𝟏

If n even

If n is odd

If n = 0


• Variable-size decreaseWe already have seen one example.

Which one ?gcd(m, n) = gcd(n, m mod n)

gcd( 31415, 14142 ) = gcd( 14142, 3131 )= gcd( 3131, 1618 )= gcd( 1618, 1513 )= gcd( 1513, 105 )= gcd( 105, 43 )= gcd( 43, 19 )= gcd( 19, 5 )= gcd( 5, 4 )= gcd( 4, 1 )= gcd( 1, 0 )

Decr. By 11011Decr. By 1513Decr. By 105Decr. By 1408Decr. By 62Decr. By 24Decr. By 14

Decr. By 1Decr. By 3

Decr. By 1


• Apply decrease by 1 technique to sorting an array A[0..n-1]

• We assume that the smaller instance A[0..n-2] has already been sorted.

• How can we solve for A[0..n-1] now?

Find the appropriate position for A[n-1]among the sorted elements and insert it

there.

Scan the sorted subarray A[0..n-2] from right to left until an element smaller thanor equal to A[n-1] is found, then insert A[n-1]right after that element.

(Straight)INSERTION

SORT

Insertion SortALGORITHM InsertionSort(A[0..n-1])for i <- 1 to n-1 do

v <- A[i]j <- i-1while j ≥ 0 and A[j] > v do

A[j+1] <- A[j]j <- j-1

A[j+1] <- v

89 | 45 68 90 29 34 17

45 89 | 68 90 29 34 17

45 68 89 | 90 29 34 17

45 68 89 90 | 29 34 17

29 45 68 89 90 | 34 17

29 34 45 68 89 90 | 17

17 29 34 45 68 89 90

Input size: nBasic op: A[j] > v

Why not j ≥ 0 ?C(n) depends on input type ?

Insertion Sort (contd.)ALGORITHM InsertionSort(A[0..n-1])for i <- 1 to n-1 do

v <- A[i]j <- i-1while j ≥ 0 and A[j] > v do

A[j+1] <- A[j]j <- j-1

A[j+1] <- v

What is the worst case scenario ?A[j] > v executes highest # of times

When does that happen ?A[j] > A[i] for j = i-1, i-2, …, 0

Worst case input: An array of strictly decreasing values

Cworst(n) = = = є Θ(n2)

What is the best case ?A[i-1] ≤ A[i] for i = 1, 2, …, n-1

Cbest(n) = n-1 є Θ(n)

For almost sorted files,insertion sort’s performance

is excellent!

Cavg(n) ≈ є Θ(n2)

In place? Stable ?

Can you improve it ?

Topological Sorting• Ordering of the vertices of a directed

graph such that for every edge uv, u comes before v in the ordering

• First studied in 1960s in the context of PERT (Project Evaluation and Review Technique) for scheduling in project management.– Jobs are vertices, there is an edge from x to

y if job x must be completed before job y can be started

– Then topological sorting gives an order in which to perform the jobs

Directed Graph or Digraph• Directions for all edges

a b

c

d

e

0 1 1 0 0

1 0 1 0 0

0 0 0 0 0

0 0 1 0 1

0 0 0 0 0

a

bc

de

a b c d e

abcde

b ca c

c e

Not symmetric

One node for one edge

Digraph (contd.)a b

c

d

e

abcde

b ca c

c e

a

b

c

d

e

DFS forest

Tree edgeBack edgeForward edgeCross edgeDirected cycle: a, b, a

If no directed cycles, the digraph is called “directed acyclic graph” or “DAG”

Topological Sorting Scenario

Consider a robot professor Dr. Botstead. He gets dressed in the morning. The professor must put on certain garments before others (e.g., socks before shoes). Other items may beput on in any order (e.g., socks and pants).

undershorts

pants

belt

socks

shoes

shirt

tie

jacket

watch

Let us help him to decideon a correct order usingtopological sorting!

Wont workif directedcycle exists!

Method 1: DFS u

p

b

so

sho

shi

t

j

w

u1,

u -> p -> shop -> sho -> bb -> jshi -> b -> tt -> jso -> show -> nullsho -> nullj -> null p2,

sho3,

b4,

j5,

shi6,

t7,

so8,

w9,

1

2

3

45

6

7

8

9

t6 u5 p4 b3 j2 sho1w9 so8 shi7

Reverse the pop order!

Method 2: Decrease (by 1) and Conquer

u

p

b

so

sho

shi

t

j

w

u, p, shi, b, t, j, so, sho, w

Topological Sorting Scenario 2

• Set of 5 courses: { C1, C2, C3, C4, C5 }

• C1 and C2 have no prerequisites• C3 requires C1 and C2

• C4 requires C3

• C5 requires C3 and C4C1

C4

C5

C2

C3

C1, C2, C3, C4, C5 !

Topological Sorting: Usage

• A large project – e.g., in construction, research, or software development – that involves a multitude of interrelated tasks with known prerequisites– Schedule to minimize the total

completion time• Instruction scheduling in program

compilation, resolving symbol dependencies in linkers,

etc.

Decrease and Conquer: Generating Permutations

• Have to generate all n! permutations of the set of integers from 1 to n– They can be indices of n-element set

{a1, a2, …, an}• Assume that we already have (n-

1)! permutations of integers 1, 2, …, n-1

• Insert n in each of the n possible positions among elements of every permutation of n-1 elements

Generating Permutations (contd.)

• We shall generate all permutations of {1, 2, 3, 4}, 4! = 24 permutations in total

1Start

Insert 2 in 2 possible positions

21, 12

Insert 3 in 3 possible positions ofeach previouspermutation

321, 231, 213,312, 132, 123

Insert 4 in 4 possible positions of each previouspermutation

4321, 3421, 3241, 3214,4231, 2431, 2341, 2314,4213, 2413, 2143, 2134,

4312, 3412, 3142, 3124,4132, 1432, 1342, 1324,4123, 1423, 1243, 1234


a b

dc

2

3

1

5 8 7

a b d c aa c b d aa c d b aa d b c aa d c b a

a b c d a 2+8+1+7 = 182+3+1+5 = 115+8+3+7 = 235+1+3+2 = 117+3+8+5 = 237+1+8+2 = 18

If we can produce a permutationfrom the previous one just by exchangingtwo adjacent elements, cost calculation takesconstant time!

This is possible and calledminimal-change requirement

Every time we process one of theold permutations to get a bunch of newones, we reverse the order of insertion

112,21Right to Left

R to L 123, 132, 312,L to R 321, 231, 213

R2L:1234, 1243, 1423, 4123,L2R:4132, 1432, 1342, 1324,R2L:L2R:R2L:L2R:

3124, 3142, 3412, 4312,4321, 3421, 3241, 3214,

4213, 2413, 2143, 21342314, 2341, 2431, 4231,


• It is possible to get the same ordering of permutations of n elements without explicitly generating permutations for smaller values of n

• Associate a direction with each element k in a permutation

Element k is mobile if its arrow points toa smaller number adjacent to it.

mobile

ALGORITHM JohnsonTrotter(n)//Output: A list of all permutations of {1, …, n}initialize the first permutation with while the last permutation has a mobile element do

find its largest mobile element kswap k with the adjacent element k’s arrow points toreverse direction of all the elements that are larger than

kadd the new permutation to the list


1́ 2́�́� 1́ �́� 2́ 3́ 1́ �́� �⃗� 2́ 1́ 2́ �⃗� 1́ 2́ 1́ 3⃗ should have been the

last one!Lexicographic

orderThis version is by Shimon EvenCan be implemented in Θ(n!) time

Generating Permutations: in

Lexicographic Order ALGORITHM LexicographicPermute(n)initialize the first permutation with 12…nwhile last permutation has two consecutive elements in increasing order do

let i be its largest index such that find the largest index j such that swap ai with aj

reverse the order from ai+1 to an inclusiveadd the new permutation to the list

Narayana Pandita in 14th century India

1 2 3i i+1=j

1 3 2i i+1 j

2 3 1i i+1 j

2 1 3i i+1=j

2 3 1i i+1=j3 2 1

i i+1=j3 1 2

i i+1=j3 2 1

Generating Subsets• Generate all 2n subsets of the set A = {a1, a2, …, an}Can divide the subsets into two groups:

Those that do not contain an Those that do contain an

It’s the subsets of {a1, a2, …, an-1}

Take each subset of {a1, a2, …, an-1}and add an to it

Generating Subsets (contd.)

• Let us generate 23 subsets of {a1, a2, a3}

Ø

Ø {a1}

Ø {a1} {a2} {a1, a2}

Ø {a1} {a2} {a1, a2} {a3} {a1, a3} {a2, a3} {a1, a2, a3}

n = 0

n = 1

n = 2

n = 3


• There is a one-to-one correspondence between all 2n subsets of an n element set A = {a1, a2, …, an} and all 2n bit strings b1b2…bn of length n.

Bit strings

Subsets

000 001 010 011 100 101 110 111

Ø {a3} {a2} {a2, a3} {a1} {a1, a3} {a1, a2} {a1, a2, a3}

How to generate subsets in “squashed order” from bitstrings?

Ø {a1} {a2} {a1, a2} {a3} {a1, a3} {a2, a3} {a1, a2, a3}


Is there a minimal-change algorithm for generatingbit strings so that every consecutive two differ by

only a single bit ?In terms of subsets, every consecutive two differ by either an addition or deletion, but not both of

a single element.Yes, there is!

000 001 011 010 110 111 101 100For n = 3,

It is called “Binary reflected Gray code”


ALGORITHM BRGC(n)//Generates recursively the binary reflected Gray code of order n//Output: A list of all bit strings of length n composing the Gray codeif n = 1

make list L containing bit strings 0 and 1 in this orderelse

generate list L1 of bit strings of size n-1 by calling BRGC(n-1)

copy list L1 to list L2 in reverse orderadd 0 in front of each bit string in list L1add 1 in front of each bit string in list L2append L2 to L1 to get list L

return LL1 : 0 1L2 : 1 0 L1 : 00 01

L2 : 11 10

L : 00 01 11 10

Binary Reflected Gray Codes (contd.)

0 110 11

00 01000 001

101100

011111110

010

Decrease-by-a-Constant-Factor Algorithms

• Binary Search– Highly efficient way to search for a

key K in a sorted array A[0..n-1]

Compare K with A’s middle element A[m]If they match, stop.

Else if K < A[m] do the same for the first half of AElse if K > A[m] do the same for the second half of A

Binary Search (contd.)

A[0] … A[m-1] A[m] A[m+1] … A[n-1]

K (key)

Search here if K < A[m]

Search here if K > A[m]

Let us apply binary search for K = 70 on the following array:

3, 14, 27, 31, 39, 42, 55, 70, 74, 81, 85, 93, 980 121 2 3 4 5 6 7 8 9 10 11

Binary Search (contd.)ALGORITHM BinarySearch(A[0..n-1], K)//Input: A[0..n-1] sorted in ascending order and a search key K//Output: An index of A’s element equal to K or -1 if no such elementl <- 0r <- n-1while l ≤ r do

m <- if K = A[m]

return melse if K < A[m]

r <- m-1else

l <- m+1return -1

Input size: nBasic operation: “comparison”Does C(n) depend on input type?

YES!Let us look at the worst-case…

Worst-case input: K is absent or someK is at some special position…

Cworst(n) = Cworst() + 1Cworst(1) = 1

To simplify, assume n = 2k

Then Cworst(n) є Θ(lgn)

For any integer n > 0, Cworst(n) = =

Variable-Size-Decrease• Problem size decreases at each

iteration in variable amount• Euclid’s algorithm for computing

the greatest common divisor of two integers is one example

Computing Median and the Selection Problem• Find the k-th smallest element in a

list of n numbers• For k = 1 or k = n, we could just

scan the list• More interesting case is for k =

– Find an element that is not larger than one half of the list’s elements and not smaller than the other half; this middle element is called the “median”

– Important problem in statistics

Median Selection ProblemAny idea ?

Sort the list and select the k-th element

Time efficiency is ?O(n lgn)It is an overkill, the problem needs only the

k-th smallest, does not ask to order the entire list!

We shall take advantage of the idea of “partitioning”a given list around some value p of, say the list’s first

element. This element is called the “pivot”.p pall are < p all are ≥ p

Lomuto partitioning & Hoare’s partitioning

Lomuto Partitioning61 93 56 90 11p

s

i

A[i] not less than p

61 93 56 90 11p

s

i

A[i] less than p

i

s

61 56 93 90 11p i

s

61 56 93 90 11p i

A[i] not less than p

s

i

i

61 56 93 90 11p

A[i] less than p

s

i

s

61 56 11 90 93p i

s

11 56 61 90 93

Lomuto Partitioning (contd.)

ps i

< p ≥ p ?

A[i] < p?

p

s< p ≥ p

swap

p< p ≥ ps

Lomuto Partitioning (contd.)

ALGORITHM LomutoPartition(A[l..r])//Partition subarray by Lomuto’s algo using first element as pivot//Input: A subarray A[l..r] of array A[0..n-1], defined by its //left and right indices l and r (l ≤ r)//Output: Partition of A[l..r] and the new position of the pivot

p <- A[l]s <- lfor i <- l+1 to r do

if A[i] < ps <- s+1swap(A[s], A[i])

swap(A[l], A[s])return s

Time complexity is Θ(r-l)

Length of subarray A[l..r]

Median Selection Problem (contd.)

• We want to use the Lomuto partitioning to efficiently find out the k-th smallest element of A[0..n-1]

• Let s be the partition’s split position• If s = k-1, pivot p is the k-th smallest• If s > k-1, the k-th smallest (of entire array) is

the k-th smallest of the left part of the partitioned array

• If s < k-1, the k-th smallest (of entire array) is the [(k-1)-(s+1)+1]-th smallest of the right part of the partitioned array

Median Selection Problem (contd.)

ALGORITHM QuickSelect(A[l..r], k)//Input: Subarray A[l..r] of array A[0..n-1] of orderable elements and integer k (1 ≤ k ≤ r-l+1)//Output: The value of the k-th smallest element in A[l..r]s <- LomutoPartiotion(A[l..r])if s = k-1

return A[s]else if s > l+k-1

QuickSelect(A[l..s-1], k)else

QuickSelect(A[s+1..r], k-1-s)

Median Selection: QuickSelect

4 1 10 8 7 12 9 2 15

4 1 10 8 7 12 9 2 15

4 1 10 8 7 12 9 2 15

4 1 2 8 7 12 9 10 15

4 1 2 8 7 12 9 10 15

2 1 4 8 7 12 9 10 15

s i

s i

s i

s i

s i

s = 2 < k-1 = 4, so we proceed with the right part…

k = = 5

s

Median Selection: QuickSelect (contd.)

8 7 12 9 10 15

2 1 4 8 7 12 9 10 15

k = = 5

8 7 12 9 10 15

8 7 12 9 10 15

7 8 12 9 10 15

s i

s i

s i

Now s (=4) = k-1 (=5-1=4), we have found the median!

s

Median Selection (contd.)ALGORITHM QuickSelect(A[l..r], k)//Input: Subarray A[l..r] of array A[0..n-1] of orderable elements and integer //k (1 ≤ k ≤ r-l+1)//Output: The value of the k-th smallest element in A[l..r]s <- LomutoPartiotion(A[l..r])if s = k-1

return A[s]else if s > l+k-1

QuickSelect(A[l..s-1], k)else

QuickSelect(A[s+1..r], k-1-s)

What is the time efficiency ? Does it depend on input-type?

Partitioning A[0..n-1] takes n-1 comparisonsIf now s = k-1, this is the best-case, Cbest(n) = n є Θ(n)

What’s the worst-case ?

Say k = n and we have a strictlyincreasing array!

Then, Cworst(n) = (n-1)+(n-2)+……+1 = =

Cworst(n) is worse than sorting based solution!

Fortunately, careful mathematical analysishas shown that the average-case is linear

Why is QuickSelectin variable-size-decrease ?

Thank you !

cosc 3100 decrease and conquer

Documents