cse 202: design and analysis of algorithms

CSE 202: Design and Analysis of Algorithms

Lecture 5

Instructor: Kamalika Chaudhuri

Announcements

• Kamalika’s Office hours today moved to tomorrow,12:15-1:15pm

• Homework 1 due now

• Midterm on Feb 14

Algorithm Design Paradigms

• Exhaustive Search

• Greedy Algorithms: Build a solution incrementally piece by piece

• Divide and Conquer: Divide into parts, solve each part, combine results

• Dynamic Programming: Divide into subtasks, perform subtask by size. Combine smaller subtasks to larger ones

• Hill-climbing: Start with a solution, improve it

Divide and Conquer

• Integer Multiplication

• Closest pair of points on a Plane

• Strassen’s algorithm for matrix multiplication

Integer Multiplication: The Grade School Way

[22] 1 0 1 1 0[5] 1 0 1 --------------------------------------------------- 1 0 1 1 0 0 0 0 0 0 1 0 1 1 0 ---------------------------------------------------[110] 1 1 0 1 1 1 0

How to multiply two n-bit numbers?

1.Create array of n intermediate sums2. Add up the sums

Time per addition = O(n)Total time = O(n2).

Can we do better?

A Simple Divide and Conquer

x

xL xR

=

Problem: How to multiply two n-bit numbers x and y?

y

yL yR

=

y = yL 2n/2 + yR x = xL 2n/2 + xR


xy = (xL 2n/2 + xR)(yL 2n/2 + yR)= xLyL 2n + (xR yL + xLyR) 2n/2 + xRyR

x

xL xR

=


y

yL yR

=

y = yL 2n/2 + yR x = xL 2n/2 + xR


Operations:

1. 4 multiplications of n/2 bit #s2. Shifting by n bits3. 3 additions


x

xL xR

=


y

yL yR

=

y = yL 2n/2 + yR x = xL 2n/2 + xR


Operations:


Recurrence:T(n) = 4T(n/2) + O(n)


x

xL xR

=


y

yL yR

=

y = yL 2n/2 + yR x = xL 2n/2 + xR



Operations:


Recurrence:T(n) = 4T(n/2) + O(n)T(n) = O(n2)

What is the base case?

x

xL xR

=


y

yL yR

=

y = yL 2n/2 + yR x = xL 2n/2 + xR

A Better Divide and Conquer


Need: xLyL, xRyR, (xRyL + xLyR)

Computed by 3 multiplications as:(xRyL + xLyR) = (xL + xR)(yL + yR) - xLyL - xRyR

x

xL xR

=


y

yL yR

=

y = yL 2n/2 + yR x = xL 2n/2 + xR





Operations:


x

xL xR

=


y

yL yR

=

y = yL 2n/2 + yR x = xL 2n/2 + xR





Operations:


Recurrence:T(n) = 3T(n/2) + O(n)

x

xL xR

=


y

yL yR

=

y = yL 2n/2 + yR x = xL 2n/2 + xR



Operations:


Recurrence:T(n) = 3T(n/2) + O(n)T(n) = O(n1.59)

x

xL xR

=


y

yL yR

=

y = yL 2n/2 + yR x = xL 2n/2 + xR




Operations:


Recurrence:T(n) = 3T(n/2) + O(n)T(n) = O(n1.59)

Best: O(n log n 2O(log * n)) [Furer07])

x

xL xR

=


y

yL yR

=

y = yL 2n/2 + yR x = xL 2n/2 + xR




Divide and Conquer




Closest Pair of Points on a Plane

Given a set P of n points on the plane, find the two points p and q in P s.t d(p, q) is minimum

p q

Closest Pair of Points on a Plane


Naive solution = O(n2)

p q

What about one dimension?

Given a set P of n points on the line, find the two points p and q in P s.t d(p, q) is minimum

p q



p q

Property: The closest points are adjacent in sorted order



Algorithm 11. Sort the points 2. Find a point pi in the sorted set s.t d(pi, pi+1) is minimum

p q




Algorithm 11. Sort the points 2. Find a point pi in the sorted set s.t d(pi, pi+1) is minimum

p q


Running Time = O(n log n)

Does this work in 2D?

p

q

a

b (a, b) : closest in x-coordinate

(a, q) : closest in y-coordinate

(p, q) : closest

Sorting the points by x or y coordinate, and looking at adjacent pairs does not work!

A Divide and Conquer Approach

Find-Closest-Pair(Q):

1. Find the median x coordinate lx2. L = Points to the left of lx3. R = Rest of the points4. Find-closest-pair(L)5. Find-closest-pair(R)6. Combine the results

How to combine?

lx

RL

Problem Case: p in L, q in R (or vice versa)



1. Find the median x coordinate lx2. L = Points to the left of lx3. R = Rest of the points4. Find-closest-pair(L)5. Find-closest-pair(R)6. Combine the results

How to combine?

1. Naive combination

2. Can we do better?

Time = O(n2)

lx

RL

A Property of Closest Points

Let: t1 = min x, y in L d(x, y)t2 = min x, y in R d(x, y)t = min (t1, t2)

If d(a, b) < t, where a is in L, b in R, then, a and b lie within distance t of lx

RL

lx




lx

RL>ta




lx

RL>t

>t

a

b




lx

RL>t

>t

a

b

t tWe can safely rule out all points in the grey regions!

Another Property

Let: t1 = min x, y in L d(x, y)t2 = min x, y in R d(x, y)t = min (t1, t2)Sy = all points within distance t of lx in sorted order of y coords

If d(a, b) < t, a in L, b in R, then a and b occur within 15 positions of each other in Sy

lx

t/2

t/2

t

t1L

lx

t2 R

If d(a,b) < t, a in L, b in R, a, b are > 15 positions apart

Fact: Each box in figure can contain at most one point.

then, there are >= 3 rows between a and b

Boxes

Thus, d(a, b) >= 3t/2. Contradiction!

a

b



1. Find the median x coordinate lx2. L = Points to the left of lx3. R = Rest of the points4. (a1, b1, t1) = Find-closest-pair(L)5. (a2, b2, t2) = Find-closest-pair(R)6. t = min(t1, t2)7. Combine the results

How to combine?

lx

RLS = points in Q

within distance t of Ix

Sy = sort S by y coordsFor p in Sy

Find distance from pto next 15 pointsLet (a, b) be the pair withmin such distance

If d(a, b) < t:Return (a, b, d(a, b))

Else if t1 < t2:Return (a1, b1, t1)

Else:Return (a2, b2, t2)

Property: If a in L, b in R, and if d(a, b) < t, then a and b occur within 15 positions of each other in Sy


Find-Closest-Pair(Qx, Qy):

1. Find the median x coordinate lx2. L = Points to the left of lx3. R = Rest of the points4. (a1, b1, t1) = Find-closest-pair(Lx, Ly)5. (a2, b2, t2) = Find-closest-pair(Rx, Ry)6. t = min(t1, t2)7. Combine the results

How to combine?

S = points in Q within distance t of Ix







Implementing Divide + Combine:1. Maintain sorted lists of the input points:

Px = sorted by x, Py = sorted by y2. Computing Lx,Ly,Rx, Ry from Qx, Qy: O(|Q|) time2. Computing Sy from Qy : O(|S|) time3. Combination procedure: O(|Sy|) time


Find-Closest-Pair(Qx, Qy):

1. Find the median x coordinate lx2. L = Points to the left of lx3. R = Rest of the points4. (a1, b1, t1) = Find-closest-pair(Lx, Ly)5. (a2, b2, t2) = Find-closest-pair(Rx, Ry)6. t = min(t1, t2)7. Combine the results

How to combine?

S = points in Q within distance t of Ix







Implementing Divide + Combine:

T(n) = 2 T(n/2) + cnT(n) = (n log n)Θ

Summary: Closest Pair of Points


Find-Closest-Pair(Qx,Qy):

1. Find the median x coordinate lx2. L = Points to the left of lx3. R = Rest of the points4. (a1, b1, t1) = Find-closest-pair(Lx, Ly)5. (a2, b2, t2) = Find-closest-pair(Rx, Ry)6. t = min(t1, t2)7. Let Sy = points within distance t of Ix sorted by y 8. For each p in Sy

Find distance from p to next 15 points in Sy

Let (a, b) be the closest such pair9. Report the closest pair out of:

(a, b), (a1, b1), (a2, b2)

Recurrence:

T(n) = 2T(n/2) + cnT(n) = O(n log n)

What is a base case?

Divide and Conquer




Matrix Multiplication

Problem: Given two n x n matrices X and Y, compute Z = XY

Zij = Xik YkjΣ

X =i

j

(i,j)

X Y Z

Naive Method: O(n3) time



X =A B

C DY =

E F

G H

Z =AE + BG AF + BH

CE + DG CF + DH

Algorithm 1: 1. Compute AE, BG, CE, DG, AF, BH, CF, DH2. Compute AE + BG, CE + DG, AF + BH, CF + DH

Operations: 8 n/2 x n/2 matrix multiplications, 4 additions

Recurrence: T(n) = 8 T(n/2) + O(n2) T(n) = O(n3)

A..H: n/2 x n/2

Strassen’s Algorithm


X =A B

C DY =

E F

G H

Z =P5 + P4 - P2 + P6 P1 + P2

P3 + P4 P1 + P5 - P3 - P7

P1 = A(F - H)P2 = (A + B)H

P3 = (C + D)EP4 = D(G - E)

P5 = (A + D)(E + H)P6 = (B - D)(G + H)

P7 = (A - C)(E + F)

Operations: 7 n/2 x n/2 matrix multiplications, O(1) additions

Recurrence: T(n) = 7 T(n/2) + O(n2) T(n) = O(n2.81)

A..H: n/2 x n/2

Strassen’s Algorithm


X =A B

C DY =

E F

G H

Z =P5 + P4 - P2 + P6 P1 + P2

P3 + P4 P1 + P5 - P3 - P7

P1 = A(F - H)P2 = (A + B)H

P3 = (C + D)EP4 = D(G - E)

P5 = (A + D)(E + H)P6 = (B - D)(G + H)

P7 = (A - C)(E + F)

Operations: 7 n/2 x n/2 matrix multiplications, O(1) additionsRecurrence: T(n) = 7 T(n/2) + O(n2) T(n) = O(n2.81)

A..H: n/2 x n/2

Algorithm Design Paradigms

• Exhaustive Search

• Greedy Algorithms: Build a solution incrementally piece by piece

• Divide and Conquer: Divide into parts, solve each part, combine results

• Dynamic Programming: Divide into subtasks, perform subtask by size. Combine smaller subtasks to larger ones

• Hill-climbing: Start with a solution, improve it

function Fib1(n)if n = 1 return 1if n = 2 return 1return Fib1(n-1) + Fib1(n-2)

Dynamic Programming (DP): A Simple Example

Problem: Compute the n-th Fibonacci number

Recursive Solution Running Time:

T(n) = T(n-1) + T(n-2) + 1




Recursive Solution Running Time:

T(n) = T(n-1) + T(n-2) + 1

Running time: O(cn)

T(n) = O(cn)



function Fib2(n)Create an array fib[1..n]fib[1] = 1fib[2] = 1for i = 3 to n: fib[i] = fib[i-1] + fib[i-2]return fib[n]


Recursive Solution

Dynamic Programming Solution

Running Time:

T(n) = T(n-1) + T(n-2) + 1

Running time: O(cn)

T(n) = O(cn)





Recursive Solution


Running Time:

T(n) = T(n-1) + T(n-2) + 1

Running time: O(cn)

T(n) = O(n)

Running Time:

Running time: O(n)

T(n) = O(cn)


Why does DP do better?



Recursive Solution


Running time: O(cn)

Running time: O(n)

F(5)

F(4)

F(2)F(3)

F(2) F(1)

F(3)

F(1)F(2)

Recursion Tree

Dynamic Programming

Main Steps:

1. Divide the problem into subtasks

2. Define the subtasks recursively (express larger subtasks in terms of smaller ones)

3. Find the right order for solving the subtasks (but do not solve them recursively!)

Dynamic Programming

Main Steps:

1. Divide the problem into subtasks: compute fib[i]

2. Define the subtasks recursively (express larger subtasks in terms of smaller ones)

3. Find the right order for solving the subtasks (i = 1,..,n)



Running time: O(n)

Dynamic Programming

• String Reconstruction

• ...

cse 202: design and analysis of algorithms

Documents