ADVANCED ALGORITHMS LECTURE 6: DP, GREEDY ALGORITHMS

�1

LECTURE 6

ANNOUNCEMENTS

▸ Homework 1 due on Monday Sep 10, 11:59 PM

▸ Video issues last class

▸ Comment on notes — JeffE’s material

�2

LECTURE 6

LAST CLASS

�3

▸ Dynamic programming

▸ Recursive algorithms — observe that same sub-problems solved many times

▸ Key idea: avoid re-solving by storing answers!

▸ How many “different problem instances” arise?

▸ Examples: subset sum and coin change problems

tn s

I emojis

LECTURE 6

OVERLAP IN RECURSIVE CALLS

�4

A = {1, 2, 3, 5, 7, 11, 14, 15} , S = 20

Use first element Don’t use first element

A’ = {2, 3, 5, 7, 11, 14, 15} , S’ = 19

A’’ = {5, 7, 11, 14, 15} , S’ = 17 A’’ = {5, 7, 11, 14, 15} , S’ = 17

Key observation: every sub-problem has a suffix of A as the array, and some target sum in the range [0,…,S]. Thus only n(S+1) sub-problems!

if Afi are notintegers

If all betsafe

LECTURE 6

TODAY’S PLAN

�5

▸ Conclude DP — a couple more examples from a high level

▸ Greedy algorithms

LECTURE 6

RECAP: DYNAMIC PROGRAMMING

�6

▸ Design recursive algorithm

▸ Observe that number of distinct sub-problems is “small”

▸ Store answers! (memorize)

▸ Top-down vs. Bottom-up

▸ Challenge in using DP: formalizing sub-problems

more in Hwa

LECTURE 6

POPULAR E.G.: LONGEST INCREASING SUB-SEQUENCE

�7

▸ Natural recursive algorithm?

Problem: given a sequence of numbers A[0], A[1], …, A[n-1], find longest “sub-sequence” whose elements are increasing

A107 AfD A147 Alto

f 3g2 10

7y

Split into cases based on whether thefirstelement is in the sub sequence or not

recursive call on remaining list

procedureM's Afo All Aln B find target

increasing sub seg that includes Afo

Answer to original problem

miaxMls Afi Afia Aln D

all

4 3Mls 411,2 3

GI write recursion for Mls 1,2 3 3Mis l 2,3

Mls Ato Atl Aln B

His t Afi AH findMI Ali Alief AlnB

ouput i f m

Correctness increasing

AT Take anynsubseg that includes

Afo Let it be Afo Afif Afif Alia

We know that Afi Alo so it will beconsidered in the recursive procedure

Now we can argue inductively that

It Mislaid Ali HI rtl

r.u.i.byI sftaIhifheystudent'ssuggestiondoes Not work

a

I bsgMl o InD

LECTURE 6

#(SUB-PROBLEMS) AND RUN TIME

�8

sub problems n

Running time n

Moira defining MIS iskeyotherwiseprocedure isn'tcorrect

should be carefulabout defining the

right sub problem

LECTURE 6

TRAVELING SALESMAN PROBLEM

�9

Problem: given undirected graph with edge wts, find a path of least total weight that visits every vertex precisely once and returns to start.

LECTURE 6

TRAVELING SALESMAN PROBLEM

�10

▸ Trivial algorithm?

Problem: given undirected graph with edge wts, find a path of least total weight that visits every vertex precisely once and returns to start.

ni canp

check all permutations see

which one has leastcost

F 3,6 on fin Nn al D n

Dynamicorithn 2n nu

key define sub problem verycarefully

LECTURE 6

TRAVELING SALESMAN PROBLEM

�11

▸ Trivial algorithm?

▸ Many heuristics for geometric cases

Problem: given undirected graph with edge wts, find a path of least total weight that visits every vertex precisely once and returns to start.

LECTURE 6

RECURSIVE ALGORITHM

�12

LECTURE 6

SUB-PROBLEMS, RUNNING TIME

�13

LECTURE 6

SUB-PROBLEMS, RUNNING TIME

�14

LECTURE 6

CAN WE DO BETTER?

�15

▸ Likely not — NP hard problem

▸ We will see that finding “approximately good” solution possible!

LECTURE 6

GREEDY ALGORITHMS

�16

▸ Greedy == “myopic”

▸ Algorithm makes sequence of decisions — chooses best at each step, without taking future into account

Greedy algorithmsalmost neues work

often give insights abouthowto solve

sometimes theyactually

LECTURE 6

EXAMPLE

�17

Scheduling: given set of jobs with processing times p1, p2, …, pn, find an order of processing so that sum of completion times is minimized

1 i 2 4 i 3

C 1 C I 3 i Cz 7 Cg 10

Intuitive idea do jobs with small first

Process in order of increasing pi 9rdgfnh.fm

LECTURE 6

NATURAL HEURISTIC?

�18

LECTURE 6

CORRECTNESS

�19

Clem Processing jobs in this order isthe best

possibleContradict

Proof1 Suppose the optimalorder

v u Ucouldnothaveit iz im beenoptorderI suppose we swap j

ti tiz t

ti tti.tpizmtiztk.TKi Pi Epi i similarly pi Epi

LECTURE 6

CORRECTNESS — PROOF 2

�20

LECTURE 6

EXAMPLE — MATCHING

�21

Matching: suppose we have n children, n gifts and “happiness” values Hij. Assign gifts to children to maximize “total happiness”

C G Hyvalue ofgiftGj to

Cz 0 Ga child C

o ciii li y

C o x Gnn

LECTURE 6

GREEDY ALGORITHM

�22

Go throughchildren

Ci CzaC o O For each child assign

gift from the nomaininggifts that has the

most value

Xcx

Greedy does not alwaysG X10

find best solution

LECTURE 6

CORRECTNESS

�23

LECTURE 6

CORRECTNESS

�24