lecture 14 - web.stanford.edu
TRANSCRIPT
![Page 1: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/1.jpg)
Lecture14Greedyalgorithms!
![Page 2: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/2.jpg)
Announcements
• HW6DueFriday!
• TONSOFPRACTICEONDYNAMICPROGRAMMING
![Page 3: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/3.jpg)
Lastweek
![Page 4: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/4.jpg)
Roadmap
Graphs!
Asymptotic
Analysis
Dynamic
ProgrammingGreedyAlgs
MIDTERM
The
Future!
Moredetailedscheduleonthewebsite!
![Page 5: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/5.jpg)
Thisweek
• Greedyalgorithms!
• Buildsonourideasfromdynamicprogramming
![Page 6: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/6.jpg)
Greedyalgorithms
• Makechoicesone-at-a-time.
• Neverlookback.
• Hopeforthebest.
![Page 7: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/7.jpg)
Today
• Onenon-exampleofagreedyalgorithm:
• Knapsackagain
• Threeexamplesofgreedyalgorithms:
• ActivitySelection
• JobScheduling
• HuffmanCoding
![Page 8: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/8.jpg)
Non-example
• UnboundedKnapsack.
• (Frompre-lectureexercise)
![Page 9: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/9.jpg)
• UnboundedKnapsack:
• SupposeIhaveinfinitecopiesofalloftheitems.
• What’sthemostvaluablewaytofilltheknapsack?
• “Greedy”algorithmforunboundedknapsack:
• TacoshavethebestValue/Weightratio!
• Keepgrabbingtacos!
Weight:
Value:
6 2 4 3 11
20 8 14 3513
Item:
Capacity:10
Totalweight:10
Totalvalue:42
Totalweight:9
Totalvalue:39
![Page 10: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/10.jpg)
ExamplewheregreedyworksActivityselection
FrisbeePractice
Orchestra
CS161study
group
Sleep
CS110
Class
TheoryLunch
TheorySeminar
Combinatorics
Seminar
Underwaterbasket
weavingclass
Math51Class
CS161Class
CS166Class
CS161
Section
CS161Office
Hours
Swimming
lessons
Programming
teammeeting
Socialactivity
time
Youcanonlydooneactivityatatime,andyouwantto
maximizethenumberofactivitiesthatyoudo.
Whattochoose?
![Page 11: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/11.jpg)
Activityselection
• Input:
• Activitiesa1,a2,…,an• Starttimess1,s2,…,sn• Finishtimesf1,f2,…,fn
• Output:
• Howmanyactivitiescanyoudotoday?
![Page 12: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/12.jpg)
GreedyAlgorithm
a3a1
a4a2
a5
a7
a6
time
• Pickactivityyoucanaddwiththesmallestfinishtime.
• Repeat.
![Page 13: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/13.jpg)
GreedyAlgorithm
a3a1
a4a2
a5
a7
a6
time
• Pickactivityyoucanaddwiththesmallestfinishtime.
• Repeat.
![Page 14: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/14.jpg)
GreedyAlgorithm
a3a1
a4a2
a5
a7
a6
time
• Pickactivityyoucanaddwiththesmallestfinishtime.
• Repeat.
![Page 15: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/15.jpg)
GreedyAlgorithm
a3a1
a4a2
a5
a7
a6
time
• Pickactivityyoucanaddwiththesmallestfinishtime.
• Repeat.
![Page 16: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/16.jpg)
GreedyAlgorithm
a3a1
a4a2
a5
a7
a6
time
• Pickactivityyoucanaddwiththesmallestfinishtime.
• Repeat.
![Page 17: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/17.jpg)
GreedyAlgorithm
a3a1
a4a2
a5
a7
a6
time
• Pickactivityyoucanaddwiththesmallestfinishtime.
• Repeat.
![Page 18: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/18.jpg)
GreedyAlgorithm
a3a1
a4a2
a5
a7
a6
time
• Pickactivityyoucanaddwiththesmallestfinishtime.
• Repeat.
![Page 19: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/19.jpg)
GreedyAlgorithm
a3a1
a4a2
a5
a7
a6
time
• Pickactivityyoucanaddwiththesmallestfinishtime.
• Repeat.
![Page 20: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/20.jpg)
Atleastit’sfast
• Runningtime:
• O(n)iftheactivitiesarealreadysortedbyfinishtime.
• OtherwiseO(nlog(n))ifyouhavetosortthemfirst.
![Page 21: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/21.jpg)
Whatmakesitgreedy?
• Ateachstepinthealgorithm,makeachoice.
• Hey,Icanincreasemyactivitysetbyone,
• Andleavelotsofroomforfuturechoices,
• Let’sdothatandhopeforthebest!!!
• Hope thatattheendoftheday,thisresultsinagloballyoptimalsolution.
![Page 22: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/22.jpg)
Threequestions
1. Doesthisgreedyalgorithmforactivityselectionwork?
2. Ingeneral,whenaregreedyalgorithmsagoodidea?
3. The“greedy”approachisoftenthefirstyou’dthinkof…
• Whyarewegettingtoitnow,inWeek8?
![Page 23: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/23.jpg)
Answers
1. Doesthisgreedyalgorithmforactivityselectionwork?
• Yes.
2. Ingeneral,whenaregreedyalgorithmsagoodidea?
• Whentheyexhibitespeciallyniceoptimalsubstructure.
3. The“greedy”approachisoftenthefirstyou’dthinkof…
• Whyarewegettingtoitnow,inWeek8?
• Relatedtodynamicprogramming!(WhichwedidinWeek7).
• Provingthatgreedyalgorithmsworkisoftennotsoeasy.
(Seemsto:IPython notebook…) (Butnowlet’sseewhy…)
![Page 24: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/24.jpg)
Whydoesitwork?
• Wheneverwemakeachoice,wedon’truleoutanoptimalsolution.
a3a1
a4a2
a5
a7
a6
time
a5a3
a7
There’ssomeoptimalsolutionthat
containsournextchoiceOurnext
choicewould
bethisone:
![Page 25: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/25.jpg)
Toseethis,consider
OptimalSubstructure
• Subproblem i :
• A[i]=NumberofactivitiesyoucandoafterActivityi finishes.
ai
a2
a7
a6
time
a4
aka3
Wanttoshow:whenwemakeachoiceak,theoptimalsolution
tothesmallersub-problemkwillhelpussolvesub-problemi
![Page 26: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/26.jpg)
Claim
• Letak havethesmallestfinishtimeamongactivitiesdo-ableafterai finishes.
• ThenA[i]=A[k]+1.
akai
a2
a7
a6
time
a4
aka3
A[k]:howmany
activitiescanIdohere?
A[i]:howmanyactivitiescanIdohere?
![Page 27: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/27.jpg)
Proof• Letak havethesmallestfinishtimeamongactivitiesdo-ableafterai finishes.
• ThenA[i]=A[k]+1.
a1ai
a2
a7
a6
time
a4
aka3
• ClearlyA[i]≥ A[k]+1• SincewehaveasolutionwithA[k]+1activities.
![Page 28: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/28.jpg)
ai
a2
Proof• Letak havethesmallestfinishtimeamongactivitiesdo-ableafterai finishes.
• ThenA[i]=A[k]+1.
• SupposetowardacontradictionthatA[i]> A[k]+1.
• There’ssomebettersolutiontosubproblem(i)that
doesn’tuseak• Sayaj endsfirstafterai inthatbettersolution.
• Removeaj andaddak fromthebettersolution.
akai
a2
a7
a6
time
a4
a3 a7a4
a3
aj
Thesetwodon’tcount
forsub-problem(i)so
let’sgreythemout.
![Page 29: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/29.jpg)
Proof• Letak havethesmallestfinishtimeamongactivitiesdo-ableafterai finishes.
• ThenA[i]=A[k]+1.
• SupposetowardacontradictionthatA[i]> A[k]+1.
• There’ssomebettersolutiontosubproblem(i)that
doesn’tuseak• Sayaj endsfirstafterai inthatbettersolution.
• Removeaj andaddak fromthebettersolution.
• Nowyouhaveasolutionofthesamesize…
butitincludesak soitmusthavesize≤A[k]+1.ak
ai
a2
a7
a6
time
aj
a3 a7a3
![Page 30: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/30.jpg)
Proof• Letak havethesmallestfinishtimeamongactivitiesdo-ableafterai finishes.
• ThenA[i]=A[k]+1.
a1ai
a2
a7
a6
time
a4
aka3
• ClearlyA[i]≥ A[k]+1• SincewehaveasolutionwithA[k]+1activities.
• Andwejustshowed A[i]≤ A[k]+1• Bycontradiction
• Thatprovestheclaim.
![Page 31: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/31.jpg)
Weneverruleoutanoptimalsolution
• We’ve shown:
• Ifwechooseak havethesmallestfinishtimeamongactivitiesdo-ableafterai finishes,thenA[i]=A[k]+1.
• Thatis:
• Assumethatwehaveanoptimalsolutionuptoai• Byaddingak wearestillontracktohitthatoptimalvalue
ai
a2
a7
a6
time
a4
aka3
![Page 32: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/32.jpg)
Sothealgorithmiscorrect
• Weneverruleoutanoptimalsolution
• Attheendofthealgorithm,we’vegotasolution.
• It’snotnotoptimal.
• Soitmustbeoptimal.
LuckytheLackadaisicalLemur
![Page 33: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/33.jpg)
Sothealgorithmiscorrect
• InductiveHypothesis:• Afteraddingthet’th thing,thereisanoptimalsolutionthatextendsthecurrentsolution.
• Basecase:• Afteraddingzeroactivities,thereisanoptimalsolutionextendingthat.
• Inductivestep:• TODO
• Conclusion:• Afteraddingthelastactivity,thereisanoptimalsolutionthatextendsthecurrentsolution.
• Thecurrentsolutionistheonlysolutionthatextendsthecurrentsolution.
• Sothecurrentsolutionisoptimal.
PluckythePedanticPenguin
![Page 34: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/34.jpg)
Inductivestep
• Supposethatafteraddingthet’th thing(Activityi),thereisanoptimalsolution:
• XactivitiesdoneandA[i]activitiesleft.
• Thenweaddthe(t+1)’st thing(Activityk).
• A[k]=A[i]- 1(bytheclaim)
• Now:
• X+1activitiesdoneandA[i]– 1activitiesleft.
• Samenumberasbefore!
• Stilloptimal.
![Page 35: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/35.jpg)
Sothealgorithmiscorrect
• InductiveHypothesis:• Afteraddingthet’th thing,thereisanoptimalsolutionthatextendsthecurrentsolution.
• Basecase:• Afteraddingzeroactivities,thereisanoptimalsolutionextendingthat.
• Inductivestep:• TODO
• Conclusion:• Afteraddingthelastactivity,thereisanoptimalsolutionthatextendsthecurrentsolution.
• Thecurrentsolutionistheonlysolutionthatextendsthecurrentsolution.
• Sothecurrentsolutionisoptimal.
PluckythePedanticPenguin
![Page 36: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/36.jpg)
Commonstrategyforgreedyalgorithms
• Makeaseriesofchoices.
• Showthat,ateachstep,ourchoicewon’truleoutanoptimalsolution attheendoftheday.
• Afterwe’vemadeallourchoices,wehaven’truledoutanoptimalsolution,sowemusthavefoundone.
![Page 37: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/37.jpg)
Commonstrategy(formally)forgreedyalgorithms
• InductiveHypothesis:
• Aftergreedychoicet,youhaven’truledoutsuccess.
• Basecase:
• Successispossiblebeforeyoumakeanychoices.
• Inductivestep:
• TODO
• Conclusion:
• Ifyoureachtheendofthealgorithmandhaven’truledoutsuccessthenyoumusthavesucceeded.
![Page 38: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/38.jpg)
DPviewofactivityselection
• Thisalgorithmismostnaturallyviewedasa
greedyalgorithm.• Makegreedychoices
• Neverruleoutsuccess
• But,wecouldviewitasaDPalgorithm• Takeadvantageofoptimalsub-structureandfill
inatable.
• We’lldothatnow.• Justforpedagogy!
• (Thisisn’tthebestwaytothinkaboutactivity
selection).
![Page 39: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/39.jpg)
RecipeforapplyingDynamicProgramming
• Step1:Identifyoptimalsubstructure.
• Step2:Findarecursiveformulationforthevalueoftheoptimalsolution.
• Step3:Usedynamicprogrammingtofindthevalueoftheoptimalsolution.
• Step4:Ifneeded,keeptrackofsomeadditionalinfosothatthealgorithmfromStep3canfindtheactualsolution.
• Step5:Ifneeded,codethisuplikeareasonableperson.
![Page 40: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/40.jpg)
Optimalsubstructure
• Subproblem i:
• A[i]=numberofactivitiesyoucandoafterActivityi finishes.
ai
a2
a7
a6
time
a4
a1a3
![Page 41: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/41.jpg)
RecipeforapplyingDynamicProgramming
• Step1:Identifyoptimalsubstructure.
• Step2:Findarecursiveformulationforthevalueoftheoptimalsolution.
• Step3:Usedynamicprogrammingtofindthevalueoftheoptimalsolution.
• Step4:Ifneeded,keeptrackofsomeadditionalinfosothatthealgorithmfromStep3canfindtheactualsolution.
• Step5:Ifneeded,codethisuplikeareasonableperson.
![Page 42: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/42.jpg)
Wedidthatalready
• Letak havethesmallestfinishtimeamongactivitiesdo-ableafterai finishes.
• ThenA[i]=A[k]+1.
a1ai
a2
a7
a6
time
a4
aka3
A[k]:howmany
activitiescanIdohere?
A[i]:howmanyactivitiescanIdohere?
![Page 43: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/43.jpg)
RecipeforapplyingDynamicProgramming
• Step1:Identifyoptimalsubstructure.
• Step2:Findarecursiveformulationforthevalueoftheoptimalsolution.
• Step3:Usedynamicprogrammingtofindthevalueoftheoptimalsolution.
• Step4:Ifneeded,keeptrackofsomeadditionalinfosothatthealgorithmfromStep3canfindtheactualsolution.
• Step5:Ifneeded,codethisuplikeareasonableperson.
![Page 44: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/44.jpg)
Top-downDP
• InitializeaglobalarrayAto[None,…,None]
• Makea“dummy”activitythatendsattime-1.
• def findNumActivities(i):
• IfA[i]!=None:
• Return A[i]
• LetActivitykbetheactivityIcanfitinmyscheduleafterActivityi withthesmallestfinishtime.
• If thereisnosuchactivityk,setA[i]=0
• Else,A[i]=findNumActivities(k)+1
• Return A[i]
• Return findNumActivities(0)
Thisisaterriblewaytowritethis!
Theonlythingthatmattershereisthatthe
highlightedlinesareourrecursiverelationship.
SeeIPython notebookfor
implementation
![Page 45: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/45.jpg)
RecipeforapplyingDynamicProgramming
• Step1:Identifyoptimalsubstructure.
• Step2:Findarecursiveformulationforthevalueoftheoptimalsolution.
• Step3:Usedynamicprogrammingtofindthevalueoftheoptimalsolution.
• Step4:Ifneeded,keeptrackofsomeadditionalinfosothatthealgorithmfromStep3canfindtheactualsolution.
• Step5:Ifneeded,codethisuplikeareasonableperson.
![Page 46: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/46.jpg)
Top-downDP
• InitializeaglobalarrayAto[None,…,None]
• InitializeaglobalarrayNextto[None,…,None]
• Makea“dummy”activitythatendsattime-1.
• def findNumActivities(i):• IfA[i]!=None:
• Return A[i]• LetActivitykbetheactivityIcanfitinmyscheduleafterActivityi withthesmallestfinishtime.
• If thereisnosuchactivityk,setA[i]=0• Else,A[i]=findNumActivities(k)+1and Next[i]=k• Return A[i]
• findNumActivities(0)
• Stepthrough“Next”arraytogetschedule.
Thisisaterriblewaytowritethis!
Theonlythingthatmattershereisthatthe
highlightedlinesareourrecursiverelationship.
SeeIPython notebookfor
implementation
![Page 47: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/47.jpg)
Let’sstepthroughit.(SeeIPython notebookforcodewithsomeprintstatements)
Thislooksprettyfamiliar!!
![Page 48: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/48.jpg)
Let’sstepthroughit.
a3a1
a4a2
a5
a7
a6
time
• Startwiththeactivitywiththesmallestfinishtime.
![Page 49: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/49.jpg)
Let’sstepthroughit
a3a1
a4a2
a5
a7
a6
time
• Nowfindthenextactivitystilldo-ablewiththesmallestfinishtime,andrecurse afterthat.
![Page 50: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/50.jpg)
Let’sstepthroughit
a3a1
a4a2
a5
a7
a6
time
• Nowfindthenextactivitystilldo-ablewiththesmallestfinishtime,andrecurse afterthat.
![Page 51: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/51.jpg)
Let’sstepthroughit
a3a1
a4a2
a5
a7
a6
time
• Nowfindthenextactivitystilldo-ablewiththesmallestfinishtime,andrecurse afterthat.
![Page 52: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/52.jpg)
Let’sstepthroughit
a3a1
a4a2
a5
a7
a6
time
• Ta-da!
It’sexactlythesame*asthegreedysolution!
*ifyouimplementthetop-downDPsolutionappropriately.
![Page 53: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/53.jpg)
Sub-problemgraphview
• Divide-and-conquer:
Bigproblem
sub-problemsub-problem
sub-sub-
problem
sub-sub-
problem
sub-sub-
problem
sub-sub-
problem
sub-sub-
problem
![Page 54: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/54.jpg)
Sub-problemgraphview
• DynamicProgramming:
Bigproblem
sub-problemsub-problem
sub-sub-
problemsub-sub-
problem
sub-sub-
problem
sub-sub-
problem
sub-problem
![Page 55: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/55.jpg)
Sub-problemgraphview
• Greedyalgorithms:
Bigproblem
sub-sub-
problem
sub-problem
![Page 56: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/56.jpg)
Sub-problemgraphview
• Greedyalgorithms:
Bigproblem
sub-sub-
problem
sub-problem
• Notonlyisthereoptimalsub-structure:• optimalsolutionstoaproblemaremadeup
fromoptimalsolutionsofsub-problems
• buteachproblemdependsononlyone
sub-problem.
![Page 57: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/57.jpg)
Answers
1. Doesthisgreedyalgorithmforactivityselectionwork?
• Yes.
2. Ingeneral,whenaregreedyalgorithmsagoodidea?
• Whentheyexhibitespeciallyniceoptimalsubstructure.
3. The“greedy”approachisoftenthefirstyou’dthinkof…
• Whyarewegettingtoitnow,inWeek8?
• Relatedtodynamicprogramming!(WhichwedidinWeek7).
• Provingthatgreedyalgorithmsworkisoftennotsoeasy.
![Page 58: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/58.jpg)
Let’sseeafewmoreexamples
![Page 59: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/59.jpg)
Anotherexample:
Scheduling
Overcommitted
StanfordStudent
CS161HW!
Callyourparents!
MathHW!
EconHW!
Practicemusicalinstrument!
ReadCLRS!
Haveasociallife!
Sleep!
Administrativestuffforyourstudentclub!
Dolaundry!
Meditate!
![Page 60: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/60.jpg)
Scheduling
• ntasks
• Taski takesti hours
• Everythingisalreadylate!
• Foreveryhourthatpassesuntiltaski isdone,payci
• CS161HW,thenSleep:costs10⋅ 2+(10+8)⋅ 3=74units• Sleep,thenCS161HW:costs8⋅ 3+(10+8)⋅ 2=60units
CS161HW!
Sleep!
10hours
8hours
Cost:2 unitsper
houruntilit’sdone.
Cost:3unitsper
houruntilit’sdone.
![Page 61: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/61.jpg)
Optimalsubstructure
• Thisproblembreaksupnicelyintosub-problems:
JobA JobB JobC JobD
Supposethisistheoptimalschedule:
Thenthismustbetheoptimal
scheduleonjustjobsB,C,D.
![Page 62: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/62.jpg)
Optimalsubstructure
• Seemsamenabletoagreedyalgorithm:
JobA JobB JobC JobD
Takethebestjobfirst Thensolvethisproblem
JobBJobC JobD
Takethebestjobfirst Thensolvethisproblem
JobBJobD
Takethebestjobfirst
(Thatone’seasyJ )
Thensolvethisproblem
![Page 63: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/63.jpg)
Whatdoes“best”mean?
• Recipeforgreedyalgorithmanalysis:
• Wemakeaseriesofchoices.
• Weshowthat,ateachstep,ourchoicewon’truleoutanoptimalsolution attheendoftheday.
• Afterwe’vemadeallourchoices,wehaven’truledoutanoptimalsolution,sowemusthavefoundone.
JobA JobB JobC JobD
“Best”means:won’truleoutanoptimalsolution.
Theoptimalsolutiontothisproblemextendsanoptimalsolutiontothewholething.
![Page 64: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/64.jpg)
Head-to-head
• Ofthesetwojobs,whichshouldwedofirst?
• Cost(AthenB)=x⋅z+(x+y) ⋅ w• Cost(BthenA)=y ⋅w+(x+y) ⋅z
JobA
JobB
xhours
y hours
Cost:z unitsper
houruntilit’sdone.
Cost:w unitsper
houruntilit’sdone.
AthenBisbetterthanBthenAwhen:
𝑥𝑧 + 𝑥 + 𝑦 𝑤 ≤ 𝑦𝑤 + 𝑥 + 𝑦 𝑧𝑥𝑧 + 𝑥𝑤 + 𝑦𝑤 ≤ 𝑦𝑤 + 𝑥𝑧 + 𝑦𝑧
𝑤𝑥 ≤ 𝑦𝑧𝑤𝑦 ≤
𝑧𝑥
Whatmattersistheratio:
costofdelaytimeittakes
Dothejobwiththe
biggestratiofirst.
![Page 65: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/65.jpg)
Lemma
• GivenjobssothatJobi takestime ti withcostci ,
• Thereisanoptimalschedulesothatthefirstjobistheonethatmaximizestheratioci/ti
• Proof:
• SayJobBmaximizesthisratio,andit’snotfirst:
• SwitchAandB!Nothingelsewillchange,andweshowedonthepreviousslidethatthecostwon’tincrease.
• RepeatuntilBisfirst.
JobA JobB
cA/tA >=cB/tB
JobC JobD
JobAJobBJobC JobD
![Page 66: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/66.jpg)
Choosegreedily:Biggestcost/timeratiofirst
• Jobi takestime ti withcostci
• Thereisanoptimalschedulesothatthefirstjobistheonethatmaximizestheratioci/ti
• Soifwechoosejobsgreedilyaccordingtoci/ti,weneverruleoutsuccess!
![Page 67: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/67.jpg)
GreedySchedulingSolution
• scheduleJobs(JOBS):
• SortJOBSbytheratio:
• 𝒓𝒊 = 𝒄𝒊𝒕𝒊 =
costofdelayingjobitimejobitakestocomplete
• Saythatsorted_JOBS[i] isthejobwiththei’th biggestri• Return sorted_JOBS
TherunningtimeisO(nlog(n))
Nowyoucangoaboutyourschedule
peacefully,intheoptimalway.
![Page 68: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/68.jpg)
Formally,useinduction!
• Inductivehypothesis:
• Thereisanoptimalorderingsothatthefirsttjobsaresorted_JOBS[:t].
• Basecase:
• Whent=0,thisreads:“Thereisanoptimalorderingsothatthefirst0jobsare[]”
• That’strue.
• InductiveStep:
• Boilsdownto:thereisanoptimalorderingonsorted_JOBS[t:]sothatsorted_JOBS[t]isfirst.
• ThisfollowsfromtheLemma.
• Conclusion:
• Whent=n,thisreads:“Thereisanoptimalorderingsothatthefirstnjobsaresorted_JOBS.”
• aka,whatwereturnedisanoptimalordering.
SLIDESKIPPEDINCLASS
![Page 69: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/69.jpg)
Whathavewelearned?
• Agreedyalgorithmworksforscheduling
• Thisfollowedthesameoutlineasthepreviousexample:
• Identifyoptimalsubstructure:
• Findawaytomake“safe”choicesthatwon’truleoutanoptimalsolution.
• largestratiosfirst.
JobA JobB JobC JobD
![Page 70: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/70.jpg)
OnemoreexampleHuffmancoding
• everyday english sentence• 01100101011101100110010101110010011110010110010001100001011110010010000001100101011011100110011101101100011010010111001101101000001000000111001101100101011011100111010001100101011011100110001101100101
• qwertyui_opasdfg+hjklzxcv• 01110001011101110110010101110010011101000111100101110101011010010101111101101111011100000110000101110011011001000110011001100111001010110110100001101010011010110110110001111010011110000110001101110110
![Page 71: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/71.jpg)
OnemoreexampleHuffmancoding
• everyday english sentence• 01100101 0111011001100101 01110010011110010110010001100001011110010010000001100101 011011100110011101101100011010010111001101101000001000000111001101100101 011011100111010001100101 011011100110001101100101
• qwertyui_opasdfg+hjklzxcv• 01110001011101110110010101110010011101000111100101110101011010010101111101101111011100000110000101110011011001000110011001100111001010110110100001101010011010110110110001111010011110000110001101110110
ASCIIisprettywasteful.Ife
showsupsooften,weshould
haveamoreparsimoniousway
ofrepresentingit!
![Page 72: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/72.jpg)
Supposewehavesomedistributiononcharacters
![Page 73: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/73.jpg)
Supposewehavesomedistributiononcharacters
A B C D E F
Percentage
Letter
45
1312
16
9
5
Forsimplicity,
let’sgowiththis
made-upexample
Howtoencodethemas
efficientlyaspossible?
![Page 74: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/74.jpg)
Try0(likeASCII)
A B C D E F
Percentage
Letter
45
1312
16
9
5
000 011001 010 100 101
• Everyletterisassignedabinarystring
ofthreebits.
Wasteful!
• 110and111areneverused.
• Weshouldhaveashorterwayof
representingA.
![Page 75: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/75.jpg)
Try1
A B C D E F
Percentage
Letter
45
1312
16
9
5
0 100 01 10 11
• Everyletterisassignedabinarystring
ofoneortwobits.
• Themorefrequentlettersgetthe
shorterstrings.
• Problem:
• Does000meanAAAorBAorAB?
![Page 76: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/76.jpg)
Try2:prefix-freecoding
A B C D E F
Percentage
Letter
45
1312
16
9
5
01 00101 110 111 100
• Everyletterisassignedabinarystring.
• Morefrequentlettersgetshorterstrings.
• Noencodedstringisaprefixofanyother.
10010101
Confusingly,“prefix-freecodes”arealsosometimes
called“prefixcodes”(includinginCLRS).
![Page 77: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/77.jpg)
Try2:prefix-freecoding
A B C D E F
Percentage
Letter
45
1312
16
9
5
01 00101 110 111 100
• Everyletterisassignedabinarystring.
• Morefrequentlettersgetshorterstrings.
• Noencodedstringisaprefixofanyother.
10010101 F
Confusingly,“prefix-freecodes”arealsosometimes
called“prefixcodes”(includinginCLRS).
![Page 78: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/78.jpg)
Try2:prefix-freecoding
A B C D E F
Percentage
Letter
45
1312
16
9
5
01 00101 110 111 100
• Everyletterisassignedabinarystring.
• Morefrequentlettersgetshorterstrings.
• Noencodedstringisaprefixofanyother.
10010101 FB
Confusingly,“prefix-freecodes”arealsosometimes
called“prefixcodes”(includinginCLRS).
![Page 79: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/79.jpg)
Try2:prefix-freecoding
A B C D E F
Percentage
Letter
45
1312
16
9
5
01 00101 110 111 100
• Everyletterisassignedabinarystring.
• Morefrequentlettersgetshorterstrings.
• Noencodedstringisaprefixofanyother.
10010101 FBA
Question:Whatisthemost
efficientwaytodoprefix-free
coding?(Thisisn’tit).
Confusingly,“prefix-freecodes”arealsosometimes
called“prefixcodes”(includinginCLRS).
![Page 80: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/80.jpg)
Aprefix-freecodeisatree
D:16A:45
B:13F:5 C:12 E:9
0
0 0
0 0 1
1
1
1
1
00 01
100 101 110 111Aslongasalltheletters
showupasleaves,this
codeis prefix-free.
B:13belowmeansthat‘B’
makesup13%ofthe
charactersthateverappear.
![Page 81: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/81.jpg)
Sometreesarebetterthanothers
D:16A:45
B:13F:5 C:12 E:9
0
0 0
0 0 1
1
1
1
1
00 01
100 101 110 111
• Imaginechoosingaletteratrandomfromthelanguage.
• Notuniform,butaccordingtoourhistogram!
• Thecostofatreeistheexpectedlengthoftheencodingofthatletter.
Expectedcostofencodingaletterwiththistree:
𝟐 𝟎. 𝟒𝟓 + 𝟎. 𝟏𝟔 + 𝟑 𝟎. 𝟎𝟓 + 𝟎. 𝟏𝟑 + 𝟎. 𝟏𝟐 + 𝟎. 𝟎𝟗 = 𝟐. 𝟑𝟗
Cost=
K 𝑃 𝑥 ⋅ depth(𝑥)�
QRSTRUV P(x)isthe
probability
ofletterx
Thedepthinthe
treeisthelength
oftheencoding
![Page 82: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/82.jpg)
Question
• GivenadistributionP onletters,findthelowest-costtree,where
cost(tree) = K 𝑃 𝑥 ⋅ depth(𝑥)�
XYZ[Y\V P(x)isthe
probability
ofletterx
Thedepthinthe
treeisthelength
oftheencoding
![Page 83: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/83.jpg)
Optimalsub-structure
• Supposethisisanoptimaltree:
10
Thenthisisan
optimaltreeon
fewerletters.
Otherwise,wecould
changethissub-tree
andendupwitha
betteroveralltree.
![Page 84: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/84.jpg)
Inordertodesignagreedyalgorithm
• Thinkaboutwhatlettersbelonginthissub-problem...
10What’sasafe
choicetomake
fortheselower
sub-trees?
Infrequent
elements!Wewantthemaslow
downaspossible.
![Page 85: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/85.jpg)
Solutiongreedilybuildsubtrees,startingwiththeinfrequentletters
D:16A:45 B:13 F:5C:12 E:9
14
0 1
![Page 86: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/86.jpg)
Solutiongreedilybuildsubtrees,startingwiththeinfrequentletters
D:16A:45 B:13 F:5C:12 E:9
14
0 1
25
0 1
![Page 87: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/87.jpg)
Solutiongreedilybuildsubtrees,startingwiththeinfrequentletters
D:16A:45 B:13 F:5C:12 E:9
14
0 1
25
0 1
30
1
0
![Page 88: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/88.jpg)
Solutiongreedilybuildsubtrees,startingwiththeinfrequentletters
D:16A:45 B:13 F:5C:12 E:9
14
0 1
25
0 1
30
1
0
551
0
![Page 89: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/89.jpg)
Solutiongreedilybuildsubtrees,startingwiththeinfrequentletters
D:16A:45 B:13 F:5C:12 E:9
14
0 1
25
0 1
30
1
0
551
0
1001
0
![Page 90: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/90.jpg)
Solutiongreedilybuildsubtrees,startingwiththeinfrequentletters
D:16
A:45
B:13
F:5
C:12
E:9
14
0 1
25
0 1
30
10
5510
100
10
0
100 101 110
1110 1111
Expectedcostofencodingaletter:
𝟏 ⋅ 𝟎. 𝟒𝟓+
𝟑 ⋅ 𝟎. 𝟒𝟏+
𝟒 ⋅ 𝟎. 𝟏𝟒= 𝟐. 𝟐𝟒
![Page 91: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/91.jpg)
Whatexactlywasthealgorithm?
• Createanodelikeforeachletter/frequency
• Thekeyisthefrequency(16inthiscase)
• LetCURRENT bethelistofallthesenodes.
• while len(CURRENT)>1:
• X andY← thenodesinCURRENT withthesmallestkeys.
• CreateanewnodeZ withZ.key =X.key +Y.key
• SetZ.left =X,Z.right =Y
• AddZ toCURRENT andremoveX andY
• returnCURRENT[0]
D:16
F:5 E:9
14
0 1
Y
Z
XD:16A:45 B:13 C:12
![Page 92: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/92.jpg)
Doesitwork?
• Yes.
• Samestrategy:
• Showthatateachstep,thechoiceswearemakingwon’truleoutanoptimalsolution.
• Lemma:
• Supposethatxandyarethetwoleast-frequentletters.Thenthereisanoptimaltreewherexandyaresiblings.
D:16A:45 B:13 F:5C:12 E:9
14
0 1
![Page 93: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/93.jpg)
Lemmaproofidea
• Saythatanoptimaltreelookslikethis:
• Whathappenstothecostifweswapxfora?• thecostcan’tincrease;awasmorefrequentthanx,andwejustmadeitsencodingshorter.
• Repeatthislogicuntilwegetanoptimaltreewithxandyassiblings.• Thecostneverincreasedsothistreeisstilloptimal.
Ifxandyarethetwoleast-frequentletters,there
isanoptimaltreewherexandyaresiblings.
x
a
Lowest-levelsibling
nodes:atleastoneof
themisneitherxnory
![Page 94: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/94.jpg)
Lemmaproofidea
• Saythatanoptimaltreelookslikethis:
• Whathappenstothecostifweswapxfora?• thecostcan’tincrease;awasmorefrequentthanx,andwejustmadeitsencodingshorter.
• Repeatthislogicuntilwegetanoptimaltreewithxandyassiblings.• Thecostneverincreasedsothistreeisstilloptimal.
x y
Lowest-levelsibling
nodes:atleastoneof
themisneitherxnory
Ifxandyarethetwoleast-frequentletters,there
isanoptimaltreewherexandyaresiblings.
![Page 95: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/95.jpg)
Proofstrategyjustlikebefore
• Showthatateachstep,thechoiceswearemakingwon’truleoutanoptimalsolution.
• Lemma:
• Supposethatxandyarethetwoleast-frequentletters.Thenthereisanoptimaltreewherexandyaresiblings.
D:16A:45 B:13 F:5C:12 E:9
14
0 1
![Page 96: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/96.jpg)
Proofstrategyjustlikebefore
• Showthatateachstep,thechoiceswearemakingwon’truleoutanoptimalsolution.
• Lemma:
• Supposethatxandyarethetwoleast-frequentletters.Thenthereisanoptimaltreewherexandyaresiblings.
That’senoughtoshowthatwe
don’truleoutoptimalityafter
thefirststep.
Whataboutoncewestart
groupingstuff?
D:16A:45 B:13 F:5C:12 E:9
0 1
25
01
1
014
30
![Page 97: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/97.jpg)
Lemma2thisdistinctiondoesn’treallymatter
D:16
F:5E:9
14
0 1
25
0 1
30
10
5510
100
10
C:12B:13
A:45 A:4555
10
100
10
G:25H:30
Thefirstthingisanoptimal
treeon{A,B,C,D,E,F}
ifandonlyif
thesecondthingisan
optimaltreeon{A,G,H}
![Page 98: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/98.jpg)
• Foraproof:
• SeeCLRS,Lemma16.3
• Rigorousalthoughpresentedinaslightlydifferentway
• SeeLectureNotes14
• Abitsketchier,butpresentedinthesamewayashere
• Proveityourself!
• Thisisthebest!
Siggi theStudiousStork
Gettingallthedetails
isn’tthatimportant,but
youshouldconvince
yourselfthatthisistrue.
Lemma2thisdistinctiondoesn’treallymatter
![Page 99: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/99.jpg)
Together
• Lemma1:
• Supposethatxandyarethetwoleast-frequentletters.Thenthereisanoptimaltreewherexandyaresiblings.
• Lemma2:
• WemayaswellimaginethatCURRENTcontainsonlyleaves.
• Theseimply:
• Ateachstep,ourchoicedoesn’truleoutanoptimaltree.
![Page 100: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/100.jpg)
Thewholeargument
• Inductivehypothesis:• afterthet’th step,
• thereisanoptimaltreecontainingthecurrentsubtreesas“leaves”
• Basecase:• afterthe0’thstep,
• thereisanoptimaltreecontainingallthecharacters.
• Inductivestep:• TODO
• Conclusion:• afterthelaststep,
• thereisanoptimaltreecontainingthiswholetreeasasubtree.
• aka,• afterthelaststepthetreewe’veconstructedisoptimal.
Afterthet’th step,we’vegotabunchofcurrentsub-trees:
Inductivehyp.asserts
thatoursubtreescanbe
assembledintoan
optimaltree:
![Page 101: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/101.jpg)
Inductivestep
• Supposethattheinductivehypothesisholdsfort-1
• Aftert-1steps,thereisanoptimaltreecontainingallthecurrentsub-treesas“leaves.”
• Wanttoshow:
• Aftertsteps,thereisanoptimaltreecontainingallthecurrentsub-treesasleaves.
We’vegotabunchofcurrentsub-trees:
xy
saythatxandyarethetwosmallest.
wz
![Page 102: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/102.jpg)
Inductivestep
• Supposethattheinductivehypothesisholdsfort-1
• Aftert-1steps,thereisanoptimaltreecontainingallthecurrentsub-treesas“leaves.”
• ByLemma2,mayaswelltreatas
We’vegotabunchofcurrentsub-trees:
xyw
saythatxandyarethetwosmallest.
aa
yxw
z
z
![Page 103: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/103.jpg)
Inductivestep
• Supposethattheinductivehypothesisholdsfort-1
• Aftert-1steps,thereisanoptimaltreecontainingallthecurrentsub-treesas“leaves.”
• ByLemma2,mayaswelltreatas
• Inparticular,optimaltreesonthisnewalphabetcorrespondtooptimaltreesontheoriginalalphabet.
We’vegotabunchofcurrentsub-trees:
xyw
saythatxandyarethetwosmallest.
aa
zwyx
z
![Page 104: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/104.jpg)
Inductivestep
• Supposethattheinductivehypothesisholdsfort-1
• Aftert-1steps,thereisanoptimaltreecontainingallthecurrentsub-treesas“leaves.”
• Ouralgorithmwoulddothisatlevelt:
We’vegotabunchofcurrentsub-trees:
xyw
saythatxandyarethetwosmallest.
xy
wa a=x+y
z
zwyx
z
![Page 105: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/105.jpg)
Inductivestep
• Supposethattheinductivehypothesisholdsfort-1
• Aftert-1steps,thereisanoptimaltreecontainingallthecurrentsub-treesas“leaves.”
• Ouralgorithmwoulddothisatlevelt:
We’vegotabunchofcurrentsub-trees:
xyw
saythatxandyarethetwosmallest.
zw
a
yx
xy
wa a=x+y
Lemma1impliesthatthere’s
anoptimalsub-treethatlooks
likethis;aka,whatour
algorithmdidokay.
z
z
![Page 106: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/106.jpg)
Inductivestep
• Supposethattheinductivehypothesisholdsfort-1
• Aftert-1steps,thereisanoptimaltreecontainingallthecurrentsub-treesas“leaves.”
• Ouralgorithmwoulddothisatlevelt:
We’vegotabunchofcurrentsub-trees:
xyw
saythatxandyarethetwosmallest.
w
a
xy
wa a=x+y
Lemma2againsaysthat
there’sanoptimaltreethat
lookslikethis
z
yxz
z
![Page 107: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/107.jpg)
Inductivestep
• Supposethattheinductivehypothesisholdsfort-1
• Aftert-1steps,thereisanoptimaltreecontainingallthecurrentsub-treesas“leaves.”
• Ouralgorithmwoulddothisatlevelt:
We’vegotabunchofcurrentsub-trees:
xyw
saythatxandyarethetwosmallest.
w
a
xy
wa a=x+y
Lemma2againsaysthat
there’sanoptimaltreethat
lookslikethis
z
yxz
Thisiswhatwe
wantedtoshowfor
theinductivestep.
z
![Page 108: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/108.jpg)
Inductiveoutline:
• Inductivehypothesis:• afterthet’th step,
• thereisanoptimaltreecontainingthecurrentsubtreesas“leaves”
• Basecase:• afterthe0’thstep,
• thereisanoptimaltreecontainingallthevertices.
• Inductivestep:• TODO
• Conclusion:• afterthelaststep,
• thereisanoptimaltreecontainingthiswholetreeasasubtree.
• aka,• afterthelaststepthetreewe’veconstructedisoptimal.
Afterthet’th step,we’vegotabunchofcurrentsub-trees:
Inductivehyp.asserts
thatoursubtreescanbe
assembledintoan
optimaltree:
![Page 109: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/109.jpg)
Whathavewelearned?
• ASCIIisn’tanoptimalwaytoencodeEnglish,sincethedistributiononlettersisn’tuniform.
• HuffmanCodingisanoptimalway!
• Tocomeupwithanoptimalschemeforanylanguageefficiently,wecanuseagreedyalgorithm.
• Tocomeupwithagreedyalgorithm:
• Identifyoptimalsubstructure
• Findawaytomake“safe”choicesthatwon’truleoutanoptimalsolution.
• Createsubtreesoutofthesmallesttwocurrentsubtrees.
![Page 110: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/110.jpg)
RecapI
• Greedyalgorithms!
• Threeexamples:
• ActivitySelection
• SchedulingJobs
• HuffmanCoding
![Page 111: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/111.jpg)
RecapII
• Greedyalgorithms!
• Ofteneasytowritedown
• Butmaybehardtocomeupwithandhardtojustify
• Thenaturalgreedyalgorithmmaynotalwaysbecorrect.
• Aproblemisagoodcandidateforagreedyalgorithmif:
• ithasoptimalsubstructure
• thatoptimalsubstructureisREALLYNICE
• solutionsdependonjustoneothersub-problem.
![Page 112: Lecture 14 - web.stanford.edu](https://reader034.vdocuments.site/reader034/viewer/2022052616/628e231968b3351a3f4c9158/html5/thumbnails/112.jpg)
Nexttime
• GreedyalgorithmsforMinimumSpanningTree!
• Pre-lectureexercise:candidategreedyalgorithmsforMST
Before nexttime