10/1/20151 cs 3343: analysis of algorithms lecture 1: introduction some slides courtesy from jeff...
TRANSCRIPT
![Page 1: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/1.jpg)
04/19/23 1
CS 3343: Analysis of Algorithms
Lecture 1: Introduction
Some slides courtesy from Jeff Edmonds @ York University
![Page 2: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/2.jpg)
04/19/23 2
The course
• Instructor: Dr. Jianhua Ruan– [email protected]– Office: NPB 3.202– Office hours: T 1-2pm or by appointment
• TA: Erdal Akin
• Grader: Chih-Wei Yu
• Contact info online
![Page 3: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/3.jpg)
04/19/23 3
The course
• Purpose: a rigorous introduction to the design and analysis of algorithms
• Textbook: Introduction to Algorithms, Cormen, Leiserson, Rivest, Stein– An excellent reference you should own – Go to course website for a link to the errata– http://cs.utsa.edu/~jruan/teaching/cs3343_spring_2015/
• Or go to http://cs.utsa.edu/~jruan/ then follow “teaching”.
– Under “textbook”
![Page 4: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/4.jpg)
04/19/23 4
Course Format
• Two lectures + 1 recitation / week• Recitation
– Mandatory– CS3343.002 should attend CS3341.003– CS3343.001 can attend CS3341.001/002– No recitation the first week
• ~8 homework assignments– Problem sets– Occasional programming assignments– Typically due in 1-1.5 week
• Occasional in-class quizzes and exercises• Two midterms + final exam
![Page 5: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/5.jpg)
04/19/23 5
Grading policy
• Homework: 30%• midterm 1: 16%• midterm 2: 16%• Final exam: 32%• Quiz and participation 6%• One lowest grade in homework will be dropped • I reserve the right to slightly adjust the weights
of individual components if necessary.
![Page 6: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/6.jpg)
04/19/23 6
Late homework submissions
• 10% penalty if submitted the same day after the instructor left classroom
• 15% penalty each additional day after the submission deadline
• Submission will not be accepted once TA shows solution in recitation or instructor puts solution online
• Email submission is acceptable in case of emergency. Include “CS3343” in subject.
![Page 7: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/7.jpg)
04/19/23 7
Exams
• Exams cannot be made up, cannot be taken early, and must be taken in class at the scheduled time.
• Proofs are needed for exceptions or true emergencies
![Page 8: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/8.jpg)
04/19/23 8
Cheating
• You are not allowed to read, copy, or rewrite the solutions written by others (in this or previous terms). Copying materials from websites, books or any other sources is considered equivalent to copying from another student.
• If two people are caught sharing solutions, then both the copier and copiee will be held equally responsible, which will result in zero point in homework.
• Cheating on an exam will result in failing the course.
![Page 9: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/9.jpg)
04/19/23 9
Getting answers from the internet isCHEATING
Getting answers from your friends isCHEATING
I will send it to the Dean!You will be nailed!
However, teamwork is encouraged.
Group size at most 3.
Clearly acknowledge who you worked with.
![Page 10: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/10.jpg)
04/19/23 10
Do NOT get answers from other groups!
Do NOT do half the assignmentand your partner does the other half.
Each try all on your own.
Discuss ideas verbally at a high-level but write up on your own.
![Page 11: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/11.jpg)
04/19/23 11
Attendance
• Missing 3 or more classes / recitations (whenever attendance is checked) will result in a minimum of 5 points taken off your final grade
• In reality, attendance and final grade are highly correlated, even without the penalty
![Page 12: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/12.jpg)
04/19/23 12
Feedbacks
• We appreciate your feedbacks
• Your feedbacks help me know how I can better deliver my lectures, which will ultimately benefit you
• You get bonus points in homework for your feedbacks
![Page 13: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/13.jpg)
04/19/23 13
Introduction
• Why should you study algorithms
• What is an algorithm
• What you can expect to learn from this course
![Page 14: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/14.jpg)
04/19/23 14
Please feel free to ask questions!
Help me know what people are not understanding
We do have a lot of material
It’s your job to slow me down
![Page 15: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/15.jpg)
04/19/23 15
So you want to be a computer scientist?
![Page 16: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/16.jpg)
04/19/23 16
Is your goal to be a mundane programmer?
![Page 17: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/17.jpg)
04/19/23 17
Or a great leader and thinker?
![Page 18: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/18.jpg)
04/19/23 18
Boss assigns task:
– Given today’s prices of pork, grain, sawdust, …– Given constraints on what constitutes a hotdog.– Make the cheapest hotdog.
Everyday industry asks these questions.
![Page 19: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/19.jpg)
04/19/23 19
• Um? Tell me what to code.
With more sophisticated software engineering systems,the demand for mundane programmers will diminish.
Your answer:
![Page 20: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/20.jpg)
04/19/23 20
Your answer:
• I learned this great algorithm that will work.
Soon all known algorithms will be available in libraries.
Your boss might change his mind. He now wants to make the most profitable hotdogs.
![Page 21: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/21.jpg)
04/19/23 21
Your answer:
• I can develop a new algorithm for you.
Great thinkers will always be needed.
![Page 22: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/22.jpg)
04/19/23 22
How do I become a great thinker?
Maybe I’ll never be…
![Page 23: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/23.jpg)
04/19/23 23
Learn from the classical problems
![Page 24: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/24.jpg)
04/19/23 24
Shortest path
Start
end
![Page 25: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/25.jpg)
04/19/23 25
Traveling salesman problem
![Page 26: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/26.jpg)
04/19/23 26
Knapsack problem
![Page 27: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/27.jpg)
04/19/23 27
• There is only a handful of classical problems. – Nice algorithms have been designed for them
• If you know how to solve a classical problem (e.g., the shortest-path problem), you can use it to do a lot of different things– Abstract ideas from the classical problems– Map your boss’ requirement to a classical problem– Solve with classical algorithms– Modify it if needed
![Page 28: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/28.jpg)
04/19/23 28
• What if you can NOT map your boss’ requirement to any existing classical problem?
• How to design an algorithm by yourself?• Learn some meta algorithms
– A meta algorithm is a class of algorithms for solving similar abstract problems
– There is only a handful of them• E.g. divide and conquer, greedy algorithm, dynamic
programming
– Learn the ideas behind the meta algorithms• Design a concrete algorithm for your task
![Page 29: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/29.jpg)
04/19/23 29
Useful learning techniques
• Read Ahead. Read the textbook before the lectures. This will facilitate more productive discussion during class.
• Explain the material over and over again out loud to yourself, to each other, and to your stuffed bear.
• Be creative. Ask questions: Why is it done this way and not that way?
• Practice. Try to solve as many exercises in the textbook as you can.
![Page 30: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/30.jpg)
04/19/23 30
What will we study?
• Expressing algorithms– Define a problem precisely and abstractly– Presenting algorithms using pseudocode
• Algorithm validation– Prove that an algorithm is correct
• Algorithm analysis– Time and space complexity– What problems are so hard that efficient algorithms are
unlikely to exist
• Designing algorithms– Algorithms for classical problems– Meta algorithms (classes of algorithms) and when you
should use which
![Page 31: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/31.jpg)
04/19/23 31
What is an algorithm?
• Algorithms are the ideas behind computer programs.
• An algorithm is the thing that stays the same regardless of programming language and the computing hardware
![Page 32: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/32.jpg)
04/19/23 32
What is an algorithm? (cont’)
• An algorithm is a precise and unambiguous specification of a sequence of steps that can be carried out to solve a given problem or to achieve a given condition.
• An algorithm accepts some value or set of values as input and produces a value or set of values as output.
• Algorithms are closely intertwined with the nature of the data structure of the input and output values
![Page 33: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/33.jpg)
04/19/23 33
How to express algorithms?
Nature language (e.g. English)
Pseudocode
Real programming languages
Increasing precision
Ease of expression
Describe the ideas of an algorithm in nature language.Use pseudocode to clarify sufficiently tricky details of the algorithm.
![Page 34: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/34.jpg)
04/19/23 34
How to express algorithms?
Nature language (e.g. English)
Pseudocode
Real programming languages
Increasing precision
Ease of expression
To understand / describe an algorithm:Get the big idea first.Use pseudocode to clarify sufficiently tricky details
![Page 35: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/35.jpg)
04/19/23 35
Example: sorting
• Input: A sequence of N numbers a1…an
• Output: the permutation (reordering) of the input sequence such that a1 ≤ a2 … ≤ an.
• Possible algorithms you’ve learned so far– Insertion, selection, bubble, quick, merge, …– More in this course
• We seek algorithms that are both correct and efficient
![Page 36: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/36.jpg)
04/19/23 36
Insertion Sort
InsertionSort(A, n) {for j = 2 to n {
}
}1 j
sorted
1. Find position i in A[1..j-1] such that A[i] ≤ A[j] < A[i+1]2. Insert A[j] between A[i] and A[i+1]
▷ Pre condition: A[1..j-1] is sorted
▷ Post condition: A[1..j] is sorted
![Page 37: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/37.jpg)
04/19/23 37
InsertionSort(A, n) {for j = 2 to n {
key = A[j];i = j - 1;while (i > 0) and (A[i] > key)
{A[i+1] = A[i];i = i – 1;
}A[i+1] = key
}}
Insertion Sort
1 i j
Keysorted
![Page 38: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/38.jpg)
04/19/23 38
Correctness
• What makes a sorting algorithm correct?– In the output sequence, the elements are
ordered non-decreasingly– Each element in the input sequence has a
unique appearance in the output sequence• [2 3 1] => [1 2 2] X• [2 2 3 1] => [1 1 2 3] X
![Page 39: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/39.jpg)
04/19/23 39
Correctness
• For any algorithm, we must prove that it always returns the desired output for all legal instances of the problem.
• For sorting, this means even if (1) the input is already sorted, or (2) it contains repeated elements.
• Algorithm correctness is NOT obvious in some problems (e.g., optimization)
![Page 40: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/40.jpg)
04/19/23 40
How to prove correctness?
• Given a concrete input, eg. <4,2,6,1,7>trace it and prove that it works.
• Given an abstract input, eg. <a1, … an> trace it and prove that it works.
• Sometimes it is easier to find a counterexample to show that an algorithm does NOT work.– Think about all small examples– Think about examples with extremes of big and small– Think about examples with ties– Failure to find a counterexample does NOT mean that
the algorithm is correct
![Page 41: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/41.jpg)
04/19/23 41
An Example: Insertion SortInsertionSort(A, n) {for j = 2 to n {
key = A[j];i = j - 1;▷Insert A[j] into the sorted sequence A[1..j-1]
while (i > 0) and (A[i] > key) {A[i+1] = A[i];i = i – 1;
}A[i+1] = key
}} 1 i j
Keysorted
![Page 42: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/42.jpg)
04/19/23 42
Example of insertion sort
5 2 4 6 1 3
2 5 4 6 1 3
2 4 5 6 1 3
2 4 5 6 1 3
1 2 4 5 6 3
1 2 3 4 5 6 Done!
![Page 43: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/43.jpg)
04/19/23 43
Use loop invariants to prove the correctness of loops
• A loop invariant (LI) is a formal statement about the variables in your program which holds true throughout the loop
• Claim: at the start of each iteration of the for loop, the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order.
• Proof by induction– Initialization: the LI is true prior to the 1st iteration– Maintenance: if the LI is true before the jth iteration, it remains
true before the (j+1)th iteration– Termination: when the loop terminates, the LI gives us a useful
property to show that the algorithm is correct
![Page 44: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/44.jpg)
04/19/23 44
Prove correctness using loop invariantsInsertionSort(A, n) {for j = 2 to n {
key = A[j];i = j - 1;▷Insert A[j] into the sorted sequence A[1..j-1]
while (i > 0) and (A[i] > key) {A[i+1] = A[i];i = i – 1;
}A[i+1] = key
}}
Loop invariant: at the start of each iteration of the for loop, the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order.
![Page 45: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/45.jpg)
04/19/23 45
InitializationInsertionSort(A, n) {for j = 2 to n {
key = A[j];i = j - 1;▷Insert A[j] into the sorted sequence A[1..j-1]
while (i > 0) and (A[i] > key) {A[i+1] = A[i];i = i – 1;
}A[i+1] = key
}}
Subarray A[1] is sorted. So loop invariant is true before the
loop starts.
Loop invariant: at the start of each iteration of the for loop, the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order.
![Page 46: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/46.jpg)
04/19/23 46
MaintenanceInsertionSort(A, n) {for j = 2 to n {
key = A[j];i = j - 1;▷Insert A[j] into the sorted sequence A[1..j-1]
while (i > 0) and (A[i] > key) {A[i+1] = A[i];i = i – 1;
}A[i+1] = key
}}
Assume loop variant is true prior to iteration j
1 i j
Keysorted
Loop variant will be true before iteration j+1
Loop invariant: at the start of each iteration of the for loop, the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order.
![Page 47: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/47.jpg)
04/19/23 47
TerminationInsertionSort(A, n) {for j = 2 to n {
key = A[j];i = j - 1;▷Insert A[j] into the sorted sequence A[1..j-1]
while (i > 0) and (A[i] > key) {A[i+1] = A[i];i = i – 1;
}A[i+1] = key
}} 1 j=n+1
Sorted
Upon termination, A[1..n] contains all the original
elements of A in sorted order.
n
The algorithm is correct!
Loop invariant: at the start of each iteration of the for loop, the subarray A[1..j-1] consists of the elements originally in A[1..j-1] but in sorted order.
![Page 48: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/48.jpg)
04/19/23 48
Efficiency
• Correctness alone is not sufficient• Brute-force algorithms exist for most
problems• To sort n numbers, we can enumerate all
permutations of these numbers and test which permutation has the correct order– Why cannot we do this?– Too slow! – By what standard?
![Page 49: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/49.jpg)
04/19/23 49
How to measure complexity?
• Accurate running time is not a good measure
• It depends on input• It depends on the machine you used and
who implemented the algorithm• It depends on the weather, maybe
• We would like to have an analysis that does not depend on those factors
![Page 50: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/50.jpg)
04/19/23 50
Machine-independent
• A generic uniprocessor random-access machine (RAM) model– No concurrent operations– Each simple operation (e.g. +, -, =, *, if, for)
takes 1 step.• Loops and subroutine calls are not simple
operations.
– All memory equally expensive to access• Constant word size• Unless we are explicitly manipulating bits
![Page 51: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/51.jpg)
04/19/23 51
Running Time
• Number of primitive steps that are executed– Except for time of executing a function call
most statements roughly require the same amount of time
• y = m * x + b• c = 5 / 9 * (t - 32 )• z = f(x) + g(x)
• We can be more exact if need be
![Page 52: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/52.jpg)
04/19/23 52
Asymptotic Analysis
• Running time depends on the size of the input– Larger array takes more time to sort– T(n): the time taken on input with size n– Look at growth of T(n) as n→∞.
“Asymptotic Analysis”• Size of input is generally defined as the
number of input elements– In some cases may be tricky
![Page 53: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/53.jpg)
04/19/23 53
Running time of insertion sort
• The running time depends on the input: an already sorted sequence is easier to sort.
• Parameterize the running time by the size of the input, since short sequences are easier to sort than long ones.
• Generally, we seek upper bounds on the running time, because everybody likes a guarantee.
![Page 54: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/54.jpg)
04/19/23 54
Kinds of analyses
• Worst case– Provides an upper bound on running time– An absolute guarantee
• Best case – not very useful• Average case
– Provides the expected running time– Very useful, but treat with care: what is “average”?
• Random (equally likely) inputs• Real-life inputs
![Page 55: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/55.jpg)
04/19/23 55
Analysis of insertion Sort
InsertionSort(A, n) {for j = 2 to n {
key = A[j]i = j - 1;while (i > 0) and (A[i] > key) {
A[i+1] = A[i]i = i - 1
}A[i+1] = key
}
}
How many times will this line execute?
![Page 56: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/56.jpg)
04/19/23 56
Analysis of insertion Sort
InsertionSort(A, n) {for j = 2 to n {
key = A[j]i = j - 1;while (i > 0) and (A[i] > key) {
A[i+1] = A[i]i = i - 1
}A[i+1] = key
}
}
How many times will this line execute?
![Page 57: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/57.jpg)
04/19/23 57
Analysis of insertion SortStatement cost time__
InsertionSort(A, n) {
for j = 2 to n { c1 n
key = A[j] c2 (n-1)
i = j - 1; c3 (n-1)
while (i > 0) and (A[i] > key) { c4 S
A[i+1] = A[i] c5 (S-(n-1))
i = i - 1 c6 (S-(n-1))
} 0
A[i+1] = key c7 (n-1)
} 0
}
S = t2 + t3 + … + tn where tj is number of while expression evaluations for the jth for loop iteration
![Page 58: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/58.jpg)
04/19/23 58
Analyzing Insertion Sort• T(n) = c1n + c2(n-1) + c3(n-1) + c4S + c5(S - (n-1)) + c6(S - (n-1)) + c7(n-1)
= c8S + c9n + c10
• What can S be?– Best case -- inner loop body never executed
• tj = 1 S = n - 1 • T(n) = an + b is a linear function
– Worst case -- inner loop body executed for all previous elements
• tj = j S = 2 + 3 + … + n = n(n+1)/2 - 1• T(n) = an2 + bn + c is a quadratic function
– Average case• Can assume that on average, we have to insert A[j] into the
middle of A[1..j-1], so tj = j/2• S ≈ n(n+1)/4• T(n) is still a quadratic function
![Page 59: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/59.jpg)
04/19/23 59
Asymptotic Analysis
• Ignore actual and abstract statement costs• Order of growth is the interesting measure:
– Highest-order term is what counts• As the input size grows larger it is the high order term that
dominates
0 50 100 150 2000
1
2
3
4x 10
4
n
T(n
)
100 * n
n2
![Page 60: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/60.jpg)
04/19/23 60
Comparison of functions
log2n n nlog2n n2 n3 2n n!
10 3.3 10 33 102 103 103 106
102 6.6 102 660 104 106 1030 10158
103 10 103 104 106 109
104 13 104 105 108 1012
105 17 105 106 1010 1015
106 20 106 107 1012 1018
For a super computer that does 1 trillion operations per second, it will be longer than 1 billion years
![Page 61: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/61.jpg)
04/19/23 61
Order of growth
1 << log2n << n << nlog2n << n2 << n3 << 2n << n!
(We are slightly abusing of the “<<“ sign. It means a smaller order of growth).
![Page 62: 10/1/20151 CS 3343: Analysis of Algorithms Lecture 1: Introduction Some slides courtesy from Jeff Edmonds @ York University](https://reader034.vdocuments.site/reader034/viewer/2022042822/56649e755503460f94b75911/html5/thumbnails/62.jpg)
04/19/23 62
Asymptotic notations
• We say InsertionSort’s worst-case running time is Θ(n2)– Properly we should say running time is in
Θ(n2)– It is also in O(n2 )– What’s the relationship between Θ and O?
• Formal definition next time