lecture 1 preliminaries
DESCRIPTION
Lecture 1 Preliminaries. COMP 523: Advanced Algorithmic Techniques Lecturer: Dariusz Kowalski. Announcements. Textbook: J. Kleinberg, E. Tardos. Algorithm Design . Addison-Wesley, 2005 Lectures: Wednesday 9-11, Friday 15-16 in 3.10 Tutorials: Thursday 11-12 in 1.01 Ashton Bldg - PowerPoint PPT PresentationTRANSCRIPT
Lecture 1: Preliminaries 1
Lecture 1Preliminaries
COMP 523: Advanced Algorithmic Techniques
Lecturer: Dariusz Kowalski
Lecture 1: Preliminaries 2
Announcements• Textbook: J. Kleinberg, E. Tardos. Algorithm
Design. Addison-Wesley, 2005• Lectures: Monday 13-15 in BROD-405, Thursday
12-13 in BROD-405, Friday 10-11 in MATH-105• Tutorials: Monday 15-16 in ELEC-204• Office hours (room 3.11, Ashton Bldg):
– Monday 15-16– Friday 11-13
• Info and contact:– http://www.csc.liv.ac.uk/~darek/comp523.html– [email protected]
Assessment and feedback
• Assessment: Exam (75%) and 2 tasks (25% total)
• Feedback: – Presentation and discussion of solutions after
assignment deadline– Short individual feedback after assignments, by
email– In case of specific questions, individual
appointments
Lecture 1: Preliminaries 3
Lecture 1: Preliminaries 4
Algorithms - what does it mean?Algorithm - a list/structure of instructions, which are carried
out in a fixed order – to find the answer to a question– to calculate
Al-Khwarizmi (Muhammed Ibn Musa) - Arabian mathematician who introduced the decimal positional number system to the Western world, the author of Algebra
This module: techniques to design algorithms for problems concerning data commonly processed by computers (written in hard discs, propagated in networks, entered by user or external device, etc.)
Lecture 1: Preliminaries 5
Types of algorithms
• Constructive vs. Non-constructive
• Discrete vs. Numerical
• Deterministic vs. non-deterministic (e.g., Randomized, Quantum)
• Sequential vs. Concurrent vs. DNA vs. …
• Exact vs. Approximate
• and many others
Lecture 1: Preliminaries 6
Example
Problem: Find if a given word occurs in a given text
Input: Text and word represented as lists
Output: Word found or not
o n c e u p o n a t
o n a
Lecture 1: Preliminaries 7
Example - algorithmNaïve solution (exhaustive search):Intuition:• Check letter after letter, starting from the beginning of the lists, if the
corresponding letters are equal• If some corresponding letters are not equal, re-start comparing from the second
letter of the text (and the first letter of the word)• Etc. from the 3rd letter, 4th letter, until the end of text or successful comparison
Implementation:• Initiate three pointers:
– blue_pointer and black_pointer at the beginning of blue_list – yellow_pointer at the beginning of yellow_list
• Initiate Boolean variable successful into false
Lecture 1: Preliminaries 8
Naïve solution cont.• Repeat
– stop := false– Move blue_pointer to the next element – Set black_pointer to blue_pointer– Move yellow_pointer to the first element of yellow_list– Repeat
• Move black_pointer and yellow_pointer to the next elements in corresponding lists
• If values pointed by black_pointer and yellow_pointer are different then stop := true
until yellow_pointer or black_pointer is at the end of its list or stop– If yellow_pointer is at the end of yellow_list and not stop then
successful := true
until successful or blue_pointer is at the end of blue_listOutput: successful
Lecture 1: Preliminaries 9
Example - how it works
o n c e u p o n a t
o n a
• Repeat• stop := false, • move blue_pointer to the next element, set black_pointer to blue_pointer, • move yellow_pointer to the first element of yellow_list– Repeat
• Move black_pointer and yellow_pointer to the next elements in corresponding lists• If values pointed by black_pointer and yellow_pointer are different then stop := true
until yellow_pointer or black_pointer is at the end of its list or stop– If yellow_pointer is at the end of yellow_list and not stop then successful := true
until successful or blue_pointer is at the end of blue_list
Lecture 1: Preliminaries 10
What we expect from algorithms?• Correctness: computes desired output• Termination: stops eventually (or with high probability)• Efficiency: with respect to
– Performance measure• Time (or total number of computation steps)• Size of memory used• Number of messages sent• …
– Methods of measuring• Worst-case• Average-case• Smoothed (a subset of possible inputs, specific distribution of inputs, etc.)• Competitive (comparing to the best solution for particular input)• Expected• …
Analysis depends on
Model!
Lecture 1: Preliminaries 11
Example - correctness
• Successful is set to true iff after some execution of the internal loop the yellow_pointer is at the end of the yellow_list
• yellow_pointer is at the end of the yellow_list iff it came through the whole yellow_list without coming back to the beginning
• it happens if the values pointed by black_pointer and yellow_pointer have been the same during all checks of internal loop in the current run (thus a copy of the word exists in the text)
Repeat• stop := false, • move blue_pointer to the next element,
set black_pointer to blue_pointer, • move yellow_pointer to the first
element of yellow_list– Repeat
• Move black_pointer and yellow_pointer to the next elements in corresponding lists
• If values pointed by black_pointer and yellow_pointer are different then stop := true
until yellow_pointer or black_pointer is at the end of its list or stop
– If yellow_pointer is at the end of yellow_list and not stop then successful := true
until successful or blue_pointer is at the end of blue_list
Lecture 1: Preliminaries 12
Example - termination• External Loop Invariant:
At the beginning of the external loop, blue_pointer advances, and it has a finite number of advances to take before reaching the end of the text
• Internal Loop Invariant:
Each run of the internal loop finishes eventually;
It happens because yellow_pointer keeps advancing every iteration (first line of the loop), unless stop condition becomes true or it reaches the end of the word (or black_ pointer reaches the end of the text)
Repeat• stop := false, • move blue_pointer to the next element,
set black_pointer to blue_pointer, • move yellow_pointer to the first
element of yellow_list– Repeat
• Move black_pointer and yellow_pointer to the next elements in corresponding lists
• If values pointed by black_pointer and yellow_pointer are different then stop := true
until yellow_pointer or black_pointer is at the end of its list or stop
– If yellow_pointer is at the end of yellow_list and not stop then successful := true
until successful or blue_pointer is at the end of blue_list
Lecture 1: Preliminaries 13
Example - efficiency
• Complexity measures: time, size of additional memory• Size of additional memory:
– 3 single pointers and 2 binary variables
• Time complexity:– Worst-case: if the sequence is not included in the text then
time is (almost) proportional to the multiplication of sizes of blue and yellow lists, e.g.,
text: ‘aab’ repeated n times
word: aaa
1, 2 or 3 internal loop runs, 3n times, gives about 6n runs
Lecture 1: Preliminaries 14
Example - efficiency cont.
• Time complexity (cont.):– Average case: text or sequence are randomly selected;
average short word should be find quickly in the beginning of an average long text, in time proportional to the squared length of the word (exercise)
– Smoothed analysis: text must be randomly selected from “reasonable” set of texts (usually complicated analysis, depends on the family of texts - dictionary)
Lecture 1: Preliminaries 15
Efficiency - asymptotic notation• n - integer, size of data, f() - function over integers• Complexity O(f(n)) : there is a positive constant c such
that for every positive integer n complexity is at most cf(n)
• Complexity (f(n)) : there is a positive constant c such that for every positive integer n complexity is at least cf(n)
• Complexity o(f(n)) : complexity divided by f(n) comes to 0 with n going to infinity (strictly smaller than f(n))
• Complexity (f(n)) : complexity divided by f(n) comes to with n going to infinity (strictly bigger than f(n))
Lecture 1: Preliminaries 16
Example - asymptotic complexity
• Worst case time: – If size of text is n and size of word is n/2 :
O((n/2)(n/2)) = O(n2) – If size of text is m and size of sequence is k :
O((m-k)k)
Lecture 1: Preliminaries 17
Examples
5n3+100 = O(n3) , 5n3+100 ≠ O(n2)5n3+100 = (n3) , 5n3+100 ≠ (n4)log n = o(na) for any positive constant ana = o(cn) for any positive constants a,clog (4n) = log n + log 4 = O(log n)log (n4)= 4 log n = O(log n)(4n)3 = 64n3 = O(n3)(n4)3 = n12 = (n4) (3n)4 = 81n = (3n)Logarithms are to the base 2
Lecture 1: Preliminaries 18
Symmetric properties and tight bound
If f(n) = O(g(n)) then g(n) = (f(n)) and vice versa
If f(n) = o(g(n)) then g(n) = (f(n)) and vice versa
Definition:
If f(n) = O(g(n)) and f(n) = (g(n)) then
f(n) = (g(n)) and g(n) = (f(n))
Example:
9n2 + n +7 = (3n2) = (n2)
Lecture 1: Preliminaries 19
Transitive properties (order)
If f(n) = O(g(n)) and g(n) = O(h(n)) then
f(n) = O(h(n))
If f(n) = (g(n)) and g(n) = (h(n)) then
f(n) = (h(n))
If f(n) = (g(n)) and g(n) = (h(n)) then
f(n) = (h(n))
Lecture 1: Preliminaries 20
Sum and maximum
f1(n) + … + fa(n) = (max(f1(n), … , fa(n)))
for any constant positive integer a
Example:
• 3n2 + n +7 = (3n2) = (n2)
If the range of the summation index is not constant:
in i = n(n+1)/2 = (n2)
(max{1, … , n}) = (n)
Lecture 1: Preliminaries 21
Logarithmic time O(log n)
• Reduce input to any fraction of it in constant time
• Example: searching if a given element x is in a sorted array
Lecture 1: Preliminaries 22
Linear time O(n)
• It is sufficient to look at each element of the input constant number of times
• Example: finding maximum in an (unsorted) array
Lecture 1: Preliminaries 23
Time O(n log n)
• Split the input into two pieces of the similar size and merge both solutions in linear time
• Example: sorting an array (split into halves, sort each part separately, merge sorted parts in linear time)
Lecture 1: Preliminaries 24
Quadratic time O(n2)
• It is enough to consider pairs of elements
• Examples: – finding the closest pair of points located in a
plane– sorting by comparison of subsequent elements
(insertion sort)
Lecture 1: Preliminaries 25
Polynomial time O(na)
• Algorithm has many nested loops
• Example: are there any two disjoint sets among given family of n sets, each of at most n elements (time O(n3))
Lecture 1: Preliminaries 26
Exponential time O(cn)
• Algorithm considers many subsets of the input
• Example: exhaustive search to find the largest clique contained in a given graph (time O(2n))
Lecture 1: Preliminaries 27
Beyond exponential time: O(n!), O(nn), …
• Algorithm searching in a large space
• Example: search performed in the set of all permutations (time O(n!))
Lecture 1: Preliminaries 28
Graphs• Set of nodes |V| = n• Set of edges |E| = m
– Undirected edges/graph: pairs of nodes {v,w}– Directed edges/graph: pairs of nodes (v,w)
• Set of neighbors of v : set of nodes connected by an edge with v (directed: in-neighbors, out-neighbors)
• Degree of a node: number of its neighbors• Path: a sequence of (non-repeating) nodes such that every
two consecutive nodes constitute an edge• Length of a path: number of nodes minus 1• Distance between two nodes: the length of the shortest
path between these nodes• Diameter: the longest distance in the graph
Lecture 1: Preliminaries 29
Lines, cycles, trees, cliques
Line Cycle
Clique
Tree
Lecture 1: Preliminaries 30
Conclusions
• Algorithm: list/structure of instructions
• Algorithmic methods for problems arising from computer and communication applications
• Guarantee correctness, termination and efficiency– Model is important!
• Asymptotic analysis of efficiency
Lecture 1: Preliminaries 31
Textbook and exercises
READING:• Chapter 2, up to section 2.4
OBLIGATORY EXERCISES: • Exercises 1,2,3 page 67 • Solved exercises 1,2 pages 65,66
OPTIONAL:• Exercises 3,4,5,6 pages 67,68,69• Exercises 7,8 pages 69,70