Download - Introduction to Algorithms
![Page 1: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/1.jpg)
Introduction to Algorithms
Rabie A. [email protected]
http://www. rabieramadan.org
2Some of the sides are exported from different sources to clarify the
topic
![Page 2: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/2.jpg)
Importance of algorithms
Algorithms are used in every aspect in our life.
Let’s take an Example ……….
![Page 3: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/3.jpg)
Example
Suppose you are implementing a spreadsheet program, in which you must maintain a grid of cells. Some cells of the spreadsheet contain numbers, but other cells contain expressions that depend on other cells for their value. However, the expressions are not allowed to form a cycle of dependencies: for example, if the expression in cell E1 depends on the value of cell A5, and the expression in cell A5 depends on the value of cell C2, then C2 must not depend on E1.
![Page 4: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/4.jpg)
Example
Describe an algorithm for making sure that no cycle of dependencies exists (or finding one and complaining to the spreadsheet user if it does exist).
If the spreadsheet changes, all its expressions may need to
be recalculated. Describe an efficient method for sorting the expression evaluations, so that each cell is recalculated only after the cells it depends on have been recalculated.
![Page 5: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/5.jpg)
Another Example
Order the following items in a food chain
fish
human
shrimp
sheep
wheatplankton
tiger
![Page 6: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/6.jpg)
Solving Topological Sorting Problem
Solution: Verify whether a given digraph is a dag and, if it is, produce an ordering of vertices.
Two algorithms for solving the problem. They may give different (alternative) solutions.
DFS-based algorithm• Perform DFS traversal and note the order in which vertices become dead
ends (that is, are popped of the traversal stack).
• Reversing this order yields the desired solution, provided that no back edge has been encountered during the traversal.
![Page 7: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/7.jpg)
Example
Complexity: as DFS
![Page 8: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/8.jpg)
Solving Topological Sorting Problem
Source removal algorithm
• Identify a source, which is a vertex with no incoming edges and delete it along with all edges outgoing from it.
• There must be at least one source to have the problem solved.
• Repeat this process in a remaining diagraph.
• The order in which the vertices are deleted yields the desired solution.
![Page 9: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/9.jpg)
Example
![Page 10: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/10.jpg)
Source removal algorithm Efficiency
Efficiency: same as efficiency of the DFS-based algorithm, but how would you identify a source?
A big Problem
![Page 11: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/11.jpg)
Analysis of algorithms
Issues:• Correctness
• space efficiency
• time efficiency
• optimality
Approaches: • theoretical analysis
• empirical analysis
![Page 12: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/12.jpg)
Space Analysis
When considering space complexity, algorithms are divided into those that need extra space to do their work and those that work in place.
Space analysis would examine all of the data being stored to see if there were more efficient ways to store it.
Example : As a developer, how do you store the real numbers ? • Suppose we are storing a real number that has only one place of precision
after the decimal point and ranges between -10 and +10.
• How many bytes you need ?
![Page 13: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/13.jpg)
Space Analysis
Example : As a developer, how do you store the real numbers ? • Suppose we are storing a real number that has only one place of precision
after the decimal point and ranges between -10 and +10.
• How many bytes you need ?
Most computers will use between 4 and 8 bytes of memory. If we first multiply the value by 10. We can then store this as an integer
between -100 and +100. This needs only 1 byte, a savings of 3 to 7 bytes. A program that stores 1000 of these values can save 3000 to 7000 bytes.
It makes a big difference when programming mobile or PDAs or when you have large input .
![Page 14: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/14.jpg)
Theoretical analysis of time efficiencyTime efficiency is analyzed by determining the number of repetitions of the
basic operation as a function of input size
Basic operation: the operation that contributes the most towards the running time of the algorithm
T(n) ≈ copC(n)
running time execution timefor basic operationor cost
Number of times basic operation is executed
input size
Note: Different basic operations may cost differently!
![Page 15: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/15.jpg)
Why Input Classes are Important? Input determines what the path of execution through an
algorithm will be. If we are interested in finding the largest value in a list of N
numbers, we can use the following algorithm:
![Page 16: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/16.jpg)
Why Input Classes are Important?
If the list is in decreasing order,• There will only be one assignment done before the loop starts.
If the list is in increasing order, • There will be N assignments (one before the loop starts and N -1 inside the
loop). Our analysis must consider more than one possible set of input, because Our analysis must consider more than one possible set of input, because
if we only look at one set of input, it may be the set that is solved the if we only look at one set of input, it may be the set that is solved the fastest (or slowest).fastest (or slowest).
![Page 17: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/17.jpg)
Input size and basic operation examples
ProblemProblem Input size measureInput size measure Basic operationBasic operation
Searching for key in a list Searching for key in a list of of nn items items
Number of list’s items, i.e. Number of list’s items, i.e. nn
Key comparisonKey comparison
Multiplication of two Multiplication of two matricesmatrices
Matrix dimensions or total Matrix dimensions or total number of elementsnumber of elements
Multiplication of two Multiplication of two numbersnumbers
Checking primality of a Checking primality of a given integer given integer nn
n’n’size = number of digits (in size = number of digits (in binary representation)binary representation)
DivisionDivision
Typical graph problemTypical graph problem #vertices and/or edges#vertices and/or edgesVisiting a vertex or Visiting a vertex or traversing an edgetraversing an edge
![Page 18: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/18.jpg)
Importance of the analysis It gives an idea about how fast the algorithm
If the number of basic operations
C(n) = ½ n (n-1) = ½ n2 – ½ n ≈ ½ n2
How much longer if the algorithm doubles its input size?
Increasing input size increases the complexity We tend to omit the constants because they have no effect with large inputs Everything is based on estimation
T(n) ≈ copC(n)
4)(
2
1
)2(2
1
)(
)2(
2
2
)()2(
n
n
nCC
nCC
OP
OPnTnT
![Page 19: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/19.jpg)
Empirical analysis of time efficiency Select a specific (typical) sample of inputs
Use physical unit of time (e.g., milliseconds)
or
Count actual number of basic operation’s executions
Analyze the empirical/experimental data
![Page 20: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/20.jpg)
Cases to consider in Analysis Best-case, average-case, worst-case
For some algorithms, efficiency depends on form of input:
Worst case: Cworst(n) – maximum over inputs of size n
Best case: Cbest(n) – minimum over inputs of size n
Average case: Cavg(n) – “average” over inputs of size n
• The toughest to do
![Page 21: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/21.jpg)
Best-case, average-case, worst-case Average case: Cavg(n) – “average” over inputs of size n
• Determine the number of different groups into which all possible input sets can be divided.
• Determine the probability that the input will come from each of these groups.
• Determine how long the algorithm will run for each of these groups.
n is the size of the input,m is the number of groups, pi is the probability that the input will be from group i, ti is the time that the algorithm takes for input from group i.
![Page 22: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/22.jpg)
Example: Sequential search
Worst case
Best case
Average case
n key comparisons
1 comparison
(n+1)/2, assuming K is in A
![Page 23: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/23.jpg)
Computing the Average Case for the Sequential search
Neither the Worst nor the Best case gives the yield to the actual performance of an algorithm with random input.
The Average Case does Assume that:
• The probability of successful search is equal to p(0≤ p ≤1)
• The probability of the first match occurring in the ith position is the same for every i .
• The probability of a match occurs at ith position is p/n for every i
• In the case of unsuccessful search , the number of comparison is n with probability (1-p).
)1(2
)1()1.(
2
)1(.
)1.(]...321[
)1.(]....3.2.1[)(
pnnp
pnnn
n
p
pnnn
p
pnn
pn
n
p
n
p
n
pnCavg
![Page 24: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/24.jpg)
Computing the Average Case for the Sequential search
If p =1 If p =1 (I found the key k)
• The average number of comparisons is (n+1)/2 (n+1)/2 If p=0 If p=0
• The average number of key comparisons is nn
The average Case is more difficult than the Best and Worst cases
)1(2
)1()1.(
2
)1(.
)1.(]...321[
)1.(]....3.2.1[)(
pnnp
pnnn
n
p
pnnn
p
pnn
pn
n
p
n
p
n
pnCavg
![Page 25: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/25.jpg)
25
Mathematical Background
![Page 26: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/26.jpg)
Mathematical Background Logarithms
![Page 27: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/27.jpg)
Logarithms Which Base ?
Loga n = Loga b Logb n
Loga n = c Logb n
In terms of complexity , we tend to ignore the constant
![Page 28: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/28.jpg)
Mathematical Background
![Page 29: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/29.jpg)
Mathematical Background
![Page 30: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/30.jpg)
Mathematical Background
![Page 31: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/31.jpg)
Types of formulas for basic operation’s count
Exact formula
e.g., C(n) = n(n-1)/2
Formula indicating order of growth with specific multiplicative constant
e.g., C(n) ≈ 0.5 n2
Formula indicating order of growth with unknown multiplicative constant
e.g., C(n) ≈ cn2
![Page 32: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/32.jpg)
32
Order of growth
![Page 33: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/33.jpg)
Order of growth Of greater concern is the rate of increase in
operations for an algorithm to solve a problem as the size of the problem increases.
This is referred to as the rate of growth of the algorithm.
![Page 34: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/34.jpg)
Order of growth The function based on x2 increases
slowly at first, but as the problem size gets larger, it begins to grow at a rapid rate.
The functions that are based on x both grow at a steady rate for the entire length of the graph.
The function based on log x seems to not grow at all, but this is because it is actually growing at a very slow rate.
![Page 35: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/35.jpg)
Values of some important functions as n
![Page 36: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/36.jpg)
Order of growth
Second point to consider : Because the faster growing functions increase at such a
significant rate, they quickly dominate the slower-growing functions.
This means that if we determine that an algorithm’s complexity is a combination of two of these classes, we will frequently ignore all but the fastest growing of these terms.
Example : if the complexity is
we tend to ignore 30x term
![Page 37: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/37.jpg)
Classification of GrowthAsymptotic order of growth
A way of comparing functions that ignores constant factors and small input sizes.
O(g(n)): class of functions f(n) that grow no faster than g(n)• All functions with smaller or the same order of growth as g(n)
Ω(g(n)): class of functions f(n) that grow at least as fast as g(n)• All functions that are larger or have the same order of growth as g(n)
Θ(g(n)): class of functions f(n) that grow at same rate as g(n)
• Set of functions that have the same order of growth as g(n)
)(),()1(5.0),(5100),( 23222 nOnnOnnnOnnOn
),(5100),()1(5.0),( 2223 nnnnnnn
)( 22 nbnan
![Page 38: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/38.jpg)
Big-oh•O(g(n)): class of functions t(n) that grow no faster than g(n)• if there exist some positive constant c and some nonnegative n0 such that
0)()( nnallforncgnt
)O(n5100n that Prove :Ex 2
5101
101n101n
5)n all(for n 100n 5100n
0
2
nandc
You may come up with different c and n0
![Page 39: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/39.jpg)
Big-omegaΩ(g(n)): class of functions t(n) that grow at least as fast as g(n)
?)(
)()(23
0
nnthatprove
nnallforncgnt
01
0
0
23
nandc
nallfornn
![Page 40: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/40.jpg)
Big-thetaΘ(g(n)): class of functions t(n) that grow at same rate as g(n)
02 )(1)()( nnallforngcntngc
021 ,, nandcc
gettoneedYou
![Page 41: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/41.jpg)
(g(n)), functions that grow at least as fast as g(n)
(g(n)), functions that grow at the same rate as g(n)
O(g(n)), functions that grow no faster than g(n)
g(n)
>=
<=
=
Summary
![Page 42: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/42.jpg)
Theorem If t1(n) O(g1(n)) and t2(n) O(g2(n)), then
t1(n) + t2(n) O(max{g1(n), g2(n)}).• The analogous assertions are true for the -notation and -
notation. Implication: The algorithm’s overall efficiency will be determined
by the part with a larger order of growth, i.e., its least efficient part.• For example, 5n2 + 3nlogn O(n2)
Proof. There exist constants c1, c2, n1, n2 such that
t1(n) t1(n) c1c1**g1(n), g1(n), for allfor all n n n1n1
t2(n) t2(n) c2c2**g2(n), g2(n), for allfor all n n n2n2
Define Define c3 = c1 + c2c3 = c1 + c2 and and n3 = max{n1,n2}. Thenn3 = max{n1,n2}. Then
t1(n) + t2(n) t1(n) + t2(n) c3* c3*max{g1(n), g2(n)}, max{g1(n), g2(n)}, for allfor all n n n3n3
![Page 43: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/43.jpg)
Some properties of asymptotic order of growth
f(n) O(f(n))
f(n) O(g(n)) iff g(n) (f(n))
If f (n) O(g (n)) and g(n) O(h(n)) , then f(n) O(h(n))
If f1(n) O(g1(n)) and f2(n) O(g2(n)) , then
f1(n) + f2(n) O(max{g1(n), g2(n)})
Also, 1in (f(i)) = (1in f(i))
![Page 44: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/44.jpg)
Orders of growth of some important functions
All logarithmic functions loga n belong to the same class (log n) no matter what the logarithm’s base a > 1 is
because
All polynomials of the same degree k belong to the same class: aknk + ak-1nk-1 + … + a0 (nk)
Exponential functions an have different orders of growth for different a’s
ann bba log/loglog
![Page 45: Introduction to Algorithms](https://reader035.vdocuments.site/reader035/viewer/2022070402/568137d3550346895d9f735d/html5/thumbnails/45.jpg)
Basic asymptotic efficiency classes11 constantconstant
log log nn logarithmiclogarithmic
nn linearlinear
n n log log nn n-n-loglog-n-n
nn22 quadraticquadratic
nn33 cubiccubic
22nn exponentialexponential
nn!! factorialfactorial