big-oh and other notations in algorithm analysis
DESCRIPTION
BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS. Lydia Sinapova, Simpson College. 1. Mark Allen Weiss: Data Structures and Algorithm Analysis in Java. Big-Oh and Other Notations in Algorithm Analysis. Classifying Functions by Their Asymptotic Growth Theta, Little oh, Little omega - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/1.jpg)
BIG-OH AND OTHER NOTATIONS IN
ALGORITHM ANALYSIS
Lydia Sinapova, Simpson College
Mark Allen Weiss: Data Structures and Algorithm Analysis in Java
1
![Page 2: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/2.jpg)
Big-Oh and Other Notations in Algorithm Analysis
• Classifying Functions by Their Asymptotic Growth
• Theta, Little oh, Little omega• Big Oh, Big Omega• Rules to manipulate Big-Oh
expressions• Typical Growth Rates
2
![Page 3: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/3.jpg)
Classifying Functions by Their Asymptotic Growth
Asymptotic growth : The rate of growth of a function
Given a particular differentiable function f(n), all other differentiable functions fall into three classes:
.growing with the same rate
.growing faster
.growing slower
3
![Page 4: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/4.jpg)
f(n) and g(n) have same rate of growth, if
lim( f(n) / g(n) ) = c, 0 < c < ∞, n -> ∞
Notation: f(n) = Θ( g(n) ) pronounced "theta"
Theta4
![Page 5: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/5.jpg)
f(n) grows slower than g(n) (or g(n) grows faster than f(n)) if
lim( f(n) / g(n) ) = 0, n → ∞
Notation: f(n) = o( g(n) ) pronounced "little oh"
Little oh5
![Page 6: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/6.jpg)
f(n) grows faster than g(n)
(or g(n) grows slower than f(n)) if
lim( f(n) / g(n) ) = ∞, n -> ∞
Notation: f(n) = ω (g(n)) pronounced "little omega"
Little omega6
![Page 7: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/7.jpg)
if g(n) = o( f(n) )
then f(n) = ω( g(n) )
Examples: Compare n and n2
lim( n/n2 ) = 0, n → ∞, n = o(n2)lim( n2/n ) = ∞, n → ∞, n2 = ω(n)
Little omega and Little oh7
![Page 8: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/8.jpg)
Theta: Relation of Equivalence
R: "having the same rate of growth":relation of equivalence, gives a partition over the set of all differentiable functions - classes of equivalence.
Functions in one and the same class are equivalent with respect to their growth.
8
![Page 9: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/9.jpg)
Algorithms with Same Complexity
Two algorithms have same complexity, if the functions representing the number of operations have same rate of growth.
Among all functions with same rate of growth we choose the simplest one to represent the complexity.
9
![Page 10: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/10.jpg)
Examples
Compare n and (n+1)/2
lim( n / ((n+1)/2 )) = 2,
same rate of growth
(n+1)/2 = Θ(n)
- rate of growth of a linear function
10
![Page 11: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/11.jpg)
Examples
Compare n2 and n2+ 6n
lim( n2 / (n2+ 6n ) )= 1
same rate of growth.
n2+6n = Θ(n2) rate of growth of a quadratic function
11
![Page 12: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/12.jpg)
Examples
Compare log n and log n2
lim( log n / log n2 ) = 1/2
same rate of growth.
log n2 = Θ(log n) logarithmic rate of growth
12
![Page 13: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/13.jpg)
Θ(n3): n3
5n3+ 4n
105n3+ 4n2 + 6n
Θ(n2): n2
5n2+ 4n + 6
n2 + 5
Θ(log n): log n
log n2
log (n + n3)
Examples
13
![Page 14: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/14.jpg)
Comparing Functions
• same rate of growth: g(n) = Θ(f(n))• different rate of growth: either g(n) = o (f(n))
g(n) grows slower than f(n), and hence f(n) = ω(g(n))
or g(n) = ω (f(n)) g(n) grows faster than f(n), and hence f(n) = o(g(n))
14
![Page 15: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/15.jpg)
f(n) = O(g(n))
if f(n) grows with
same rate or slower than g(n).
f(n) = Θ(g(n)) or f(n) = o(g(n))
The Big-Oh Notation15
![Page 16: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/16.jpg)
n+5 = Θ(n) = O(n) = O(n2) = O(n3) = O(n5)
the closest estimation: n+5 = Θ(n)the general practice is to use the Big-Oh notation: n+5 = O(n)
Example
16
![Page 17: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/17.jpg)
The inverse of Big-Oh is Ω
If g(n) = O(f(n)), then f(n) = Ω (g(n))
f(n) grows faster or with the same rate as g(n): f(n) = Ω (g(n))
The Big-Omega Notation17
![Page 18: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/18.jpg)
Rules to manipulateBig-Oh expressions
Rule 1:a. If
T1(N) = O(f(N))and
T2(N) = O(g(N)) then T1(N) + T2(N) = max( O( f (N) ), O( g(N) ) )
18
![Page 19: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/19.jpg)
Rules to manipulateBig-Oh expressions
b. If T1(N) = O( f(N) )
and T2(N) = O( g(N) )
then
T1(N) * T2(N) = O( f(N)* g(N) )
19
![Page 20: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/20.jpg)
Rules to manipulateBig-Oh expressions
Rule 2:
If T(N) is a polynomial of degree k, then
T(N) = Θ( Nk )
Rule 3:
log k N = O(N) for any constant k.
20
![Page 21: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/21.jpg)
Examples
n2 + n = O(n2)we disregard any lower-order term
nlog(n) = O(nlog(n))
n2 + nlog(n) = O(n2)
21
![Page 22: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/22.jpg)
C constant, we write O(1)logN logarithmiclog2N log-squaredN linearNlogNN2 quadraticN3 cubic2N exponentialN! factorial
Typical Growth Rates22
![Page 23: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/23.jpg)
Problems
N2 = O(N2) true2N = O(N2) trueN = O(N2) true
N2 = O(N) false2N = O(N) trueN = O(N) true
23
![Page 24: BIG-OH AND OTHER NOTATIONS IN ALGORITHM ANALYSIS](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813ae2550346895da3387b/html5/thumbnails/24.jpg)
Problems
N2 = Θ (N2)true2N = Θ (N2) falseN = Θ (N2) false
N2 = Θ (N) false2N = Θ (N) trueN = Θ (N) true
24