data structure: algorithm and analysis
TRANSCRIPT
DATA STRUCTURES ALGORITHM AND ANALYSIS
Rajdeep Chatterjee
Assistant Professor
School of Computer Engineering
KIIT University
OVERVIEW
Algorithm
Analysis of Algorithm
Space Complexity
Time Complexity
Step Counts
Asymptotic Notations
Big-oh Notations
Rate of Growth
Types of Time Complexity
Best Case Complexity
Worst Case Complexity
Average Case Complexity
ALGORITHM
Algorithm is a finite set of well defined
computational instructions written in a proper
sequence in order to solve a problem.
Criteria-
Input
Output
Definiteness
Finiteness
Effectiveness
EXAMPLES
ADD: INTEGER a, b, c
1. Read a & b
2. add a & b
3. store the sum of a & b to c
4. print c
In C, exp(1)
void add(int a, int b){
int c;
c=a+b;
printf(“%d”,c);
}
In C, exp(2)
int add(int a, int b){
int c;
c=a+b;
return c;
}
ANALYZING AN ALGORITHM
Why is it important?
Predict the feasibility requirement of your code
(algorithm).
Usual requirements
Execution time
Memory space
The complexity of the algorithm is determined in
terms of space and time.
Space complexity ← execution time
Time complexity ← memory space
SPACE COMPLEXITY
Amount of computer memory required during
program execution.
Instruction space – space required to store the
code.
Fixed space requirement – input independent
Variable space requirement – input dependent
S = F(I) otherwise S = C
EXAMPLES
Exp(1)
int main(){
printf(“KIIT UNIVERSITY”);
}
Exp(2)
void bubble(int a[], int n){
int i,j;
for(i=0;i<n-1;i++){
for(j=0;j<n-1-i;j++)
if(a[j]>a[j+1])
swap (a[j],a[j+1])
}
}
TIME COMPLEXITY
The amount of computer time that it needs to run
to completion.
It is determined without considering the following
information-
The machine we are executing,
Its machine language instruction set,
The time required by each machine instruction,
The translation, a compiler/interpreter will make from
the source code to machine language.
EXAMPLES
Exp(1)
x=x+1;
Exp(2)
for(i=1 ; i<=n ; i++)
x=x+1;
Exp(3)
for(i=1 ; i<=n ; i++)
for(j=1 ; j<=n ; j++)
x=x+1;
Our concern should be the order of magnitude/ growth of an algorithm
for an input n.
Exp(4)
int main(){
printf(“KIIT UNIVERSITY”);
}
STEP COUNT
Instructions / code Step Count
x=x+1 constant, c
int main(){
printf(“KIIT UNIVERSITY”);
return 0;
}
?
for(i=1 ; i<=n ; i++)
x=x+1;
n
for(i=1 ; i<=n ; i++)
for(j=1 ; j<=n ; j++)
x=x+1;
n2
EXAMPLE - 1
1. int i, f=1, n=5; -------------- 1 time
2. for(i=1 ; i<=n ; i++) -------------- n+1 times
3. f=f*i; -------------- n times
4. printf(“%d”,f); -------------- 1 time
n i f=f*i , f=1
5 1 1*1=1
2 1*2=2
3 2*3=6
4 6*4=24
5 24*5=120
6
T(n) = 1+(n+1)+n+1 = 2n+3 ≈ O(n)
if n=0 then T(n) = 1+1 ≈ O(1)
EXAMPLE - 2 1. int a=0,b=1,c, i, n=5; ------------- 1
2. printf(“%d %d”,a,b); ------------- 1
3. for(i=1 ; i<=n-2 ; i++ ){ ------------- (n-2)+1 = (n-1)
4. c=a+b; ------------- (n-2)
5. printf(“%d”,c); ------------- (n-2)
6. a=b; ------------- (n-2)
7. b=c; } ------------- (n-2)
i a b c=a+b
1 0 1 1
2 1 1 2
3 1 2 3
4
T(n)= 1+1+(n-1)+2*(n-2) = 2+n-1+2n-4 = 3n-3 ≈ O(n)
EXAMPLE - 3
1. int i, j, n=5, a[5]={12, 2, 51, 35, 7};
2. for(i=0;i<n-1;i++){
3. for(j=0;j<n-1-i;j++)
4. if(a[j]>a[j+1]){
5. a[j] = a[j] + a[j+1];
6. a[j+1] = a[j] - a[j+1];
7. a[j] = a[j] - a[j+1]; }
8. for(i=0 ; i<n ; i++)
9. printf(“%d”, a[i]);
T(n) = ?
ASYMPTOTIC NOTATIONS
O (Big-oh) Notation
Consider a function f(n) which is non-negative for all
integers n=0. we say that f(n)is Big-oh g(n), which we
write f(n)=O(g(n)), if there exits an integer n0 and a
constant c>0 such that for all integers n=n0, f(n)=cg(n).
In other words,
O(g(n))={f(n): ∃ positive constants c and n0 ∋ 0 ≤ f(n) ≤cg(n) ∀ 𝑛 ≥ 𝑛0 }
O-notation to give an upper bound on a function to within a
constant factor.
EXAMPLE
Two students, X & Y
Let, performance of X & Y are TX and TY respectively,
Conclusion is for a large n, TX(n) ≻ TY(n)
So, who is a better professional ? Ans. X
When X outperforms Y and by what factor ? Ans. n0 and c
# Events X Y TX ∼ TY
1 10th 70% 85% TX(1) ≺ TY(1)
2 (10+2)th 65% 78% TX(2) ≺ TY(2)
3 B.Tech 8.8 cgpa 8.7 cgpa TX(3) ≻ TY(3)
4 M.Tech GATE rank 120 GATE rank 1700 TX(4) ≻ TY(4)
5 Ph.D 5 publications 2 publications TX(5) ≻ TY(5)
6 Job 20 lakhs/p.a. 10 lakhs/p.a. TX(6) ≻ TY(6)
EXAMPLE
Suppose C=1 then, f(n)=6n+135
f(n) = cn2
6n +135 = cn2 = n2 (since c=1)
0 = n2 - 6n -135
0 = (n-15)(n+9)
Since (n+9) > 0 for all values n ≥ 0, then (n-15)=0
n0=15
Find n0 when c=2 & c=4.
NOTATIONS
Big-oh : 0≤ f(n) ≤ cg(n)
Theta : 0≤ 𝑐1𝑔 𝑛 ≤ f(n) ≤ c2 𝑔 𝑛 )
Omega : 0 ≤ cg(n) ≤ f(n)
RATE OF GROWTH
𝒍𝒐𝒈𝟐𝒏 n n𝒍𝒐𝒈𝟐𝒏 𝒏𝟐 𝒏𝟑 𝟐𝒏
0 1 0 1 1 2
1 2 2 4 8 4
2 4 8 16 64 16
3 8 24 64 512 256
4 16 64 256 4096 65536
TYPES OF TIME COMPLEXITY
Best Case Time complexity
Exp(1) – Linear search/Binary search
Linear search, key=3
a[0]==key, T(n)=O(1)
Binary search, key=24
m=(0+4)/2 = 2
a[2]==key, T(n)=O(1)
3 5 24 45 78
TYPES OF TIME COMPLEXITY
Worst Case Time complexity
Exp(2) – Linear search/Binary search
Linear search, key=78 / 80
a[0]==key, T(n)=O(n) / O(n+1) ≈ O(n)
Binary search, key=78 / 80
m=(0+4)/2 = 2 …
a[2]!=key,…
finally T(n)=O(log2 𝑛) ….(*)
3 5 24 45 78
TYPES OF TIME COMPLEXITY
Average Case Time complexity
Exp(3) – Linear search/Binary search
Linear search, key=24
a[0]==key, T(n)=O((n+1)/2) ≈ O(n)
There are n cases that can occur, i.e. find at the first place, the second place, the third place and so on up to the nth place. If found at the ith place then i comparisons are required. Hence the average number of comparisons over these n cases is:
average = (1+2+3.....+n)/n = (n+1)/2
where the result was used that 1+2+3 ...+n is equal to n(n+1)/2.
Binary search, key=5
m=(0+4)/2 = 2, m=(0+1)/2=0, m=(1+1)/2=1
a[1]==key, finally T(n) ≈ O(log2 𝑛) ….(*)
3 5 24 45 78
SORTING ALGORITHMS
Algorithm Data
Structure
Time
Complexity:
Best
Time
Complexity:
Average
Time
Complexity:
Worst
Space
Complexity:
Worst
Quick Sort Array O(n log(n)) O(n log(n)) O(n2) O(log(n))
Merge sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(n)
Heap sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(1)
Smooth sort Array O(n) O(n log(n)) O(n log(n)) O(1)
Bubble sort Array O(n) O(n2) O(n2) O(1)
Insertion
sort Array O(n) O(n2) O(n2) O(1)
Selection
sort Array O(n2) O(n2) O(n2) O(1)