1. AlgorithmsSandeep Kumar Poonia Head Of Dept. CS/IT B.E., M.Tech., UGC-NET LM-IAENG, LM-IACSIT,LM-CSTA, LM-AIRCC, LM-SCIEI, AM-UACEE

2. Algorithms Introduction Proof By Induction Asymptotic notation 3. Pre Computer Era: In the pre computer era various models of computation were proposed. The models worked on a predefined set of operations. The primary focus in this era was on What can be computed and what cannot be computed? This field of study was called Computability Theory.Sandeep Kumar Poonia 4. Post Computer Era: With ultimate computability in terms of Turing Machines, the primary focus in the post computer era was Complexity Theory. People were more interested to find out How well can it be computed?Sandeep Kumar Poonia 5. Sandeep Kumar Poonia 6. Sandeep Kumar Poonia 7. Sandeep Kumar Poonia 8. As a corollary to the above mentioned result we can easily see that a number n must be halved floor(log2n) + 1 times to reach 0.Sandeep Kumar Poonia 9. Algorithms An algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output.An algorithm is thus a sequence of computational steps that transform the input into the output.Sandeep Kumar Poonia 10. AlgorithmsSandeep Kumar Poonia 11. AlgorithmsSandeep Kumar Poonia 12. Properties of AlgorithmsSandeep Kumar Poonia 13. Steps to develop an algorithmSandeep Kumar Poonia 14. Algorithms as a technology Suppose computers were infinitely fast and computer memory was free.Would you have any reason to study algorithms?......YES(See an example) let us assume a faster computer (computer A) running insertion sort against a slower computer (computer B) running merge sort. They each must sort an array of one million numbers. Suppose that computer A executes one billion instructions per second and computer B executes only ten million instructions per second, so that computer A is 100 times faster than computer B in raw computing power. insertion sort, takes time roughly equal to c1n2 to sort n items, merge sort, takes time roughly equal to c2n lg n Sandeep Kumar Poonia 15. Algorithms as a technology To sort one million numbers, Computer A takes Computer B takesBy using an algorithm whose running time grows more slowly, even with a poor compiler, computer B runs 20 times faster than computer A Sandeep Kumar Poonia 16. Review: Induction Suppose S(k) is true for fixed constant k o Often k = 0 S(n) S(n+1) for all n >= k Then S(n) is true for all n >= kSandeep Kumar Poonia 17. Proof By Induction Claim:S(n) is true for all n >= k Basis: Show formula is true when n = k Inductive hypothesis: Assume formula is true for an arbitrary n Step: Show that formula is then true for n+1Sandeep Kumar Poonia 18. Induction Example: Gaussian Closed Form Prove 1 + 2 + 3 + + n = n(n+1) / 2 Basis: o If n = 0, then 0 = 0(0+1) / 2 Inductive hypothesis: o Assume 1 + 2 + 3 + + n = n(n+1) / 2 Step (show true for n+1): 1 + 2 + + n + n+1 = (1 + 2 + + n) + (n+1) = n(n+1)/2 + n+1 = [n(n+1) + 2(n+1)]/2 = (n+1)(n+2)/2 = (n+1)(n+1 + 1) / 2Sandeep Kumar Poonia 19. Induction Example: Geometric Closed Form Prove a0 + a1 + + an = (an+1 - 1)/(a - 1) for all a 1 Basis: show that a0 = (a0+1 - 1)/(a - 1) a0 = 1 = (a1 - 1)/(a - 1) Inductive hypothesis: o Assume a0 + a1 + + an = (an+1 - 1)/(a - 1) Step (show true for n+1): a0 + a1 + + an+1 = a0 + a1 + + an + an+1 = (an+1 - 1)/(a - 1) + an+1 = (an+1+1 - 1)/(a - 1)Sandeep Kumar Poonia 20. Induction Weve been using weak induction Strong induction also holds Basis: show S(0) Hypothesis: assume S(k) holds for arbitrary k 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 28. An Example: Insertion Sort 301040201234i = j = key = A[j] = A[j+1] = InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 29. An Example: Insertion Sort 301040201234i=2 j=1 A[j] = 30key = 10 A[j+1] = 10InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 30. An Example: Insertion Sort 303040201234i=2 j=1 A[j] = 30key = 10 A[j+1] = 30InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 31. An Example: Insertion Sort 303040201234i=2 j=1 A[j] = 30key = 10 A[j+1] = 30InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 32. An Example: Insertion Sort 303040201234i=2 j=0 A[j] = key = 10 A[j+1] = 30InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 33. An Example: Insertion Sort 303040201234i=2 j=0 A[j] = key = 10 A[j+1] = 30InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 34. An Example: Insertion Sort 103040201234i=2 j=0 A[j] = key = 10 A[j+1] = 10InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 35. An Example: Insertion Sort 103040201234i=3 j=0 A[j] = key = 10 A[j+1] = 10InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 36. An Example: Insertion Sort 103040201234i=3 j=0 A[j] = key = 40 A[j+1] = 10InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 37. An Example: Insertion Sort 103040201234i=3 j=0 A[j] = key = 40 A[j+1] = 10InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 38. An Example: Insertion Sort 103040201234i=3 j=2 A[j] = 30key = 40 A[j+1] = 40InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 39. An Example: Insertion Sort 103040201234i=3 j=2 A[j] = 30key = 40 A[j+1] = 40InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 40. An Example: Insertion Sort 103040201234i=3 j=2 A[j] = 30key = 40 A[j+1] = 40InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 41. An Example: Insertion Sort 103040201234i=4 j=2 A[j] = 30key = 40 A[j+1] = 40InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 42. An Example: Insertion Sort 103040201234i=4 j=2 A[j] = 30key = 20 A[j+1] = 40InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 43. An Example: Insertion Sort 103040201234i=4 j=2 A[j] = 30key = 20 A[j+1] = 40InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 44. An Example: Insertion Sort 103040201234i=4 j=3 A[j] = 40key = 20 A[j+1] = 20InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 45. An Example: Insertion Sort 103040201234i=4 j=3 A[j] = 40key = 20 A[j+1] = 20InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 46. An Example: Insertion Sort 103040401234i=4 j=3 A[j] = 40key = 20 A[j+1] = 40InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 47. An Example: Insertion Sort 103040401234i=4 j=3 A[j] = 40key = 20 A[j+1] = 40InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 48. An Example: Insertion Sort 103040401234i=4 j=3 A[j] = 40key = 20 A[j+1] = 40InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 49. An Example: Insertion Sort 103040401234i=4 j=2 A[j] = 30key = 20 A[j+1] = 40InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 50. An Example: Insertion Sort 103040401234i=4 j=2 A[j] = 30key = 20 A[j+1] = 40InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 51. An Example: Insertion Sort 103030401234i=4 j=2 A[j] = 30key = 20 A[j+1] = 30InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 52. An Example: Insertion Sort 103030401234i=4 j=2 A[j] = 30key = 20 A[j+1] = 30InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 53. An Example: Insertion Sort 103030401234i=4 j=1 A[j] = 10key = 20 A[j+1] = 30InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 54. An Example: Insertion Sort 103030401234i=4 j=1 A[j] = 10key = 20 A[j+1] = 30InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 55. An Example: Insertion Sort 102030401234i=4 j=1 A[j] = 10key = 20 A[j+1] = 20InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 56. An Example: Insertion Sort 102030401234i=4 j=1 A[j] = 10key = 20 A[j+1] = 20InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Done!Sandeep Kumar Poonia 57. Insertion Sort What is the precondition InsertionSort(A, n) { for this loop? for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } }Sandeep Kumar Poonia 58. Insertion Sort InsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key } How many times will } this loop execute?Sandeep Kumar Poonia 59. Insertion Sort StatementEffortInsertionSort(A, n) { for i = 2 to n { c1n key = A[i] c2(n-1) j = i - 1; c3(n-1) while (j > 0) and (A[j] > key) { c4T A[j+1] = A[j] c5(T-(n-1)) j = j - 1 c6(T-(n-1)) } 0 A[j+1] = key c7(n-1) } 0 } T = t2 + t3 + + tn where ti is number of while expression evaluations for the ith for loop iteration Sandeep Kumar Poonia 60. Analyzing Insertion Sort T(n) = c1n + c2(n-1) + c3(n-1) + c4T + c5(T - (n-1)) + c6(T - (n-1)) + c7(n-1) = c8T + c9n + c10What can T be? Best case -- inner loop body never executed o ti = 1 T(n) is a linear function Worst case -- inner loop body executed for all previous elements o ti = i T(n) is a quadratic functionSandeep Kumar Poonia 61. Analysis Simplifications Ignore actual and abstract statement costs Order of growth is the interesting measure: o Highest-order term is what counts Sandeep Kumar PooniaRemember, we are doing asymptotic analysis As the input size grows larger it is the high order term that dominates 62. Upper Bound Notation We say InsertionSorts run time is O(n2) Properly we should say run time is in O(n2) Read O as Big-O (youll also hear it as order) In general a function f(n) is O(g(n)) if there exist positive constants c and n0 such that f(n) c g(n) for all n n0 Formally O(g(n)) = { f(n): positive constants c and n0 such that f(n) c g(n) n n0 Sandeep Kumar Poonia 63. Lower Bound Notation We say InsertionSorts run time is (n) In general a function f(n) is (g(n)) if positive constants c and n0 such that 0 cg(n) f(n) n n0 Proof: Suppose run time is an + b o Assume a and b are positive (what if b is negative?) an an + bSandeep Kumar Poonia 64. Asymptotic Tight Bound A function f(n) is (g(n)) if positive constants c1, c2, and n0 such that c1 g(n) f(n) c2 g(n) n n0 Theorem f(n) is (g(n)) iff f(n) is both O(g(n)) and (g(n))Sandeep Kumar Poonia 65. Practical Complexity 250f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2f(n) = n^3 f(n) = 2^n0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20Sandeep Kumar Poonia 66. Practical Complexity 500f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20Sandeep Kumar Poonia 67. Practical Complexity 1000f(n) = n f(n) = log(n) f(n) = n log(n) f(n) = n^2 f(n) = n^3 f(n) = 2^n0 1Sandeep Kumar Poonia35791113151719 68. Practical Complexity 50004000 f(n) = n f(n) = log(n)3000f(n) = n log(n) f(n) = n^2 2000f(n) = n^3 f(n) = 2^n10000 1Sandeep Kumar Poonia35791113151719 69. Practical Complexity 10000000 1000000 100000 10000 1000 100 10 1 1Sandeep Kumar Poonia416642561024409616384 65536 70. Other Asymptotic Notations A function f(n) is o(g(n)) if positive constants c and n0 such that f(n) < c g(n) n n0 A function f(n) is (g(n)) if positive constants c and n0 such that c g(n) < f(n) n n0 Intuitively, o() is like < O() is like Sandeep Kumar Poonia () is like > () is like () is like =