computer algoritham for chennai univarcity unit 1 and2

Design And Analysis of Algorithm –study material – B.Sheik Md Abdullah –MIIT

Unit 1

Word algorithm comes from name of Persian author Abu Jufar mohammed ibm musa al khowarizmi(825ad) Definition : Algorithm An algorithm is a finite set of instructions that, if followed, accomplishes a particular task. In addition algorithm must satisfy the following criteria: Input: Zero or more quantities are externally supplied Output: At least one quantity is produced. Definiteness: Each instruction is clear and unambiguous Finiteness: If we trace out instruction of algorithm, then for all cases the algorithm terminates after finite number of steps. Effectiveness: Every instruction must be very basic so that it can carried out.(in principle by a person using only pencil & paper)

Areas of study in algorithm: • How to device algorithm? Mastering various design strategies, techniques it will become easier to device new & useful algorithm. e.g dynamic programming , divide conquer , greedy • How to validate algorithm? Once an algorithm is devised, it is necessary to show that it computes the correct answer for all possible legal input.This process is called algorithm validation. • How to analyze algorithms? Analysis of algorithms or performance and refers to the task of determining how much computing time & storage(immediate & auxiliary) an algorithm requires. • How to test a program? Testing a program consist of two phases. Debugging is the process of executing programmers on sample data set to determine whether faulty result occur & if so,to correct them. Profiling: Or performance measurement is time & space it takes to compute the result Algorithm specification: Pseudocode convention: 1.//comments 2. blocks 3.identifier begins with a letter. 4.data types: not explicitly declared.


5.compound data types node=record datatype1 data-1; datatype2 varn; node *link; 6.simple datatypes: integer,float,char,Boolean 7.assignment variable:=expression; 8.while loop takes following form While < condition > do <statement 1> . . . <statement n>

Recursive Algorithm

An algorithm that is defined in terms of itself is called Recursive algorithm An algorithm that calls itself is direct recursive. An algorithm A is said to be indirect recursive if it calls another algorithm,

which in turn calls A. Problem itself is recursively defined then choosing mechanism of recursive

algorithm is appropriate. e.g Towers of Hanoi, Permutation generator Performance Analysis

1) Space complexity 2) Time complexity 3) Asymptotic Notations 4) Performance measurement 1.Space complexity

Space complexity of an algorithm is the amount of memory it needs to run to completion


E.g Algorithm abc(a,b,c) return a+b+b*c+(a+b-c)/(a+b) +4.0; The space needed by each of these algorithm is seen to be the sum of the following components: 1.A fixed part that is independent of the characteristics (e.g ,number , size ) of the inputs and outputs, space for fixed size component variable, constants and so on. 2. A variable part that consist of the space needed by component variables whose size is dependent on the particular problem instance, recursion stack space etc. So the space requirement S(P) of any algorithm P may therefore be written as S(P) = c + SP(instance characteristics),where c is constant. 2.Time complexity Time complexity of an algorithm is the amount of computer time it needs to run to completion. Program step It is loosely defined as a syntactically meaningful segment of a program that has an execution time that is independent of the instance characteristics. Example 1 When the statements to increment count are introduced into a Algorithm the result is following Algorithm. The change in the value of count by the time this program terminates is the number of steps executed by Algorithm It is easy to see that in the for loop, the value of count will increase by a total of 2n. If count is zero to start with, then it will be 2n + 3 on termination. So each invocation of Sum Algorithm executes a total of 2n + 3 steps.


Recursive formula for stem count is tRSum(n) = tRSum(n) = 2 + tRSum(n-1) = 2 + 2 + tRSum(n-2)

= 2(2) + tRSum(n-2) . .

Example 2 When the statements to increment count are introduced into Recursive algorithm then count will be calculated in following way.. Let tRsum(n) be theincrease in the value of count when Algorithm terminates. We see that tRSum(O) = 2. When n > 0, count increases by 2 plus whatever increase results from the invocation of RSumfrom within the else clause. From the definition of tRSum, it follows that this additional increase is tRsum(n-1). So, if the value of count is zero initially, its value at the time of termination is 2+tRsum(n-1), n>Q .

2 if n=0 2 + tRSum(n-1) if n>0


. =n(2) + tRSum(0) =2n+2, n>=0 So the step count for RSum is 2n+2 StepTable Method Example Algorithm Iterative function for sum

1 Algorithm Sum(a, n) 2 3 s := 0.0;

4 for i := 1 to n do 5 s := s + a[i]; 6 return s;

7

1 Algorithm RSum(a,n) 2

3 if (n <= 0) then return 0.0; 4 else return RSum(a, n - 1) + a[n]; 5

Algorithm: Recursive function Rsum Step table for Algorithm sum


Step table for Algorithm Rsum

3. Asymptotical Notations


4.Performance Measurement Performance measurement is concerned with obtaining the space and time requirements of a particular algorithm .These quantities depends on the compiler and options used as well as on the computer on which the algorithm is run Limitation of Asymptotic analysis

Asymptotic analysis tells the behavior only for sufficiently large values on n. for smaller values for n , the run time may not follow the asymptotic curve.

Even in the region where the asymptotic behavior is exhibited, the times may

not lie exactly on the periodic curve


RANDOMIZED ALGORITHMS Informal Description: It makes use of randomizer(random number generator) Categorized into two classes

• Los Vegas Execution time is depends on o/p of randomizer • Monte carlo O/p might differ from run to run for same i/p . i.e algorithm might Give incorrect answer.

Error probability(e)

Randomized algorithms runs quickly (respectively gives the correct answer) with probability at least 1-e

DEF. LAS VEGAS: Las Vegas has a resource (time, space, etc) bound of õ (g(n)) H there exist constant c such that the amount of resource used by the alg ( on any i/p of size n) is no more than C α g(n) with probability >= 1-1/nα. eg: Las Vegas 1 - IDENTIFYING THE REPEATED ELEMENTS: Stmt: Let an array a[] of n numbers that has n/2 distinct elements & n/2 copies of another element the problem is to identify the repeated element. ⇒Any deterministic alg will need at least n/2 + 2 time steps.


⇒ But randomized las Vegas takes only õ (log n). Algorithm:- Algorithm RepeatedElement(a,n) //finds the repeated element from a[1:n] While (true) do I:=Random() mod n+1; J:=Random() mod n+1; If ((i≠j) and a[I]=a[j]) then return I; Runtime Calculation 1)An iteration of while loop will be successful(in identifying rep-element) if ‘i’ is any one of n/2 array indices corresponding to the repeated element and j is any one of the same n/2 indices other than i. 2)The probability that the algorithm quits in any given iteration of while loop is

Which is ≥ 1/5 for all n≥10 3)This implies that the probability that the algorithm does not quit is < 4/5

4) There fore the algorithm quit in 10 iteration is <(4/5)10 < .1074 & will terminate(probability is) > 0.8926 5) If 100 iterations then Probability (for algorithm does not quit) is (4/5)100 < 2.04 * 10-10

This is almost certainly algorithm will quit in 100 iterations.

N/2(n/2-1) P= N2


6) In general the algorithm the algorithm doesn’t quit in the first c α logn iteration is <(4/5) c α logn which will be less than n-α if c>=1/log(5/4) Note: There fore (4/5) c α logn = n -c α logn(5/4)

7) Then the algorithm terminates in 1/log(5/4) α logn with propability >=1- n-α

8) Hence runtime of algorithm is log n i.e õ (log n) 2 - Miller Rabin’s Algorithm. (Primality Testing)

1) If input is prime, it will never give an incorrect answer. 2) If n is composite, it will detect the compositeness of x if the randomly chosen ‘a ‘

either leads to the discovery of nontrivial square root of 1 or violates fermat’s equation.

3) The probability that a randomly chosen ‘a’ will be a witness to the compositeness of x can be answered by the following theorem.

Theorem There are at least ( n-1)/2 witness to the compositeness of x if x is composite and odd Time complexity calculation

1) Assume that n is composite 2) Probability [ for chosen ‘a’ will be a witness ] is > (n-1)/2n which is -~½. 3) Probability [ for ‘a’ will fail to be a witness ] is < ½. 4) ∴Probability [none of the first α log n ]a’s chosen as witness is ≤(1/2)α log n =n-α

that means this algorithm give an incorrect answer with only probability ≤ n-α . hence it satisfies the definition of o

5) Run time of outer most while loop is O (log n). Run time of for loop is O(log n) Since it is a nested loop (for and while), the total runtime is O(log n)* O(log n) = O(log2 n)

Algorithm prime(n,a) q:=n-1; For i:=1 to αlog(n) do m:=q;y:=1;


a:=random()mod q+1; z:=a; //compute an-1 mod n while(m>0) do while (m mod z=0) do x:=z;z:=z2 mod n;

//if x is a nontrivial square

// root of 1,n is not prime

if((z=1)and(x≠1) and (x≠q))then

return false;

m:=mm/2 ;

m:=m-1;y:=(y*z)mod n;

If(y≠1) return false;

Return true;


Def: General method

• This strategy suggest, splitting the input into 'k' distinct subsets, 1<k<=n yielding k sub problems. This sub problems must be solved and then a method must be found to combine sub solutions into a solution of a whole.

• We can write a control abstraction that mirrors the way divide & conquer

algorithm will work.

• By a control abstraction we mean a procedure whose flow of control is clear. But whose primary operation are specified by other procedure (left undefined.). We are calling this algorithm DAndC(p) whose 'p' is a problem to be solved.

Control Abstraction

C

***

Divide and Conquer method

Algorithm DAndC(P) if Small(P) then return S(P) else divide P into smaller instances P1,P2....Pk,k>=1; Apply DAndC to each of these subproblems; return Combine(DAndC(P1),DAndC(P2),...DAndC(PK));

omputing time of DAndC is described by

T(n)=

Where T(n) is the time for DandC on any input of size ‘n’ g(n) time to compute the answer directly form small input. f(n) time for dividing ‘p’ and Combining the solutions to sub problems

g(n) ‘n’ small T(n1)+T(n2)+….+T(nk)+fn) otherwise

Eq.1


The complexity of many divide and conquer algorithm is given by recurrences of the form

T(n)= Where a,b --- known constants T(n)--- is known N is a power of ‘b’ (i.e.)n=bk

One of the methods for solving any such recurrence relation is called Substitution method. Example: T(n)=2T(n/2) + n =2 [2T(n/4)+n/2] +n =4 T(n/4) + 2n =4 [2T(n/8)+n/4] + 2n =8T(n/8)+3n …… * In general we see that T(n)=2iT(n/2i)+in for any log2n >=I>=1. * In particular T(n)=2 log2n+nlog2n where I=long2n

• Thus T(n)=nT(1)+nlog2n=nlog2n+2n

With recurrence (eq.2)& using substitution method it can be shown that

T (n) = n log b a [T (1)+u (n)

Where u (n) = Σj=1

k h (b j) & h (n) = f(n)/n log b a

Table shows u(n) values for various h(n) values .

Eg Look at the following:

T(1) n=1 aT(n/b)+f(n) n>1

h(n) u(n) Ɵ (nr) , r <0 Ɵ(1) Ɵ ((log n )I)I>=0 Ɵ((log n)i+1/(I+1)) Ω(nr) , r>0 Ɵ(h(n)

Eq.2


Comparing with(eq2) ; we see that a=1,b=2 &f(n)=c So, log b a =0 and h (n) = f(n)/n log b a h(n) =c h(n)=c(log n)0 h(n) = Ɵ((log n)0) From the above table We obtain u (n) = Ɵ (log n). So,

BINARY SEARCH

1. Algorithm Binary Search(a,n,x) 2. // Given an array a[1:n] of elements in non- 3. // decreasing order, n>=0, determine whether x is 4. // present, and if so, return j such that x=a[j]; 5. // else return 0. 6. 7. low:= 1; high:= n; 8. while(low<=high) do 9. 10. mid:=[(low+high)/2]; 11. if(x<a[mid])then high:=mid-1; 12. elseif(x>a[mid])then low:=mid+1; 13. else return mid; 14. 15. return 0; 16.

INFORMAL PROOF OF BINARY SEARCH Theorm: Algorithm Binarysearch (a,n,x) works correctly.

T (n) = T (1) n=1 T (n/2)+c n>1

T(n) = n log b a[c+ Ɵ(log n) ] = Ɵ (log n)


Proof: * Initially low:= 1, high:= n, n>=0&a[1]<=a[2] <=a[3]….<a[n]. * If n=0, while loop is not entered. So, ‘0’ is returned. * Otherwise possible elements to be checked with ‘x’ in the loop(each time is)

a[low],a[low+1],….a[mid]…a[high]. * If x=a[mid], then the alg. Terminates successfully. * Otherwise the range is narrowed by increasing low to(mid+1) or decreasing high to

(mid-1). This narrowing of the range doesn’t effect the search. * If ‘low’ becomes greater than ‘high’, then ‘x’ is not present. Hence, the loop is exited.

RUN-TIME CALCULATION: THEOREM 3.2: If ‘n’ is in the range[2k-1,2k], then Binary search makes at most ‘k’ element comparisons for a successful search and either ‘k-1’ or ‘k’ comparisons for an unsuccessful search(In other words the time for a successful search is O(log n)and for an unsuccessful search is θ(log n)).

Binary decision tree for binary search n = 14 First comparison – x with a[7] if x < a[7], x with a[3]

Next comparison if x > a[7], x with a[11]

7

3 1

1 5 9 13

4 6 8 10 12 142


Each page through the tree represent a sequence of comparison in the binary search method.

• If x is present, alg will end at one of the circular nodes[ that lists the index into the array where x was found]

• If x is not present, alg will terminate at one of the square nodes.

Theorem

If n is in the range, (2k-1, 2k) then binary search make k element comparisons for a successful search and either k-1 or k comparisons for an successful search ( in other words the time for a successful search is O(log n) and for unsuccessful search is O(log n)

Proof : Consider the binary decision tree describing the action of Binary search on n elements. All Structural searches end at circular node and unsuccessful searches end at

square node. If 2k-1 <= n <= 2k, then all circular nodes are at level 1,2,….k [ Note that root is at

level 1] All square nodes are at levels k and k+1 Number of element comparisons needed to terminate at a circular node on level I

is I (Whereas for square node at level I is only I-1).The theorem follows.

Average Case Behavior Distance of Node : From the root is one less than it’s level. Internal path length I

I = Σ distance of al internal nodes. External Path Length E

E = Σ Distance of all External Nodes. For any Binary Tree with n internal nodes

E = I + 2n → eq1

Eg: - N=7 I = 10 E = I + 2n = 10 + 14 = 24 //


Proof (for average case) Let A(0) – Average no of comparisons in a Let Au (n) - “ “ “ The no of comparisons needed to find an element represented by an internal node is As(n) = 1 + I / n -------------(2) Since each Bin tree has n+1 external nodes Au(n) = E / (n+1) ------------ (3) From (1), (2) (3) As (n) = (1 + 1/n) au (n) –1

Hence As(n) & Au(n) is directly related Minimum As(n) is achieved (hence Au(n)) when Bin delition tree has

minimum external &n internal path length I,e E,I. This is achieved by the bin tree all of whose external node are on adjacent

levels. (This is precisely the tree produced by binary search) From theorem 3.2 E is proportional to n logn Hence Au(n), As?(n) is proportional to logn

Successful Searches Un Successful searches Ө (1) Ө log n Ө (logn) Ө (logn) Best Average worst cases for all cases. Finding maximum and minimum Prob.stmt: To find the maximum and minimum of items in a set of n elements. Stright method(stright maxmin) It requires 2(n-1) element comparisons in best,avg,worst case.


B W A MA//i e e /m/MM/ii

Algorithm stright maxmin (a,n,max,min) max:= min= a[1]; for i:=2 to n do if (a[i]>max) then max:=a[i]; if (a[i]<min) then min :=a[i];
est case (when elements are in increasing order )
Time complexity =θ(n-1) orst case (when elements are in decreasing order )

Time complexity=2(n-1) vg case ( a[i] wil be > max. half the time )

Time complexity=θ(3n/2-1)

inMax by Divide & Conquer Method lgorithm MaxMin(i,j,max,min)

/a[1:n] – global array //i,j are integers /Aim is to set max,min to largest &smallest of a[i:j] respectively f(i=j)then max:=min:=a[i] n=1

lse if(i=j-1) then if(a[i]<a[j]) then max:=a[j];min:=a[i]; n=2

else max:=a[i];min:=a[j]; lse /Pis not small,divide p into sub problems

id:=(i+j)/2; /solve the sub problems

axMin(i,mid,max,min); axMin(mid+1,j,max1,min1); n>2

/ Combine the solution f(max<max1) then max:=max1; f(min>min1) then min:=min1;


Example min max A: [1] [2] [3] [4] [5] [6] [7] [8] [9] 22 13 -5 -8 15 60 17 31 47

1. The procedure MinMax is initially invoiced by the stmt MaxMin (1, n, x, y ) ( o/p parameters) 2. we can represent recursive calls into a tree in the tree – A node for each call

- Each node has info of i , j , max , min 3. For the given array a[] the tree of recursive calls of Maxmin in

(9) (8) (5) (3) (4) (6) (7) (1) (2)

1 , 9 , 60 , -80

1 ,5, 22, -8 6 , 9, 60, 17

1, 3, 22 ,-5 4, 5 ,15 ,-8 6,7 , 60, 17 8, 9, 47 , 31

1, 2 , 22 , 13 3 , 3 , -5 ,-5

Numbers in brackets represents the order in which max & min are assigned values


(1) (2) 4. From the three the maximum depth of recursion is 4. Time Complexity MinMax It is based on no.of elements compared Since Maxmin is recursive we have the recurrence relation T(n) =

When n is power of two n = 2k & k>=1 T(n) = 2 T(n/2) + 2 =2 (2 T(n/4) + 2 ) + 2 =4 T(n/4) + 4 + 2 . . .

3 , 3 , -5 ,-5

1 , 3 , max ,min

1 ,5 , max ,min

1 , 2 , 22 , 13

1, 3, max,

1, 2, 22, 13 3, 3, -5 , -5

1 2

1, 3, 22, -5

3

0 n=1 1 n=2 T(n/2) + T (n/2) +2 n>2


=2k-1 T(2) + Σi=1to k-1 2i =2k-1 + 2k-2 = 3n/2 –2 Advantage Compared to straight forward method this is saving of 25% in comparisions. Disadvantage In terms of storage minmax is worse then stright forward method

Observations: - 1. Divide & conquer strategy is seen to be only a guide to better alg design

which may not always succeed. 2. Use of asymptotic notation is not enough of a discriminator.

Since both maxmin & Strightmaxmin are Ө(n) MERGE SORT PROBLEM DESCRIPTION: Given a sequence of n elements (also called keys) a[1]…a[n], the general idea is to imagine them in split into two sets a[1],…a[n/2] and a[(n/2)+1],…a[n]. Each set is individually sorted and resulting sorted sequences are merged to produce a single sequence of n elements. Algorithm Merge Sort (Low,High) if (low<high) then mid := [(low+high)/2; MergeSort (low,mid); MergeSort (mid+1,high); Merge (low,mid,high); Algorithm Merge (low,mid,high) h :=low; i:=low, j:=mid+1; while ((h<=mid) and (j<=high)) do if (a[h]<=a[j]) then b[i] := a[h]; h:= h+1;


else b[i]:= a[j];j:=j+1; i:=i+1;

if(h>mid) then for k:=j to high do

b[i]:= a[k]; i:=i+1; else for k = h to mid do b[i]=a[k];i = i+1 for k= low to high do a[k] = b[k]; Example: - A[1 : 10] = 310, 285, 179, 652, 351, 423, 861, 254, 450, 520

310 | 285|179 | 652, 351 | 423, 861, 254, 450, 520 where vertical bars indicates the boundaries of sub arrays (height indicates the order) a[1] & a[2] are merged to yield

285, 310 | 179| 652, 351 | 423, 861, 254, 450, 520 Then a [3] is merged with a [1:2] to yield

179, 285, 310 | 632, 351 | 423, 861, 450, 520 then a[4] & a [5] are merged (after making them in to – two list)

179, 285, 310, 351, 652 | 423 | 861 | 254 | 450, 520 Elements a[6] & a[7] are merged

179, 285, 310, 351, 652 | 423, 861 | 254 | 450, 520 Elements a[6:8] & a[8] are merged

179, 285, 310, 351, 652 | 254, 423, 861 | 450, 520


Next a[6:8] a [9:10] are merged (after making a[9 : 10] in to two lists and a merge) 179, 285, 310, 351, 652 | 254, 423, 450, 520, 861 Now there are only two sorted list – So final merge produces the result. 179, 254, 285, 310, 351, 423, 450, 520, 652, 861 Time complexity If the time of merging operation is proportional to n, then the recurrence relation

describes the computing time for merge sort: T (n) = a n = 1,A a constant 2T(n/2)+cn n > 1, A a constant When n is a power of 2. ie n=2k then substitution method can be applied:

T (n) = 2(2T(n/4)+cn/2)+cn

= 4T(n/4)+2cn = 4(2T(n/8)+cn/4)+2cn . . . =2k T (1)+kcn =an+cn log n It is easy to show that if 2k<n< =2k+1,then T (n)<=T (2k+1) Therefore T (n)=O (n log n) QUICK SORT In quick sort , the division in to two sub arrays is made so that the sorted sub array do not need to be merged later. This is accomplished by rearranging the elements in a[1:n] such that a[i] <= a[j] for all i between 1 and m and all j between m+1 and n for some m.(1 <= m <= n) Partitioning: The rearrangement of the element is accomplished by picking some elements of a[] , say t = a[s] and then recording the other elements so that all elements appearing before t in a[1:n] are less than or equal to t and all elements appearing are greater than or equal to t. This rearranging is referred to as partitioning.

Algorithm QuickSort(P,q) if (p<q) then j = partition(a,P,q+1) QuickSort(P,j-1) QuickSort(P,j-1)

Algorithm Partition(a,m,p) v:=a[m]; i=m; j=p; repeat repeat i=i+1 until ( [i] >= v); repeat j= j-1; until(a[j] <= v); if (i<j) then interchange (a,i,j); until (i>=j); a[m]=a[j]; a[j] =v; return j;


a[1] a[2] 3 4 5 6 7 8 9 10 i j 65 70 75 80 85 60 55 50 45 +∞ 2 9 65 45 75 80 85 60 55 50 70 +∞ 3 8 65 45 50 80 85 60 55 75 70 +∞ 4 7 65 45 50 55 85 60 80 75 70 +∞ 5 6 65 45 50 55 60 85 80 75 70 +∞ 6 5 60 45 50 55 65 85 80 75 70 +∞

Analysis (Wost case) Only no.of comparison are counted. No.of element comparison in each call of partition >= p-m +1 Let r = total number of elements in all the calls to partition at any level of recursion.

o At level 1 – partition(a,1,n+1) is made and r=n o At level 2 – at most two calls are made – so r = n-1. o Hence r is at least one less than r at the previous level

hence Cw(n) is the sum on r as 2 to n .i.e n2 i.e O(n2) Randomized sorting Algorithm One of the i/p on which Quicksort worst-case occur is elements are already in sorted order. Solution to this is the use of randomizer . While sorting array a[p:q] instead of picking a[m], pick a random element as the partition element.

Algorithm Interchange(a,i,j) P=a[i]; a[i]=a[j];a[j]=p;

Example for one partition call Elements which are interchanged to produce the next row (in bold)

change made by last stmt of Partition


SELECTION In this problem, we are given n elements a [1:n] and are required to determine the kth smallest element. Algorithm Select1(a, n, k) low :=1; up := n+1; a[n+1] := ∞; repeat j := partition(a, low, up) if(k = j) then return; else if (k<j) then up := j; else low := j+1; until(false); Analysis of Select1 (worst-case) Partition requires ΟΟΟΟ(p-m) time On each successive call to partition, either m increases by at least one or j

decreases by at least one. Initially m=1 & j=n+1

Hence at most n calls to partition can be made. Thus, the worst-case complexity

of Selection1 is ΟΟΟΟ(n2).

Average time

• Let TAK(n) be the average time to find the kth smallest element in a[1:n]

• This average is taken over all n! different permutations of n distinct elements.


Example: - A [2 ,5, 6] [2, 5, 6] Then all its permutations are [2, 6, 5] 3! = 6 [5, 2, 6] [6, 2, 5] [5, 6, 2] [6, 5, 2] Average time of select1 algorithm TA (n) is defined as follows

TA(n) = 1/n ∑ T k A (n) 1 ≤ k ≤ n and

R(n) = max Tk A(n) It is easy to see that TA(n) ≤ R (n) Theorem 3.3 The average computing time TA(n) of select1 is 0(n) Proof: - On the first call to partition, the partitioning, element V is the ith – smallest element with

probability 1/n. The time required by partition and it statement in select 1 is 0(n) Hence there is a constant C, C > 0 such that K – 1 n

Tk A (n) ≤ cn + 1/n [∑ TA k-1 (n-i) + ∑ Tk A (i-1)], where n ≥ 2 i =1 i = k +1 k n

So, R(n) ≤ cn + 1/n max ∑ R (n – i) + ∑ R (i – 1)m i =1 i = k +1 n – 1 n-1

R(n) ≤ cn + 1/n max ∑ R(i) + ∑ R (i) n-k+1 k

e.q 1


We assume C is chosen such that R (1) ≤ C & show, by induction on n, that R (n) ≤ 4 cn. Induction base : For n = 2, eq (1) gives R(n) ≤ 2c + ½ max R(1), R(1) ≤ 2.5c < 4cn Induction Hypothesis : Assume R(n) ≤ 4cn for all n, 2≤ n ¸m Induction Step : For n = m (3.8) gives m-1 m-1

R(m) ≤ cm + 1/m max ∑ R(i) + ∑ R(i) m-k+1 K Since R(n) is nondecreasing fn of n, it follows that this is maximized if k = m/2 when mix even (and k = m+1 where m is odd) 2 m - 1

Thus if m I even R(m) ≤ cm + 2/m ∑ R(i) M/2 ≤ 4cm

Since TA(n) ≤ R(n), it follows that TA (n) ≤ 4cn Hence TA (n) is O(n) A worst-case Optimal Algorithm for SELECTION (Using mm rule) By choosing the partition element v more carefully can obtain a selection algorithm with worst case complexity O(n) To obtain such algorithm v must be chosen so that at least some fraction of the elements is smaller than v & at least some fraction of elements is greater than v. Such a selection of v can be made by Median of median s rule MM Rule In this rule the n elements are divided in to n/r groups of r elements each. The remaining n-r[n/r] elements are not used The median mi of each of these [n/r] group is found Then, the median mm of the mi‘s is found. The median mm is used as the partitioning element.


Example The median of Medians where r = r , n =35.

Unit 2 V.STRASSENS’S MATRIX-MULTIPLICATION ALGORITHM Let A & B be two n x n matrices. Where n is a power of 2. A & B can be partitioned into four (n/2) x (n/2) matrices and the product of A & B can be expressed interms of these (n/2) x (n/2) matrices as: A11 A12 B11 B12 = C11 C12 A21 A22 B21 B22 C21 C22 Where, C11 = A11* B11 + A12 *B21 C12 = A11* B12 + A12 *B22 C21 = A21* B11 + A22* B21 C22 = A21* B12 + A22* B22 Note: A11,……B11,……C11,……C22 are matrices of size (n/2) x (n/2) By applying D and C algorithm (a recursive) T(n) can be given as T(n) <= mT (n/2) + a.n2/4 , n>2 equation.3 Where, mT (n/2)__ cost of multiplying m pairs of (n/2) x (n/2) matrices. a.n2 /4__ cost of doing ‘a’ additions By applying substitution method we can obtain T(n) = O(n3) But this is no improvement over the conventional method.

. . . . .

. . . . .

. . . . .

. . mm . .

. . . . .

. . . . .

. . . . .

Elements < = mm

Elements > = mm

Non decreasing order


STRASSEN’s METHOD: He discovered a clever method of multiplying two 2x2 matrices with elements from an arbitrary ring using only Seven Multiplications. He was able to multiply two n x n matrices in time O(n log n ) which is of order ~ n 2.81 Method: To complete the matrix product

C11 C12 = a11 a12 b11 b12

C21 C22 a21 a22 b21 b22 Step1:

m1= (a12 – a22)* (b21+b22)

m2= (a11 +a22) *(b11+b22)

m3= (a11 – a21) *(b11+b12)

m4= (a11 + a12) *(b11+b12)

m5=a11*(b12-b22)

m6=a22*(b21-b11)

m7=(a21+a22)*b11 Step 2:

c11=m1+m2-m4+m6 c21=m6+m7 c12=m4+m5 c22=m2-m3+m5-m7

Hence we have 7 Multiplications 18 Additions /Subtractions (in normal method we need 8 Multiplications) Run Time Calculations: Let n=2k Let T(n) be the no. of arithmetic operations needed to multiply n x n matrix By recurrence relation eq 3.0

T (n) = 7 t ( n / 2 ) + 18 (n / 2) 2 , n>2 Thus T(n) is O(7 log n) ie O(n log 7)


VI. CONVEX HULL DEF: Convex hull of a se s of points in the plane is defined to be the smallest convex polygon containing all the points of s. Example:

This is NOT a convex hull since this is concave polygon

This is NOT a Convex hull. Since it is not covering all point of plane

This is what a Convex hull

Convex Hull Algorithms

1)Quick Hull 2)Graham‘s Scan 3)An Divide and Conquer Alg


1 ) Quick hull Algorithm : Identifies two points p1,p2 of X with smallest & largest x-coordinate values

respectively. Divide set X into x1,x2 Where x1 has all points to the left of line segment <p1,p2> & X2 has all points to the right of line segment <p1,p2>

Convex hulls of x1,x2 are computed using Divide and conquer algorithm called

HULL. Algorithm HULL Determine a point of x1 called P

[Point t is obtained by computing area formed by p1,p,p2 for each p in X1and picking the one(point p that we call P3) with the largest area.]

Divide X1 into two parts .One part cof <p1,p3> (including p1 & p2).Oththe left of <p3,p2> (including p1 &

X1 upper hull

X 2 Lower hull

P
P
P

P1

ontains all ther part contaip2)

P2

P2

P1

P3

e points of x1 that are to the left ns all the points of x1 that are to


All the other points are interior points & can be dropped from consideration.

The convex hull of each part is computed recursively and merged easily by

placing one next to the other.

Time Complexity If x1 has m points then point of division p3 can be identified in O(m) time. Partitioning x1 in to two take O(m) time Merging two convex hull can be in O(1) After dividing convex hull x1 in to two we will get m1,m2 in each sub convex hull ie. m1+m2 <= m Then recurrence relation for T(m) is T(m) = T (m1) + T(m) + O (m) Which is similar to the run time of Quick sort. Hence worst-case run time is thus O(m2) Average case is O(n log n). 2)Graham’s scan:

Procedure:- If S is a set of points in the plane, Graham’s scan algorithm identifies the point P

from S with the lowest y-coordinate value. It then sorts the points of S according to the angle subtended by the points and P

with the positive x-axis Scanning through the sorted list starts at P,


• If there are three successive points, say P1,P2,P3 that form a left turn then P2 will be eliminated.

• Otherwise Continue the scan by next successive points.

Algorithm: Point=record float x; float y; Point *prev; Point *next;

Algorithm Scan(list) *P:=list; *P1:=list; repeat P2:=(P1->next) If ((P2->next)≠0 then P3:=(P2->next); Else return; Temp:=Area((P1->x),(P1->y),(P2->x),(p2->y),(P3->x),(P3->y)); If (temp≥0.0) then P1:=(P1->next); Else (P1->next):=P3; (P3->prev):=P1; delete P2; P1:=(P1->prev); until (false);

Algorithm ConvexHull(ptslist) sort(ptslist); Scan(ptslist); PrintList(ptslist); 4 5 Example:- 3 6 7 9 8 2 1 10 ••••


Sorted points 1 2 3 4 5 6 7 8 9 10 Points to be looked Definition of turn Action

Start->

P 1 2 left Move to Next Point 1 2 3 right Delete 2& take Previous point P 1 3 left MNP 1 3 4 left MNP 3 4 5 left MNP 4 5 6 right Delete 5.Take previous point 3 4 6 left MNP 4 6 7 left MNP 6 7 8 left MNP 7 8 9 right Delete 8,Take previous point 6 7 9 right Delete 7 &,Take previous point 4 6 9 left MNP 6 9 10 left MNP

End 9 10 P left Process terminated since P reached Result: After deletion of points 2,5,8,7 the remaining points 1,3,4,6,9,10. Time Complexity Function scan runs in O(n) time since for each triplet examined ,(either the scan

moves one node ahead or one point gets removed) The test for direction (Whether a left or right turn) is done in O(1). Function Area computes the signed area. But Major work in this Algorithm is Sorting .Since it takes O(n log n) time the

total time of Graham’s scan algorithm is O ( n log n)


3)An O(n log n) time Divide & Conquer Algorithm Given a set X of n points, the problem is reduced to finding the upper hull & lower hull separately & then putting them together Step 1: points P1,P2 are identified (least, largest x- coordinates values) step2 : given set X is separated into two (upper hull, lower hull based on<P1,P2> ) step 3: let upper hull be input. ( N No. of Points) step 4:partition the input into two equal halves q1,q2,.....q N/2 & qN/2,q N/2+2.......qN step 5: let each half of the upper hull H1,H2. step 6: a line of tangent is found for H1&H2. let <u,n> is the tangent step 7: all points of H1 that are to the right of u are dropped. all points of H2 that are to the left of v are dropped. step 8: remaining part of H1, the <u,v> tangent, remaining part of H2 forms the upper hull step 9: same set of steps will be applied for lower hull.

Time Complexity Finding points p1,p2 need Sorting the input points n The line of tangent is foun Since it is a recursive algo

time T(N) is called as foll It is equivalent to Merge S Hence runtime of this algo

Lemma: Let H1,H2 be two uppof H1, its point q of tangency w

1 tangent
u H
s O(n) time eed )(n log n) . d in ) (log n2) rithm to compute a upper hull with n input pows.

ort eq. 2T(n/2) + cn rithm is O(n log n)

er hulas wit at most m points each. If p is ith H2 can be found in O( log m ) time.

2
H
oints the

T(N) = 2T (N/2) + O(n log n)

any point


Proof: If q` is any point in H2 then we can check the following things (in O(1) time) Let x, y are left & right neighbors of q` in H2. If p q`x is right turn & pq`y is left then q is to the right of q` in H2. If pq`x is right turn & pq`y is left then q is t the right of q`. If pq`x and pq`y both are right turn then q` = q.

Otherwise (else) q is to the left of q` Thus we can perform binary search in H2 to identify Search) O(log m) time. LEMAA 3.2 If h1 & h2 are tow upper hulls with at most mcan be computed in O(log 2 n) time.

y

q`

p x

H1 H2

p

H1 H2

q`

y

p

x

H1 H2

x

q. Hence we require(Since Binary

points each, their common tangent

y

q`


Proof:

1. Let a ∈ H1 & v ∈ H2 ( where <u,v> is the line of tangent)

2. Let P be an arbitrary point of H1 & q ∈ H2. (Where <p,q> is a tangent of H2)

3. Let x,y are left & right neighbors respec4. Else If <p,q> is also tangential to H1 th5. else x is to the right of P 6. else If (angle)< xpq is a left turn then x7. To search u Binary search is applied. 8. Hence we need log m time. 9. For u,v we need total time by

Log m * log m =

p

q`

tively of p ien p=u.

is to be the

O(log 2 m)

y

x

H1

n H1.

left of P

H2


Greedy Method is most straight forwarded design technique applied to a wide variety of problems with the following property. These problems have n inputs and require obtaining a sub set that satisfies some constrains. Any subset that satisfies these constrains is called a FEASIBLE SOLUTION. We need to find a feasible solution that either maximizes or minimizes a given objective function. A feasible solution that does this is called an OPTIMAL SOLUTION. Control abstraction for subset paradigm

Function Select: Selects inputs from a[] and remove it.

Variable x stores selected values.

Feasible : Boolean function

determines the inclusion of x in to solution vector.

Union : Combines x with the

solution & updates the objective function.

Knapsack Problem We are given n objects & a knapsack or bag, Object i has a weight wi and the knapsack has a capacity m. If a fraction xi, 0 <=xi , of object is placed into the knapsack ,then a profit of pixi is earned . The Objective is to obtain a filling of the knapsack that maximizes the total profit earned. Formal statement

Maximize ∑ pixi Subject to ∑ wixi < = m and 0<= xi <=1,1<=i<=n. 1<=i<=n Where pi – Profit of object i , wi – Weight of object i.

Greedy Method

Algorithm Greedy(a,n) Solution := Ø ; For i = 1 to n do x:= selection (a); if Feasible (Solution , x) then Solution:=Union (Selection ,x) return Solution;


Example: N = 3; m =20 ; (p1,p2,p3) = ( 25,24,15) and (w1,w2,w3) = (18,15,10) Some feasible solutions are : X1 X2 X3 ∑∑∑∑ wixi ∑∑∑∑piwi ½ 1/3 ¼ 16.5 24.25 1 2/15 0 20 28.5 0 2/3 1 20 31 0 1 1/2 20 31.5 In the above table all rows gives feasible solutions. Since they satisfy the criteria that total weight limit ( ∑∑∑∑ wixi <= m =20).Out of this four row gives optimal solution. Since profit (∑piwi) is maximum among all rows i.e 31.5

In this algorithm p[1:n] & w[1:n] contains the profit of n objects.

They are ordered such that p[i] / w[i] >= p[i+1] / w[i+1] m is the size of knapsack x[1:n] is the solution vector.

Demonstration: Consider the above example. Find p[i]/w[i] for i = 1 to 3 The result is (25/18, 24/15, 15/10) Now sort w[] by the previous ( order of) result . Hence w[] becomes w[1]=10; w[2]=18; w[3]= 15;

Algorithm GreedyKnapsack(m,n) for i = 1 to n do x[i] = 0.0; U :=m; for i= 1 to n do if (w[i] > U then break; x[i] := 1.0 ; U := U – w[i]; if (i< = n) then x[i] := U / w[i];

Profits Weights

Optimal solution


Now apply the w[i] for i= 1 to 3 to Greedyknapsack Algorithm. The result will be the solution vector as follows X1 = 1,x2 =1/2 ,x3 =0.

Theorem If p1/w1 > = p2/w2 >= … > = pn/wn then Greedy knapsack generates an optimal solution to the given instance of the knapsack problem Proof: Let x = (x1,x2,, … xn) be solution generated by Greedy knapsack . If all the xi = 1 then clearly the solution is optimal. Let j be the least index such that xj ! = 1 . From the algorithm it follows that xi =1 for 1<= i <= j ,xi =0 for j < i < = n and xj is 0<=x < 1 (Lemma: All Optimal Solutions will fill the knapsack exactly) From the above lemma we assume ∑∑∑∑ wiyi = m Let k be the least index such that yk ! = xk. It follows that

yk < xk .To see this ,we have the possibilities.

1. If k<j , then xk = 1 , but, yk = xk & so yk<xk 2. If k=j then ,since Σwixi=m & yi=xi for 1<= i < j

It follows that either yk < xk or Σwiyi > m 3.If k > j then Σ wiyi > m, & this is not possible.

Suppose we increase yk to xk & decrease as many of (yk+1,..yn) As necessary so that the total capacity used is still m. This result in a new solution z = (z1,…zn) with zi = xi for 1<= i <= k. and Σ wi(yi-zi) =wk(zk-yk) For z we have

Σ pizi = Σ pizi + (zk –yk) wk (pk /wk) – Σ (yi-zi) wi (pi/wi) 1<=i<=n 1<=i<=n k<=i<=n

>= Σ [ piyi + (zk-yk) wk – Σ (yi –zi) wi ] pk/wk 1<=i<=n

= Σ piyi 1<=i<=n


If Σ pizi > Σ piyi then y can not have been an optimal solution.

If these sums are equal , then either z = x & x is optimal or z != x Thus j is not optimal. Hence x is optimal. Tree Vertex Splitting

Informal Def: Given a network & a loss tolerance level , the Tree vertex splitting problem (TVSP) is to determine an optimal placement of boosters. It assumed that the boosters can only be placed in the nodes of the tree. Formal Specification of TVSP : Let T = ( V, E , w) be the weighted directed tree , V is the vertex set , E is the edge set , w is weight function.

w(i,j) is the weight of edge <i,j> є E d(p) is sum of weight on the path .d(T) is the maximum of all the path delays Let T/X be forest that result when each vertex u in x is split into two nodes ui and uo A node that gets split is correspond to a booster station. The TVSP is to identify a set x C v of minimum cardinality for which d(T/x) <= δ

A GREEDY approach to solve this problem. 1.Compute d(u) for each node u є v [ d(u ) – maximum delay from u to any other node in its sub tree ] 2.If (u has a parent v) and d(u) + w(v,u) > δ then node u gets split and d(u) is set to zero.

1

4

3

5

2 After Split -->

1

4

3o

5

23i


3.Same computations proceeds from the leaves towards the root.

Example 4 2

2 1 3 1 4 2 3 Let δ = 5 For each leaf nodes 7,8,5,9, and 10 d(u) =0 Let u be any node and c(u) is set of all its children then D(u) = max d(v) + w(u,v) v є c(u) Using this formula and the steps given before we can get the forest as follows

1

5

3

6

2

7

4

8 9 10

1

5

3

6i

2i

7

4o

8

2o

4i

6o

9 10

Signal loss from node 1 to 3


Algorithm TVS(T, δ) if (T != 0 ) then d[t] := 0; for each child v of T do TVS (v, δ) d[t] := max d[t] , d[v] + w(T,v) ; if ((T is not the root ) and (d[T] + w (Parent(T),T) > δ )) then write(T); d[T]:=0; Demonstration of TVS algorithm on the above given tree

7, 0

4 , 0

] ]

2,

T, d[T

1, 0

2,0

0

1, 0

]

7, 0

4 , 1

2,0

1,

Parent to child call

Control return tparent after endcall

T, d[T

1, 0

2,0

]

0

]

7, 0

4 , 1

2,0

1, 0

]

o its of a

d[4] value changed after the call return from its child

8 ,0

T, d[T

1, 0

n

T, d[T

T, d[T
] T, d[T
8 7, 0

4 ,4

2,0

T, d[T

Call 1 (main call)
Call 2 Call 3
Call 4

Call 4 return
Call 5 Call 5 retur
,0

1, 0


Continue the same operation for whole tree : The out put will be 4 , 2, 6. Job Sequencing with deadline Prob stmt: We are given with a set of n jobs. Associated with job i is an integer deadline di >=0 and a profit Pi > 0 . For any job i the profit Pi is earned iff the job is completed by its deadline. To complete a job one has to process the job on a machine for one unit of time. Feasible solution A Feasible solution for this problem is a subset J of jobs s.t each solution J should contains sequence of jobs that can be completed with in their deadlines. Optimal Solution: An optimal solution is a feasible solution with maximum profit

8 ,0 7, 0

4 ,0

2,0

1, 0

T, d[T]

After returning from second child d[4] will become zero because of last if stmt in the TVS algorithm. Now Write statement produces output ‘4 ‘.ie. node four has to be spitted

After the5th call return


EXAMPLE (p1,p2,p3,p4) = (100,10,15,27 ) (d1,d2,d3,d4) =(2,1,2,1) Feasible solution j Processing sequence profit 1,2 2,1 110 1,3 1,3 or 3,1 115 1,4 4,1 127 2,3 2,3 25 3,4 4,3 42 1 1 100 2 2 10 3 3 15 4 4 27 Solution in Greedy way Consider the jobs in non increasing order of the pi s. First job in the list is added in the solution vector For the remaining list every job in the list is selected in sequence and checked

with constraints (deadline). Algorithm GreedyJob ( d,J,n) // J is a set of jobs that can be completed by their deadlines. J:=1 For i := 2 to n do if (all jobs in J U i can be completed by their deadlines) then J:=J U i

HIGH LEVEL DESCRIPTION OF job sequencing algorithm

Optimal solution (maximum profit)


Optimal Storage on tapes There are n programs that are to be stored on a computer tape of length l. Associated with each program i is a tape of length li , 1 <=i <= n.

Sum of the length of all programs is at most l. Whenever a program is to be retried, the tape is initially positioned at the

front. The MRT (Mean retrieval time) is calculated by

n j

MRT= 1/n Σ tj Where tj = Σ lik

j=1 k=1

tj - Time needed to retrieve program ij Provided page are stored in the order I = i, i2…….in.

Feasible Solution All the ordering of (Permulations of) programs. Optimal solution: A particular order of programs which gives minimum MRT. Example: Let n=3 & prg lengths are (l1, l2, l3) = (5, 10, 3). Hence 3! = 6 possible ordering & their respective d values are Ordering I D(I) –sum of time needed to retrieve all prgs MRT = d(I)/3 1,2,3 (5) + (5+10) + (5+10+3) 38/3 1,3,2 (5) + (5+3) + (5+3+10) 31/3 2,1,3 (10) + (10+5) + (10+5+3) 43/3 2,3,1 (10) + (10+3) + (10+3+5) 41/3 3,1,2 (3) + (3+5) + (3+5+10) 29/3 2,3,1 (3) + (3+10) + (3+10+5) 34/3 Greedy solution. Sort the programs by their length (Shortest first) the store the programs in the sorted order. The MRT time for this arrangement is minimum. Thus Optimal; solution.

Minimum MRT time (Optimal solution)


Algorithm Store(n,m) //n is number of programs & m the number of tapes j:=0; for i:= 1 to n do Write(“append program “.i,”to permutation for tape “,j); j:=(j+1) mod m;

Theorem : If l1, <= l2, < = … ln, then the ordering ij = j, 1 <= j <= n, minimizes

n k

Σ Σ tj k=1 j=1 Over all permutations of the ij Proof:

n k n

d(I)= Σ Σ lij = Σ (n-k+1) lij k=1 j=1 k=1 If there exist a and b such that a < b an lia >lib then interchanging ia and ib results in a permutation I` with k

d(I`)= [Σ (n- k+1) lik] + (n – a + 1) lib+ (n-b+1) lia k!=a k!=b subtracting d(I`) from d(I) , we obtain d(I)- d(I`) = (n-a+1) (lia-lib ) +(n – b +1) (lib– lia) =(b-a) (lia-lib ) > 0


hence ,no permutation that is not in non decreasing order of the li’s can have minimum d.It is easy to see that all permutations in nondecreasing order of the li’s have the same d value. Hence , the ordering defined by ij =j , 1 <= j <= n , minimizes the d value.

computer algoritham for chennai univarcity unit 1 and2

Documents