it2201 algorithm design and analysis unit 5
DESCRIPTION
IT2201 Algorithm Design and Analysis UNIT 5TRANSCRIPT
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
ALGORITHM
NOTATIONS
Asymptotic Notations
Asymtotic notations are method used to estimate and represent the efficiency of an algorithm using simple formula. This can be useful for seperating algorithms that leads to different amounts of work for large inputs.
Comparing or classify functions that ignores constant factors and small inputs is called as asymtotic growth rate or asymtotic order or simply the order of functions. Complexity of an algorithm is usually represented in O, o, , O (big oh), (big omega), (big theta) notations.
Let f(n) & g(n) be any non-negative functions defined on the set of natural numbers.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
f(n) will an algorithms running time indicated by its basic operation count C(n) g(n) will be some simple function to compare the count with.
BIG - OH NOTATION (O)
This is a standard notation that has been developed to represent functions which bound the computing time for algorithms and it is used to define the worst case running time of an algorithm and concerned with very large values of N.
Definition : - A function f(n) = O(g(n)) [f(n) O(g(n))], iff there exist positive constants c & no such that f(n)cg(n) for all n, nno.
Example: The function f(n) = 8n - 2 is O(n).
Justification: By the big-Oh definition, we need to find a real constant c > 0 and an integer constant n0 >= 1 such that 8n - 2 = n0. It is easy to see that a possible choice is c = 8 and n0 = 1.
More examples: 2n + 10 is O(n), 7n - 2 is O(n), 3n3 + 20n2 + 5 is O(n3), 3log n + 5 is O(log n)
The big-Oh notation allows us to say that a function f(n) is less than or equal to another function g(n) up to a constant factor and in the asymptotic sense as n grows towards infinity. If f(n) is of the form of An + B, where A and B are constants. It's called a linear function, and it is O(n).
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
The big-Oh notation gives an upper bound on the growth rate of a function. The statement "f(n) is O(g(n))" means that the growth rate of f(n) is no more than the growth rate of g(n).
BIG - OMEGA NOTATION ()
This notation is used to describe the best case running time of algorithms and concerned with very large values of N.
Definition : - A function f(n) = (g(n)) [f(n) (g(n))], iff there exist positive constants c & no such that f(n) cg(n) for all n, nno.
Example, show that n2 is (nlogn).
BIG - THETA NOTATION ()
This notation is used to describe the average case running time of algorithms and concerned with very large values of n.
Definition : - A function f(n) = (g(n)) [f(n) (g(n))], if there exist positive constants c1,c2 & no such that c1g(n) f(n) c2g(n) for all n, nno.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Example: 3nlogn + 4n + 5logn is (nlogn).
Justification: 3nlogn
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
(little omega) f(n) is W (g(n)) if f(n) is asymptotically >g(n)
For example, let us consider sequential search ALGORITHM SequentialSearch (A[0...n-1],K) // Input : An array A[0..n-1] and a search key k. // Output : Returns the index of the first element of A that matches R or -1 if there are no matching elements.
while i < n and A[i] # k do i i + 1 if i < n return i else return - 1 Here, the best - case efficiency is 0(1) where the first element is itself the search element and the worst - case efficiency is 0(n) where the last element is the search element or the search element may not be found in the given array.
RECURSIVE ALGORITHMS:
A Recursive function is a function that is defined in terms of itself. Similarly, an algorithm is said to be recursive if the same algorithm is
invoked in the body. An algorithm that calls itself is Direct Recursive. Algorithm A is said to be Indirect Recursive if it calls another algorithm
which in turns calls A. The Recursive mechanism, are externally powerful, but even more
importantly, many times they can express an otherwise complex process very clearly. Or these reasons we introduce recursion here.
The following 2 examples show how to develop a recursive algorithms.
In the first, we consider the Towers of Hanoi problem, and in the second, we generate all possible permutations of a list of characters.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Towers of Hanoi:
.
.
.
Tower A Tower B Tower C
It is Fashioned after the ancient tower of Brahma ritual. According to legend, at the time the world was created, there was a
diamond tower (labeled A) with 64 golden disks. The disks were of decreasing size and were stacked on the tower in
decreasing order of size bottom to top. Besides these tower there were two other diamond towers
(labeled B & C). Since the time of creation, Brehman priests have been attempting to
move the disks from tower A to tower B using tower C, for intermediate storage.
As the disks are very heavy, they can be moved only one at a time. In addition, at no time can a disk be on top of a smaller disk. According to legend, the world will come to an end when the priest have
completed this task. A very elegant solution results from the use of recursion. Assume that the number of disks is n. To get the largest disk to the bottom of tower B, we move the remaining
n-1 disks to tower C and then move the largest to tower B. Now we are left with the tasks of moving the disks from tower C to B.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
To do this, we have tower A and B available. The fact, that towers B has a disk on it can be ignored as the disks larger
than the disks being moved from tower C and so any disk scan be placed on top of it.
Algorithm:
1.Algorithm TowersofHanoi(n,x,y,z) 2. //Move the top n disks from tower x to tower y. 3. { . . . 4.if(n>=1) then 5. { 6.TowersofHanoi(n-1,x,z,y); 7.Write(move top disk from tower X ,to top of tower ,Y); Towersofhanoi(n-1,z,y,x); } }
Relationship between P, NP, NP-Complete and NP-Hard
problems.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
NP-Hard problems NP-Hard are problems that are at least as hard as the hardest problems in NP. Note that NP-Complete problems are also NP-hard. However not all NP-hard problems are NP (or even a decision problem), despite having 'NP' as a prefix. That is the NP in NP-hard does not mean 'non-deterministic polynomial time'. Yes this is confusing but its usage is entrenched and unlikely to change.
NP-hard problems may be of any type: decision problems, search problems, or optimization problems Problem H is at least as hard as L, because H can be used to solve L If there is a polynomial algorithm for any NP-hard problem, then there are polynomial algorithms for all problems in NP, and hence P = NP; If P NP, then NP-hard problems have no solutions in polynomial time If an optimization problem H has an NP-complete decision version L, then H is NP-hard All optimization versions of difficult combinatorial problems are NP-Hard Very small problems can be solved by an exhaustive search algorithm, relatively small problems can be solved by dynamic programming and large problems can be solved by Branch and Bound An example of an NP-hard problem is the decision version of SUBSET-SUM which is this: Given a set of integers, does any non empty subset of them add up to zero? That is a yes/no question, and happens to be NP-complete.
INTRODUCTION TO ALGORITHM DESIGN TECHNIQUES
Algorithm Design Paradigms are the General approaches to the construction
of efficient solutions to problems.
Such methods are of interest because:
They provide templates suited to solving a broad range of diverse problems.
They can be translated into common control and data structures provided by
most high-level languages.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
The temporal and spatial requirements of the algorithms which result can be
precisely analyzed.
Although more than one technique may be applicable to a specific problem, it
is often the case that an algorithm constructed by one approach is clearly
superior to equivalent solutions built using alternative techniques.
Some of the algorithm design techniques:
Greedy Algorithms Divide and conquer Dynamic programming Backtracking Branch and bound Randomized algorithm
Greedy Algorithm
The solution is constructed through a sequence of steps, each expanding a partially constructed solution obtained so far. At each step the choice must be locally optimal this is the central point of this technique. Greedy techniques are mainly used to solve optimization problems. They do not always give the best solution.
Examples
Minimal spanning tree
Shortest distance in graphs
Greedy algorithm for the Knapsack problem
The coin exchange problem
Huffman trees for optimal encoding
We have already seen three greedy algorithms in Dijkstra's, Prim's, and Kruskal's algorithms in unit 5 Graphs. Greedy algorithms work in phases. In
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
each phase, a decision is made that appears to be good, without regard for future consequences. Generally, this means that some local optimum is chosen. This "take what you can get now" strategy is the source of the name for this class of algorithms. When the algorithm terminates, we hope that the local optimum is equal to the global optimum. If this is the case, then the algorithm is correct; otherwise, the algorithm has produced a suboptimal solution. If the absolute best answer is not required, then simple greedy algorithms are sometimes used to generate approximate answers, rather than using the more complicated algorithms generally required to generate an exact answer.
Huffman tree construction :
characters A B C D -
possibilities 0.35 0.1 0.2 0.2 0.15
Step 1 : Arrange the characters in ascending order of their possibilities or
probabilities.
0.1 B
Add the probabilities of the two least characters based on their possibilities
and then rearrange them again in ascending order
B+C= 0.1 + 0.15 =0.25
0.2 C
0.2 D
0.35 A
0.15 -
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
0.2 C
Step 2 :
0.4
Add the next two least values C & D and reaarange the characters.
0.25 0.4
Step 3 : 0.6
0.4 0.25
0.2 D
0.1 B
0.15 -
0.35 A
0.2 D
0.2 C
0.1 B
0.15 C
0.2 C
0.2 D
0.35 A
0.35 A
0.2 C
0.2 D
0.1 B
0.15 C
0.25
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
0.2
5
1 0
1 0 1 0
1 0
Finally after adding all the characters we get a tree which is the Huffmann
tree.And label all the left child nodes as 0 and the right child nodes as 1 to find
out the coding of the characters.
Step 4 :
1
0.4 0.6
A 11
B 100
C 00
D 10
Divide and conquer
Another common technique used to design algorithms is divide and conquer. Divide and conquer algorithms consist of two parts:
Divide: Smaller problems are solved recursively (except, of course, base cases).
0.2 C
0.2 D
0.35 A
0.1 B
0.15 -
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Conquer: The solution to the original problem is then formed from the solutions to the subproblems.
Divide and conquer is probably the beat known general algorithm design technique. It makes according to the following general plan. 1. A problems instance is divided into several smaller instances of the same problem,ideally of about the same size. 2. The smaller the instances are solved(typically recursively,though sometimes a different algorithm is employed when instances become small enough) 3. If necessary,the solutions obtained for smaller instances are combined to get a solution to the original problem.
Problem of
size n
Subproblem 1
of size n/2
Subproblem 2
of size n/2
Solution to subproblem
1
Solution to
subproblem 2
Solution to the
original problem
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
A problems instances of size n is divided into the instances of size n/2.More
generally ,an instances of size n can be can be divided into several instances of
size n/b,with a of them needed to be solved.(Here a &b are constances ; a>=1
and b>=1).Assuming that size n is a power of b, to simplify the analysis, we get
the following recurrence for the running time T(n).
T(n)=aT(n/b)+f(n),(general divide and conquer recurrence)
Where f(n) is a function that accounts for the time spent on dividing the
problem into smaller ones and on combining the solution.
Some of the algorithms in Divide and conquer
1.Quick sort
2.Merge sort
3.Binary search
Quick sort (partition exchange sort):
Pivot element is used to do the division process
Procedure:
1.Select the pivot from the given set of elements.
2.To divide the array pivot is selected, two index variables i and j are taken for
manipulation.
If first element =pivot, then
i=1 and j=n-1[n=no. of elements]
If last element=pivot, then
i=0 and j=n-2
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
3. The job of index variable i, is to search an element that is greater than the
pivot,so i is incremented by 1 till the value stored at at i is greater than pivot.
4. Similarly j needs to search an element that is smaller than the pivot,so, the
value j is decremented by 1 till the value stored at j is smaller than the pivot.
5. When the elements are found, they are interchanged. Again from the
current position i and j are incremented and decremented resp and exchanges
are made appropriate if desired.
6. The process ends whenever the index variables cross over or meet each
other , provided that the respective conditions are satisfied.
7. If the pivot is taken as the first element
if i and j meets each other (or)cross over do the swap between the j
th element and the pivot element.
If the pivot is taken as the last element
If i and j meet each other or cross over do the swap between the i th
element and the pivot element.
8. As a result, the whole array is divided into two parts,where all the elements
before the pivot are lesser and after the pivot are greater.
9. Now the same procedure is applied recursively for the two subarrays to get
the elements in sorted order.
NOTE:
Sometimes pivot selection will be the best case (or) the average case
(or) the worst case. It purely depends on the nature of elements.
Example:
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
n=6
0 1 2 3 4 5
56 91 35 75 48 68
Pivot i j j
Pivot
i i i j j
j
cross over(respective conditions are satisfied)
[while (a[i]pivot)
j--;]
There is a crossover between i and j, so the process ends.Make a swap
between the pivot and the j th element.
(Ele
56 48 35 75 91 68
0 1 2 3 4 5
35 48 56 75 91 68
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
ments before pivot are lesser ) (elements after the pivot are greater)
recursive quicksort
Therefore all the elements are in the sorted order.
ROUTINE:
Void main()
{
quicksort (a,0,n-1);
}
void quicksort(int a[],int low,int high)
{
int temp,i,j,pivot;
35 48 56 68 75 91
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
if(low
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
quicksort(a,low,j-1);
quicksort(a,j+1,high);
}
}
Mergesort(Divide and conquer):
It is a perfect example of the divide and conquer technique.
It sorts a given array A[0.n-1] by dividing it into two halves A[0[n/2]-
1] and A[[n/2].n-1],sorting each of them recursively,and then merging the
two smaller sorted arrays into a single sorted one.
ALGORITHM:
Mergesort(A[0n-1])
// sort array A[0n-1] by recursive mergesort
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
//input:An array A[0n-1] of orderable elements
//output:Array A[0n-1] sorted in non descending order
if n>1
copy A[0[n/2]-1] to B[0[n/2]-1]
copy A[[n/2]n-1] to C[0[n/2]-1]
mergesort(B[0[n/2]-1])
mergesort(C[0[n/2]-1])
merge(B,C,A)
Example:
Do mergesort for the following sets of elements:
8 3 13 6 2 14 5 9 10 1 7 12 4
Start doing the division, until the division is not possible.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
So after sorting the output is
1 2 3 4 5 6 7 8 9 10 12 13 14
Example 2:
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Binary Search:
Algorithm
1. Bin search(a,n,x) 2. // Given an array a[1:n] of elements in non-decreasing
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
3. //order, n>=0,determine whether x is present and 4. // if so, return j such that x=a[j]; else return 0. 5. { 6. low:=1; high:=n; 7. while (low
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
14 14 14
Found
x=-14 low high mid
1 14 7
1 6 3
1 2 1
2 2 2
2 1 Not found
x=9 low high mid
1 14 7
1 6 3
4 6 5
Found
Theorem: Algorithm Binsearch(a,n,x) works correctly.
Proof:
We assume that all statements work as expected and that comparisons such
as x>a[mid] are appropriately carried out.
Initially low =1, high= n,n>=0, and a[1]
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Otherwise, the range is narrowed by either increasing low to (mid+1) or decreasing high to (mid-1).
Clearly, this narrowing of the range does not affect the outcome of the search.
If low becomes > than high, then x is not present & hence the loop is exited.
Dynamic Programming
Its is a technique for solving problems with overlapping subproblem.
Typically,there subproblems arise from a recurrence relating a solution to a
given problem with solution to its smaller subproblems of the same type.
Dynamic programming suggests solving each smaller subproblems once and
recording the results in a table from which a solution to the original problem
can be then obtained.
Some algorithms are:
Warshalls algorithm
Floyds algorithm
Optimal binary search tree
Warshalls algorithm is used for finding the transitive closure
Floyds algorithm is used for the all-pairs shortest-paths roblems
Dynamic programming can be used for constructing an optimal binary
search tree given set of keys and known probabilities of searching for them.
WARSHALLS ALGORITHM
Warshalls algorithm is for computing the transitive closure of a directed
graph.The transitive closure of a directed graph with n vertices can be
defined as the n-by-n Boolean matrix T={tij},in which the element in the ith
row (1
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
directed path (is a directed path of a positive length) from the ith vertex to the
jth vertex;otherwise, tij is 0.
RULES
Directed matrix Adjacency matrix
0 1 0 0
0 0 0 1
0 0 0 0
1 0 1 0
TRANSITIVE CLOSURE
Warshalls algorithm constructs the transitive closure of a given
digraph with n vertices through a series of n-by-n Boolean matrices.
R(0),.R(K-1),R(K),.R(N)
0 0 1 1 1 0
1 1
a b
c d
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
R(0) is just adjacency matrix boxed row and column are used for getting R(1)
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Floyds algorithm for the all- pairs shortest path problem
Given a weighted connected graph (undirected or directed ),the all-pair shortest
path problem asks to find the distances (the length of the shortest paths)from
each vertex to all other vertices.
It is convenient to record the length of shortest path in an n-by-n matrix D
called the distances matrix : the elements dij in the ith row and the jth column of
this matrix indicates the length of the shortest path from the ith vertex to the jth
vertex (1
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
3 6 7
1
a
b
c
d
a b c d
Implements floyds algorithm for the all- pairs shortest path
problem
INPUTS: The weight matrix W of a graph
OUTPUT:The distance matrix of the shortest paths lengths
D
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
a b c d
a
b
c
d
3 6
3 2
a b c d
a
b
c
d
Distance Matrix :
The matrix indicates the length of the shortest path from the ith vertex to
the jth vertex.
Steps followed for Initial Input Weighted Matrix D0
D0 = 0 3
2 0
7 0 1
6 0
Let D1 be a matrix which contains length of the shortest path with
intermediate vertices number, not higher than one.
(i) a intermediate
b ___ a ___ c => 5
d ___ a ___ c => 9
D1 = 0 3
2 0 5
7 0 1
6 9 0
D2 is a matrix which contains length of the shortest path with
intermediate vertices numbers not higher than two.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
a b c d
a
b
c
d
a
b
c
d
a
b
c
d
a
b
c
d
a b c d
a
b
c
d
a b c d
a
b
c
d
(ii) (a,b)
D2 0 3
2 0 5 c b a => 9
9 7 0 1
6 9 0
D3 is a matrix which contains length of the shortest path with
intermediate vertices number not higher than three.
(iii) (a,b,c)
D2 = 0 10 3 4 b a c d => 6
2 0 5 6 d a c b => 16
9 7 0 1 a c d => 4
6 16 9 0 a c b => 10
D4 is a matrix which contains length of the shortest path with
intermediate vertices number, not higher than four.
(iv) (a,b,c,d)
D2= 0 10 3 4
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
2 0 5 6 c d a => 7
7 7 0 1
6 16 9 0
Optimal Binary Search Tree
Dynamic approach minimizes the average no of comparisons for a
successful search.
As an example ,consider four keys A,B,C,D to be searched for with
probabilities 0.1,0.2,0.4,0.3 respectively.
The following diagram depicts two out of 14 possible binary search trees
containing tree keys.
The average no of comparisons in a successful search in the first tree is
probability x level
0.1*1+0.2*2+0.4*3+0.3*4=2.9
While for second tree is
0.1*2+0.2*1+0.4*2+0.3*3=2.1
Neither of there two trees is optimal
We could find the optimal tree by generating all 14 binary
search trees with there keys.( total no of binary search
trees with n keys is equal to the nthcatalan number
(eg) c(n)=[2n] 1 form n>0,c(o)=1
n=4(no of key)
=14possible binary search trees .
This exhaustive search approach is unrealistic .
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
So let a1,..an be distinct keys ordered from the smallest to the
largest and let p1,..........pn be the probabilities of searching for them
Let c[i,j] be the smallest average number of comparisons made in a
successful search in a binary trees Tji made up of keys ai,,..aj,where i
,j are some integer indices,1
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
j
Let us illustrate the algorithm by applying it to the four-key set
Key A B C D
probability 0.1 0.2 0.4 0.3
The initial tables look like this
0 1 2 3 4
Main table Root table
Step 2: Do c[i,i-1]=o for 1
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
c[4,3]=0 1 < 4 < 5
c[5,4]=0 1< 5 < 5
Step 3:
Do c[i,j]=pi for 1
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Do R[i,i]=I for 1< i for 1
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
=0.5
When k=2 c[1,1]+c[3,2]+ 2s=1 ps
=0.1+0+0.3
=0.4
c[1,2]=min{when k=1 is 0.5
when k=2 is 0.4}
=0.4
Update c[1,2]=0.4 in the main table and
Update the respective k values (i.e)k=2 in the root table for
r[1,2].
1 2 2 3 4
Root table
Main table
Thus the average number of key comparisons in the optimal tree is equal
to 1.7.
Since r[1,4]=3, the root of the optimal tree contain
0 0.1
0.4
0 0.2
0 0.4
0 0.3
0
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
the third key (i.e) C.Its left subtree is made up of keys A&B. Its right
subtree contains just key D.
To find the specific structre of threse subtrees,we find first thir roots by
consulting the root table again as follows since R[1,2]=2the root n of the
optimal tree containing A&B is B, with A being itsleft child ( and the root
of the one node tree: R[1,1]=1.) Since R[4,4]=4,the root of their one node
optimal tree is its only key D.
BACKTRACKING
It is one of the most general algorithm design techniques. Many problems which deal with searching for a set of solutions or for a
optimal solution satisfying some constraints can be solved using the backtracking formulation.
To apply backtracking method, tne desired solution must be expressible as an n-tuple (x1xn) where xi is chosen from some finite set Si.
The problem is to find a vector, which maximizes or minimizes a criterion function P(x1.xn).
The major advantage of this method is, once we know that a partial vector (x1,xi) will not lead to an optimal solution that (mi+1..mn) possible test vectors may be ignored entirely.
Many problems solved using backtracking require that all the solutions satisfy a complex set of constraints.
These constraints are classified as: i) Explicit constraints.
ii) Implicit constraints.
1) Explicit constraints: Explicit constraints are rules that restrict each Xi to take values only
from a given set.
Some examples are,
Xi0 or Si = {all non-negative real nos.}
Xi =0 or 1 or Si={0,1}.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
LiXiUi or Si= {a: LiaUi}
All tuples that satisfy the explicit constraint define a possible solution space for I.
2) Implicit constraints: The implicit constraint determines which of the tuples in the
solution space I can actually satisfy the criterion functions.
Algorithm:
Algorithm IBacktracking (n)
// This schema describes the backtracking procedure .All solutions are
generated in X[1:n]
//and printed as soon as they are determined.
{
k=1;
While (k 0) do
{
if (there remains all untried
X[k] T (X[1],[2],..X[k-1]) and Bk (X[1],..X[k])) is true ) then
{
if(X[1],X[k] )is the path to the answer node)
Then write(X[1:k]);
k=k+1; //consider the next step.
}
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
else k=k-1; //consider backtracking to the previous set.
}
}
All solutions are generated in X[1:n] and printed as soon as they are determined.
T(X[1]..X[k-1]) is all possible values of X[k] gives that X[1],..X[k-1] have already been chosen.
Bk(X[1]X[k]) is a boundary function which determines the elements of X[k] which satisfies the implicit constraint.
Certain problems which are solved using backtracking method are,
1. Sum of subsets.
2. Graph coloring.
3. Hamiltonian cycle.
4. N-Queens problem.
1.SUM OF SUBSETS:
We are given n positive numbers called weights and we have to find all combinations of these numbers whose sum is M. this is called sum of subsets problem.
If we consider backtracking procedure using fixed tuple strategy , the elements X(i) of the solution vector is either 1 or 0 depending on if the weight W(i) is included or not.
If the state space tree of the solution, for a node at level I, the left child corresponds to X(i)=1 and right to X(i)=0.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Example:
Given n=6,M=30 and W(16)=(5,10,12,13,15,18).We have to generate all possible combinations of subsets whose sum is equal to the given value M=30.
In state space tree of the solution the rectangular node lists the values of s, k, r, where s is the sum of subsets,k is the iteration and r is the sum of elements after k in the original set.
The state space tree for the given problem is,
In the state space tree, edges from level i nodes to i+1 nodes are labeled with the values of Xi, which is either 0 or 1.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
The left sub tree of the root defines all subsets containing Wi. The right subtree of the root defines all subsets, which does not include
Wi.
Generation of State space tree
Maintain an array X to represent all elements in the set. The value of Xi indicates whether the weight Wi is included or not. Sum is initialized to 0 i.e., s=0. We have to check starting from the first node.
Assign X(k)
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
//generate right child and evaluate Bk.
If ((S+ r- W[k]>=m)and(S+ W[k+1]
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
The backtracking algorithm helps to find Hamiltonian cycle for any type of graph.
Procedure:
Define a solution vector X(Xi..Xn) where Xi represents the I th visited vertex of the proposed cycle.
Create a cost adjacency matrix for the given graph. The solution array initialized to all zeros except X(1)=1,bcoz the cycle
should start at vertex 1.
Now we have to find the second vertex to be visited in the cycle. The vertex from 1 to n are included in the cycle one by one by checking
2 conditions, 1.There should be a path from previous visited vertex to current vertex.
2.The current vertex must be distinct and should not have been visited earlier.
When these two conditions are satisfied the current vertex is included
in the cycle, else the next vertex is tried.
When the nth vertex is visited we have to check, is there any path from
nth vertex to first 8vertex. if no path, the go back one step and after the
previous visited node.
Repeat the above steps to generate possible Hamiltonian cycle.
Algorithm:(Finding all Hamiltonian cycle)
Algorithm Hamiltonian (k)
{
Loop
Next value (k)
If (x (k)=0) then return;
{
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
If k=n then
Print (x)
Else
Hamiltonian (k+1);
End if
}
Repeat
}
Algorithm Nextvalue (k)
{
Repeat
{
X [k]=(X [k]+1) mod (n+1); //next vertex
If (X [k]=0) then return;
If (G [X [k-1], X [k]] 0) then
{
For j=1 to k-1 do if (X [j]=X [k]) then break;
// Check for distinction.
If (j=k) then //if true then the vertex is distinct.
If ((k
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
}
3.QUEENS PROBLEM:
This 8 queens problem is to place n-queens in an N*N matrix in such a
way that no two queens attack each otherwise no two queens should be in the
same row, column, diagonal.
Solution:
The solution vector X (X1Xn) represents a solution in which Xi is the column of the th row where I th queen is placed.
First, we have to check no two queens are in same row. Second, we have to check no two queens are in same column. The function, which is used to check these two conditions, is [I, X (j)],
which gives position of the I th queen, where I represents the row and X (j) represents the column position.
Third, we have to check no two queens are in it diagonal. Consider two dimensional array A[1:n,1:n] in which we observe that
every element on the same diagonal that runs from upper left to lower right has the same value.
Also, every element on the same diagonal that runs from lower right to upper left has the same value.
Suppose two queens are in same position (i,j) and (k,l) then two queens lie on the same diagonal , if and only if |j-l|=|I-k|.
STEPS TO GENERATE THE SOLUTION:
Initialize x array to zero and start by placing the first queen in k=1 in the first row.
To find the column position start from value 1 to n, where n is the no. Of columns or no. Of queens.
If k=1 then x (k)=1.so (k,x(k)) will give the position of the k th queen. Here we have to check whether there is any queen in the same column or diagonal.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
For this considers the previous position, which had already, been found out. Check whether
X (I)=X(k) for column |X(i)-X(k)|=(I-k) for the same diagonal.
If any one of the conditions is true then return false indicating that k th queen cant be placed in position X (k).
For not possible condition increment X (k) value by one and precede d until the position is found.
If the position X (k) n and k=n then the solution is generated completely.
If kn then k th queen cannot be placed as the size of the
matrix is N*N. So decrement the k value by one i.e. we have to back track and after the
position of the previous queen.
Algorithm:
Algorithm place (k,I)
//return true if a queen can be placed in k th row and I th column. otherwise it
returns //
//false .X[] is a global array whose first k-1 values have been set. Abs
returns the //absolute value of r.
{
For j=1 to k-1 do
If ((X [j]=I) //two in same column.
Or (abs (X [j]-I)=Abs (j-k)))
Then return false;
Return true;
}
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Algorithm Nqueen (k,n)
//using backtracking it prints all possible positions of n queens in n*n
chessboard. So
//that they are non-tracking.
{
For I=1 to n do
{
If place (k,I) then
{
X [k]=I;
If (k=n) then write (X [1:n]);
Else nquenns(k+1,n) ;
}
}
}
Example: 4 queens.
Q
Q
Q
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Two possible solutions are
Solutin-1 Solution 2
(2 4 1 3) (3 1 4 2)
4. GRAPH COLORING:
Let G be a graph and m be a given positive integer. If the nodes of G can be colored in such a way that no two adjacent nodes have the same color. Yet only M colors are used. So its called M-color ability decision problem.
The graph G can be colored using the smallest integer m. This integer is referred to as chromatic number of the graph.
A graph is said to be planar iff it can be drawn on plane in such a way that no two edges cross each other.
Suppose we are given a map then, we have to convert it into planar. Consider each and every region as a node. If two regions are adjacent then the corresponding nodes are joined by an edge.
Consider a map with five regions and its graph.
Q
Q
Q
Q
Q
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
1 is adjacent to 2, 3, 4.
2 is adjacent to 1, 3, 4, 5
3 is adjacent to 1, 2, 4
4 is adjacent to 1, 2, 3, 5
5 is adjacent to 2, 4
Steps to color the Graph:
First create the adjacency matrix graph(1:m,1:n) for a graph, if there is an edge between i,j then C(i,j) = 1 otherwise C(i,j) =0.
The Colors will be represented by the integers 1,2,..m and the solutions will be stored in the array X(1),X(2),..,X(n) ,X(index) is the color, index is the node.
He formula is used to set the color is, X(k) = (X(k)+1) % (m+1)
First one chromatic number is assigned ,after assigning a number for k node, we have to check whether the adjacent nodes has got the same values if so then we have to assign the next value.
Repeat the procedure until all possible combinations of colors are found.
The function which is used to check the adjacent nodes and same color is,
1
3
5 4
A
l
g
o
r
it
h
m
m
C
o
l
o
r
i
n
g
(
k
)
2
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
If(( Graph (k,j) == 1) and X(k) = X(j))
Example:
N= 4
M= 3
Adjacency Matrix:
0 1 0 1
1 0 1 0
0 1 0 1
1 0 1 0
Problem is to color the given graph of 4 nodes using 3 colors.
Node-1 can take the given graph of 4 nodes using 3 colors.
The state space tree will give all possible colors in that ,the numbers which
are inside the circles are nodes ,and the branch with a number is the colors of
the nodes.
State Space Tree:
1
3
2
4
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Algorithm:
Algorithm mColoring(k)
// the graph is represented by its Boolean adjacency matrix G[1:n,1:n] .All
assignments //of 1,2,.,m to the vertices of the graph such that adjacent
vertices are assigned //distinct integers are printed. k is the index of the next
vertex to color.
{
repeat
{
// generate all legal assignment for X[k].
Nextvalue(k); // Assign to X[k] a legal color.
If (X[k]=0) then return; // No new color possible.
If (k=n) then // Almost m colors have been used to color the n
vertices
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Write(x[1:n]);
Else mcoloring(k+1);
}until(false);
}
Algorithm Nextvalue(k)
// X[1],X[k-1] have been assigned integer values in the range[1,m] such
that //adjacent values have distinct integers. A value for X[k] is determined in
the //range[0,m].X[k] is assigned the next highest numbers color while
maintaining //distinctness form the adjacent vertices of vertex K. If no such
color exists, then X[k] is 0.
{
repeat
{
X[k] = (X[k]+1)mod(m+1); // next highest color.
If(X[k]=0) then return; //All colors have been used.
For j=1 to n do
{
// Check if this color is distinct from adjacent color.
If((G[k,j] 0)and(X[k] = X[j]))
// If (k,j) is an edge and if adjacent vertices have the same color.
Then break;
}
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
if(j=n+1) then return; //new color found.
} until(false); //otherwise try to find another color.
}
The time spent by Nextvalue to determine the children is (mn)
Total time is = (mn n).
6.9.4.1 KNAPSACK PROBLEM USING BACKTRACKING:
The problem is similar to the zero-one (0/1) knapsack optimization problem is dynamic programming algorithm.
We are given n positive weights Wi and n positive profits Pi, and a positive number m that is the knapsack capacity, the is problem calls for choosing a subset of the weights such that,
ni
WiXi1
m and ni
PiXi1
is Maximized.
Xi Constitute Zero-one valued Vector.
The Solution space is the same as that for the sum of subsets problem. Bounding functions are needed to help kill some live nodes without
expanding them. A good bounding function for this problem is obtained by using an upper bound on the value of the best feasible solution obtainable by expanding the given live node.
The profits and weights are assigned in descending order depend upon the ratio.
(i.e.) Pi/Wi P(I+1) / W(I+1)
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Solution :
After assigning the profit and weights ,we have to take the first object weights and check if the first weight is less than or equal to the capacity, if so then we include that object (i.e.) the unit is 1.(i.e.) K 1.
Then We are going to the next object, if the object weight is exceeded that object does not fit. So unit of that object is 0.(i.e.) K=0.
Then We are going to the bounding function ,this function determines an upper bound on the best solution obtainable at level K+1.
Repeat the process until we reach the optimal solution.
Algorithm:
Algorithm Bknap(k,cp,cw)
// m is the size of the knapsack; n no.of weights & profits. W[]&P[] are
the //weights & weights. P[I]/W[I] P[I+1]/W[I+1].
//fwFinal weights of knapsack.
//fp final max.profit.
//x[k] = 0 if W[k] is not the knapsack,else X[k]=1.
{
// Generate left child.
If((W+W[k] m) then
{
Y[k] =1;
If(k fp) and (k=n)) then
{
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
fp = cp + P[k];
fw = Cw+W[k];
for j=1 to k do X[j] = Y[j];
}
}
if(Bound(cp,cw,k) fp) then
{
y[k] = 0;
if(kfp) and (k=n)) then
{
fp = cp;
fw = cw;
for j=1 to k do X[j] = Y[j];
}
}
}
Algorithm for Bounding function:
Algorithm Bound(cp,cw,k)
// cp current profit total.
//cw current weight total.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
//kthe index of the last removed item.
//mthe knapsack size.
{
b=cp;
c=cw;
for I =- k+1 to n do
{
c= c+w[I];
if (c
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
5+1
6.
Fp = (-1)
13 & 0+4 6 cw = 4,cp = 5,y(1) =1
k = k+2
23 but 7>6 so y(2) = 0
So bound(5,4,2,6)
B=5
C=4
I=3 to 3
C=6
6 6
So return 5+(1-(6-6))/(2*1)
5.5 is not less than fp. So, k=k+1 (i.e.) 3.
3=3 & 4+2 6
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
cw= 6,cp = 6, y(3)=1.
K=4.
If 4> 3 then Fp =6,fw=6,k=3 ,x(1) 1 0 1
The solution Xi 1 0 1
Profit 6
Weight 6.
BRANCH AND BOUND
Its is an algorithm design technique that enhances the idea of generating a
state-space tree with the idea of estimating the best value obtained from a
current node of the decision tree .If such an estimate is not superior to the
best solution seen up to that point in the processing, the node is eliminated
from further consideration.
Note that in the standard terminology of optimization problems, a feasible
solution is a point in the problems search space that satisfies all the
problems constraints.
(eg, a Hamiltonian circuit in the traveling saluman problem, a subset of item
whose total weight does not exceed the knapsacks capacity).
While an optimal solution is a feasible solution with the best value of the
objective function.(eg., the shortcut Hamiltonian circuit, the most valuable
subset of items that fit the knapsack).
Compared to backtracking, branch and bound required two additional items:
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
A way to provide, for every node of a state- space tree, a bound on the best
value of the objective function on any solution that can be obtained by
adding further components to the partial solution represented by the node.
The value of the best solution seen so far If this information is available, we
can compare a nodes bound value with the value of the best solution seen to
far . If the bound value is not better than the best solution seen so fa-(ie)., not
smaller for a minimization problem and not larger for a maximization
problem- the node is nonpromising and can be terminated. This is the
principal idea of the branch-and bound technique.
In general, we terminate a search path at the current node in a state-space
tree of a branch-and-bound algorithm for any one of the following three
reasons:
The value of the nodes bound is not better than the value of the best
solution seen so far.
The node represents no feasible solution because the constraints of the
problem are already violated.
The subset of feasible solution represented by the node consists of a
single point.
Branch-and- Bound algorithms
1. Assignment problem.
2. Knapsack problem.
3. Travelling saleman problem.
Assignment problem
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Let us illustrate the branch-and-bound approach by applying it
to the problem of assigning n people to n jobs so that the total cost of the
assignment is as small as possible.
Job1 job2 job3 job4
C= 9 7 8 person a
6 4 7 person b
5 8 8 person c
7 6 9 person d
Lb=2+3+1+4
Lb=10
Best- first branch- and- bound
(the lower bound value(lb) is taken into account)
We start with the root that corresponds to no elements selected from
the cost matrix.
The lower bound value for the root, denoted lb is 10.
Select one element in each row of the matrix to that no two selected
elements are in the same column and their sum is the smallest possible.
4
2
2 3
1
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Job1 job2 job3 job4
9 7 8 person a
C= 6 4 3 7 person b
5 8 8 person c
7 6 9 person d
There elements 3 and 1 are selected from the same column, but it can be
taken for the initial construction.
lb (lower bound)=2++3+1+4=10
Level 0
no of branches=4{1,2,3,4}nodes or branches.
0
Level1
1 2 3 4
Note: we have 4 live leaves (node 1 through 4)that may contain an optimal
solution.the most promising oh them is node 2 because it has the smallest
lower bound values.
start lb=2+3+1+4=10
1
4
4
4
4
4
2
2
3
a2(job2)
lb=2+3+1+4
=10
aaaaa
l
a 1(job 1)
lb=9+3+1+
4=17
a3(job3)
lb=7+4+5+4=
20
a4(job4)
lb=8+3+1+6=
18
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
=5+2+3+4
=14
( if the values selected from the same row, dont consider, go for the next
choice. If the value selected from the same colum, consider it for the initial
construction)
Note: node5 has the smallest lower bound value,so proceed with this.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
6+2+1+ 4=(13)
= 6+2+9+8
=25
If c is assigned job3 a is assigned job2
Then d is assigned job4 b is assigned job1
C is assigned job3
d is assigned job4
Note: Thus based on the smallest lower bound value , the respective branch
is extended on order to find the optimal solution for other cases.
TRAVELLING SALESMAN PROBLEM
INTRODUCTION:
It is algorithmic procedures similar to backtracking in which a new
branch is chosen and is there (bound there) until new branch is choosing for
advancing.
This technique is implemented in the traveling salesman problem [TSP]
which are asymmetric (Cij Cij) where this technique is an effective
procedure.
STEPS INVOLVED IN THIS PROCEDURE ARE AS FOLLOWS:
STEP 0: Generate cost matrix C [for the given graph g]
STEP 1: [ROW REDUCTION]
For all rows do step 2
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
STEP2: Find least cost in a row and negate it with rest of the
elements.
STEP 3: [COLUMN REDUCTION]
Use cost matrix- Row reduced one for all columns do STEP 4.
STEP 4: Find least cost in a column and negate it with rest of the elements.
STEP 5: Preserve cost matrix C [which row reduced first and then column
reduced] for the i th time.
STEP 6: Enlist all edges (i, j) having cost = 0.
STEP 7: Calculate effective cost of the edges. (i, j)=least cost in the i th
row excluding (i, j) + least cost in the j th column excluding (i, j).
STEP 8: Compare all effective cost and pick up the largest l. If two or more
have same cost then arbitrarily choose any one among them.
STEP 9: Delete (i, j) means delete ith row and jth column change (j, i) value
to infinity. (Used to avoid infinite loop formation) If (i,j) not
present, leave it.
STEP 10: Repeat step 1 to step 9 until the resultant cost matrix having
order of 2*2 and reduce it. (Both R.R and C.C)
STEP 11: Use preserved cost matrix Cn, Cn-1 C1
Choose an edge [i, j] having value =0, at the first time for a
preserved matrix and leave that matrix.
STEP 12: Use result obtained in Step 11 to generate a complete tour.
EXAMPLE: Given graph G
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
MATRIX:
1 2 3 4 5
1
2
3
4
5
PHASE I
STEP 1: Row Reduction C
25 40 31 27
5 17 30 25
19 15 6 1
9 50 24 6
22 8 7 10
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
C1 [ROW REDUCTION:
1 2 3 4 5
1
2
3
4
5
STEP 3: C1 [Column Reduction]
1 2 3 4 5
1
2
3
4
5
STEP 5:
Preserve the above in C1,
0 15 6 2
0 12 25 20
18 14 5 0
3 44 18 0
15 1 0 3
0 15 3 2
0 12 25 20
18 14 2 0
3 44 18 0
15 1 0 3
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
1 2 3 4 5
1
2
3
4
5
STEP 6:
L= (5,4)(5,3),(4,5),(3,5),(2,1),(1,2),
STEP 7:
Calculation of effective cost [E.C]
(1,2) = 2+1 =3
(2,1) = 12+3 = 15
(3,5) = 2+0 =2
(4,5) = 3+0 = 3
(5,3) = 0+12 = 12
(5,4) = 0+2 = 2
STEP 8:
L having edge (2,1) is the largest.
0 15 3 2
0 12 22 20
18 14 2 0
3 44 18 0
15 1 0 3
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
STEP 9: Delete (2,1) from C1 and make change in it as (1,2) if exists.
Now Cost Matrix =
2 3 4 5
1
3
4
5
STEP 10:
The Cost matrix 2 x 2.
Therefore, go to step 1.
PHASE II:
STEP1: C2(R, R)
2 3 4 5
1
3
4
5
15 3 2
14 2 0
44 18 0
1 0 0
13 1 0
14 2 0
44 18 0
1 0 0
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
STEP 3: C2 (C, R)
2 3 4 5
1
3
4
5
STEP 5: Preserve the above in C2
C2 =
2 3 4 5
1
3
4
5
STEP 6:
L= {(1,5), (3,5), (4,5), (5,2), (5,3), (5,4)}
STEP 7: calculation of E.C.
(1,5) = 1+0 =1
13 1 0
13 2 0
43 18 0
0 0 0
13 1 0
13 2 0
43 18 0
0 0 0
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
(3,5) = 2+0 =2
(4,5) = 18+0 =18
(5,2) = 0+13 =13
(5,3) = 0+13 =13
(5,4) = 0+1 =1
STEP 8: L having an edge (4,5) is the largest.
STEP 9: Delete (4,5) from C2 and make change in it as (5,4) =
if exists.
Now, cost matrix
2 3 4
1
3
5
STEP 10: THE cost matrix 2x2 hence go to step 1
PHASE III:
STEP 1: C3 (R, R)
2 3 4
13 1
13 2
0 0
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
1
3
5
STEP 3: C3 (C, R)
2 3 4
1
3
5
STEP 5: preserve the above in C2
STEP 6: L={(1,4), (3,4), (5,2), (5,3)}
STEP 7: calculation of E.C
(1,4)=12+0=12
(3,4)=11+0=11
(5,2)=0+11=11
(5,3)=0+12=12
STEP 8: Here we are having two edges (1,4) and (5,3) with cost = 12. Hence
arbitrarily choose (1,4)
12 0
11 0
0 0
12 0
11 0
0 0
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
STEP 9: Delete (i,j) (1,4) and make change in it (4,1) = if exists.
Now cost matrix is
2 3
2
3
STEP 10: We have got 2x2 matrix
C4 (RR)=
2 3
3
5
C4 (C, R) =
2 3
3
5
Therefore,C4 = 2 3
11
0 0
0
0 0
0
0 0
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
3
5
STEP 11: LIST C1, C2, C3 AND C4
C4 2 3
3
5
C3 2 3 4
1
3
5
C2 =
1
3
4
5
C1 =
0
0 0
0
0 0
12 0
11 0
0 0
13 1 0
13 2 0
43 18 0
0 0 0
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
1 2 3 4 5
1
2
3
4
5
TEP 12:
i) Use C4 =
2 3
3
5
Pick up an edge (I, j) =0 having least index
Here (3,2) =0
Hence, T (3,2)
Use C3 =
2 3 4
0 15 3 2
0 12 22 20
18 14 2 0
3 44 18 0
15 1 0 0
0
0 0
12 0
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
1
3
5
Pick up an edge (i, j) =0 having least index
Here (1,4) =0
Hence, T(3,2), (1,4)
Use C2=
2 3 4 5
1
3
4
5
Pick up an edge (i, j) with least cost index.
Here (1,5) not possible because already chosen index i (i=j)
(3,5) not possible as already chosen index.
(4,5) 0
Hence, T (3,2), (1,4), (4,5)
11 0
0 0
13 1 0
13 2 0
43 18 0
0 0 0
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Use C1 =
1 2 3 4 5
1
2
3
4
5
Pick up an edge (i, j) with least index
(1,2) Not possible
(2,1) Choose it
HENCE T (3,2), (1,4), (4,5), (2,1)
SOLUTION:
From the above list
32145
This result now, we have to return to the same city where we started (Here 3).
Final result:
321453
0 15 3 2
0 12 22 20
18 14 2 0
3 44 18 0
15 1 0 0
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
COST IS 15+15+31+6+7=64
6.9.5.2 KNAPSACK PROBLEM :
Knapsack problem (np-hard problems)
Given n items of know weights w1.wn of values v1vn and a knapsack
of capacity w, find the most valuable subset of the items that fit into the
knapsack.
Do the possible subsets and find the total weight that should fit into the
knapsack according to its capacity (10) and at the same time select the
highest profit (is most valuable ones)
Subset Total weight Total value
{1} 7 $42
{2} 3 $12
{3} 4 $40
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
{4} 5 $25
{1,2} 10 $36
{1,3} 11 not feasible x
{1,4} 12 not feasible x
{2,3} 7 $52
{2,4} 8 $37
{1,2,3} 14 not feasible x
{1,2,4} 15 not feasible x
{1,3,4} 16 not feasible x
{2,3,4} 12 not feasible x
{1,2,3,4} 19 not feasible x
Therefore, the items 3&4can be placed into the knapsack with more profit
Rearrange the total weight of items 3 &4 is 9 & total value is $65.
Knapsack problem (Branch and bound)
Let us now discuss how we can apply the branch and bound technique to
solve the knapsack problem.
Given n items of known weight wi and values vi , i=1,2,.n and a knapsack of
capacity w , find the most valuable subset of the items that fit in the
knapsack.
Note:
{3,4 } 9 $65
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
It is convenient to order the items of an given instance in descending order by
their value- to weight ratios
V1 > v2 > .> vn
W1 w2 wn
Ub = V + (W - w) (Vi+1 / Wi+1)
where W = capacity
w = weight
Vi+1 = value
Wi+1 = weight
1.It is natural to structure the state-space tree for this problem as a binary
tree constructed as follows.
2.Each node on the ith level of this tree, 0 < i < n, represents all the subset of
n items that include a particular selection made from the first I ordered
items.
This particular selection is uniquely determined by a path from the root to
the node: a branch going to the left indicates the inclusion of the next item
while the branch going to the right indicate its exclusion.
3.we record the total weight w and the total value v of this selection in the
node, along with some upper bound ( ub) on the value of any subject that can
be obtained by adding zero or more items to this selection .
4. A single way to compute the upper bound us is to add to v1 the total value
of the items already selected, the product of the remaining capacity of the
knapsack(W-w) & the best per unit pay off among the remaining items
,which is vi+1\wi+1.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
Not feasible. Since weight > 0
Eg :-
item 1 2 3 4
weight w=7 w=3 w=4 w=5
value v=42 v=12 v=40 v=25
Capacity weight = 10
Solution :
(3,4)
TW = 9
TV = 65
Branch and Bound Technique :
Item Weight Value Value/Weight
1 4 $40 10
Subset Total Weight
Total Value
1 7 42 2 3 42 3 4 40 4 5 25 1,2 10 54 1,3 11 82 1,4 12 67 2,3 7 52 2,4 8 37 3,4 9 65 1,2,4 15 79 1,2,3 14 94 1,3,4 16 107 2,3,4 12 117 1,2,3,4 19 119
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
1
6 5
4 3
2 1
0
1
W0
2 W 2
W0 1 W 1
8 7
1 1
1
W0
4 W 4
Wo
3
W
3
2 7 $42 6 Average in descending order
3 5 $25 5 of V/W, for items
4 3 $12 4
Capacity Weight = 10
i = 0 W=0 V=0
Ub = 0 + (10 -0)(10) = 100
W=0 ; V=0 Ub = 100
W=4 ; V=40 Ub = 76
W=0 ; V=0 Ub = 60
W=11
W=4 ; V=40 Ub = 70
W=9 ; V=65 Ub = 69
W=4 ; V=40 Ub = 64
W=12
W=9 ; V=65 Ub = 65
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
i = 1 W=4 V=40
Ub = 40 + (10-4)(6) = 76
i = 2 W = 7+4 = 11 > 10 Therefore Not feasible.
i = 2 w/0 2 W=4 V=40
Ub = 40 + (10-4)(5) = 70
i = 3 W=9 V=65
Ub = 65 + (10-9)(4) = 69
i = 3 w/o W=4
Therfore Solution is (1,3)
RANDOMIZED ALGORITHM :
Skip List
Skip Lists were invented by William Pugh in 1989. It is an alternative to BST
and other balanced trees.
Definition : A Skip list is a probabilistic data structure, where elements are
kept sorted by key. It allows quick search, insertions and deletions of
elements with simple Algorithms.
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
It is basically a linked list with additional pointers, such that intermediate
nodes can be skipped.
It uses a Random number generator to make some decision.
it randomly chooses a level for the nodes, according to the node distribution,
that is, toss a coin until you see a head, and use the no. of tosses as the node
level.
Skip List Algorithm :
Search :
Start from the highest level and move downwards to perform the
proper search.
Insertion :
Find the position, randomly select a node level and insert it.
Deletion :
Start from the highest level; move downwards; find the position;
delete it
Randomised Algorithm :
It uses random number during the Algorithm.
Why do we need Randomised Algorithm?
Algorithms sometimes suffer from bad inputs. But we do not want to
get caught. Sometimes deterministic algorithms are too slow and not even
-
WWW.VIDYARTHIPLUS.COM
WWW.VIDYARTHIPLUS.COM V+ TEAM
feasible, but randomized algorithms casn give us good results with higher
probability.
It is not possible to have true randomness. But Pseudo Randomness is
possible for more than random number generation is required.