bt 0080 fundamentals of algorithms contents · mr. deepak shetty mr. vinayak g pai assistant...

BT 0080

Fundamentals of Algorithms

Contents

Unit 1

Elementary Algorithms 1

Unit 2

Mathematical Functions and Notations 30

Unit 3

Divide and Conquer 79

Unit 4

Greedy Method 97

Unit 5

Dynamic Programming 113

Unit 6

Backtracking 123

Unit 7

Branch and Bound 138

Unit 8

Additional Features 148

Unit 9

Graphs 163

Unit 10

Trees and Algorithms 177

Edition: Spring 2009

BKID – B1092 9th

Oct. 2009

Unit 11

Traversability 206

Unit 12

Representation of Graphs 216

Unit 13

Directed Graphs 231

Unit 14

Complexity of Algorithms 250

Acknowledgement, References and Suggested Readings 266

Director & Dean Directorate of Distance Education Sikkim Manipal University of Health, Medical & Technological Sciences (SMU-DDE)

Board of Studies Dr. U. B. Pavanaja (Chairman) Mr. Nirmal Kumar Nigam General Manager – Academics HOP – IT Manipal Universal Learning Pvt. Ltd. SMU-DDE, Manipal. Bangalore.

Prof. Bhushan Patwardhan Dr. A. Kumaran Chief Academics Research Manager (Multilingual) Manipal Education Microsoft Research Labs India Bangalore. Bangalore.

Dr. Harishchandra Hebbar Mr. Ravindranath P. S. Director Director (Quality) Manipal Centre for Info. Sciences, Bangalore Yahoo India, Bangalore

Dr. N. V. Subba Reddy Dr. Ashok Kallarakkal HOD-CSE Vice President Manipal Institute of Technology, Manipal IBM India, Bangalore

Dr. Ashok Hegde Mr. H. Hiriyannaiah Vice President Group Manager MindTree Consulting Ltd., Bangalore EDS Mphasis, Bangalore

Dr. Ramprasad Varadachar Director, Computer Studies Dayanand Sagar College of Engg., Bangalore.

Content Preparation Team

Content Writing Content Editing Mr. Deepak Shetty Mr. Vinayak G Pai Assistant Professor – Mathematics Assistant Professor, Dept. of IT SMU-DDE, Manipal. SMU-DDE, Manipal.

Instructional Design Language Editing Mr. Kulwinder Pal Ms. R. K. Rao Assistant Professor – Education Lecturer - English SMU-DDE, Manipal. Upendra Pai Memorial College, Udupi.

Edition: Spring 2009

This book is a distance education module comprising a collection of learning material for our students. All rights reserved. No part of this work may be reproduced in any form by any means without permission in writing from Sikkim Manipal University of Health, Medical and Technological Sciences, Gangtok, Sikkim. Printed and published on behalf of Sikkim Manipal University of Health, Medical and Technological Sciences, Gangtok, Sikkim by Mr.Rajkumar Mascreen, GM, Manipal Universal Learning Pvt. Ltd., Manipal – 576 104. Printed at Manipal Press Limited, Manipal.

SUBJECT INTRODUCTION

The subject, Fundamentals of Algorithms(BT 0080), of four credits, contains

the various algorithms and approaches to solve problems in different

techniques. The time complexity for various algorithms is also considered

and explained in a simple manner. The different models for executing

algorithms is explained with well illustrated examples wherever necessary.

This book has been split into 14 units, the overview of which is given below:

Unit 1: Elementary Algorithms

This unit gives the different ways of expressing algorithms, the building

blocks of algorithms and their characteristics is explained in a simple

manner.

Unit 2: Mathematical Functions and Notations

This unit explains in brief the different types of functions and their notations.

The different types of Asymptotic functions and their notations is well

explained with standard examples.

Unit 3: Divide and Conquer

In this unit, we learn the divide and conquer method. Using this method, we

find the Algorithm for Binary Search, Max and Min, Merge Sort, and Quick

Sort.

Unit 4: Greedy Method

In this unit, we use the Greedy Method for Optimistic storage on tapes,

Knapsack Problem, Job Sequencing with dead lines, Optimal Merge Pattern

and Single Source Shortlist Paths.

Unit 5: Dynamic Programming

This unit deals with Dynamic Programming approach to multistage graphs,

All Pair Shortlist Paths and Travelling Salesman Problem.

Unit 6: Backtracking

In this unit, we study about Back Tracking method. In the search for

fundamental principles of algorithm design, backtracking represents one of

the most general techniques. We use Backtracking approach to solve 8-

queen problem, Sum of Subsets problem and Knapsack problem.

Unit 7: Branch and Bound

In this unit, we deal with branch and bound method and use this approach to

solve O/I Knapsack problem and Travelling Salesman problem. The Branch

and Bound refers to all state space search methods in which all children of

the E-node are generated before any other live node can become the E-

node.

Unit 8: Additional Features

In this unit, we discuss the lower boundary theory and its implications. The

concept of NP Hard and NP Complete is briefly discussed here. The

different types of Approximate Algorithms for NP Hard Problems are studied

here.

Unit 9: Graphs

In this unit, we study the Preliminary definitions of graphs. The different

Notations used in graphs are also discussed here. The concept of trees is

explained in a simple manner.

Unit 10: Trees and Algorithms

In this unit, we study the Characterization of Trees. The Rooted Trees and

its applications is discussed here with simple examples. Spanning Trees

and the algorithms is briefly discussed here with necessary steps.

Unit 11: Traversability

In this unit, we study the concept of Eulerian Graphs with application to

Hamiltonian Paths and circuits. The travelling salesman problem is briefly

discussed with application to traversability.

Unit 12: Representations of Graphs

In this unit, we discuss the concept of Adjacency and Incidence Matrix. The

application of the above concepts to Circuit Matrix and Path Matrix is

illustrated here.

Unit 13: Directed Graphs

In this unit, we study different types of Directed Graph. The concept of

Euler’s Digraphs and Matrix Representation of digraphs is illustrated with

some simple examples.

Unit 14: Complexity of Algorithms

In this unit, we discuss the Notations for the Growth Rates of Functions. We

study here the different methods of establishing NP-Completeness of

Problems.

Objectives of studying the subject:

After studying this subject you should be able to:

write the procedures in a simple manner

use a number of Mathematical Notations like , , , , mod, log, e etc

apply divide and conquer strategy to practical problems

define the Job Sequencing problem using Greedy Method

apply Dynamic Programming in Multistage graphs

analyze the 8-queen problem

solve the O/I Knapsack Problem

apply the Lower Boundary theory

use the definition of trees and properties of trees

apply the concept of trees in real life situations

identify an Euler graph

write the matrix representations of a given graph

apply the properties to flows, network and traffic problem

define a number of problems which are known to be NP-Complete

problems.

The different examples for the algorithms given in this book will be given in

the Edunxt Portal.

Fundamentals of Algorithms Unit 1

Sikkim Manipal University Page No. 1

Unit 1 Elementary Algorithms

Structure

1.1 Introduction

Objectives

1.2 Notation for Expressing Algorithms

1.3 Role and Notation for Comments

1.4 Example of an Algorithm

1.5 Problems and Instances

1.6 Characteristics of an Algorithm

1.7 Building Blocks of Algorithms

1.8 Procedure and Recursion

Procedure

Recursion

1.9 Outline of Algorithmics

1.10 Specification Methods for Algorithms

1.11 Summary

1.12 Terminal Questions

1.13 Answers

1.1 Introduction

In this unit, you are going to study about the concepts that are basically

needed to frame an elementary algorithm, which you have already

discussed in your earlier classes.

Problem may be a state of mind of a living being, of not being satisfied with

some situation. However, for our purpose, we may take the unsatisfactory /

unacceptable / undesirable situation itself, as a problem.

One way of looking at a possible solution of a problem is, as a sequence of

activities (if such a sequence exists at all), that if carried out using allowed

available tools, leads us from the unsatisfactory (initial) position to an

acceptable, satisfactory or desirable position. For example, the solution of

the problem of baking delicious pudding may be thought of as a sequence of

activities, that when carried out, gives us the pudding (the desired state)

from the raw materials that may include sugar, flour and water (constituting

the initial position) using cooking gas, oven and some utensil etc. (the tools).

The sequence of activities when carried out gives rise to a process.



Technically, the statement or description in some notation of the process is

called an algorithm, the raw materials are called the inputs and the resulting

entity (in the above case, the pudding) is called the output. In view of the

importance of the concept of algorithm, we repeat.

An algorithm is a description or statement of a sequence of activities that

constitute a process of getting the desired outputs from the given inputs.

An algorithm, when expressed in a notation that can be understood and

executed by a computer system is called a computer program or simply a

program. We should be clear about the distinction among the terms viz., a

process, a program and an algorithm.

A process is a sequence of activities actually being carried out to solve a

problem. But algorithm and programs are just description of a process in

some notation. Further, a program is an algorithm in a notation that can be

understood and be executed by a computer system.

It may be noted that for some problems and the available tools, there may

not exist any algorithm that should give the desired output. For example, the

problem of baking delicious pudding may not be solvable, if no cooking gas

or any other heating substance is available. Similarly, the problem of

reaching the moon is unsolvable, if no spaceship is available for the

purpose.

Objectives

After studying this unit, you should be able to:

use notations for expressing algorithms

write the comments wherever necessary

solve the problems and instances connected to algorithms

apply the different characteristics of an algorithm

use the different building blocks in algorithms in an effective manner

write the procedures in a simple manner

use the concept of recursion effectively in mathematical concepts

use the different outlines of algorithms wherever necessary.

1.2 Notation for Expressing Algorithms

This issue of notation for representations of algorithms will be discussed in

some detail, later. However, mainly, some combinations of mathematical



symbols, English phrases and sentences, and some sort of pseudo – high –

level language notations, shall be used for the purpose.

The symbol ‘ ’ is used for assignment. For example, x y + 3, means that

3 is added to the value of the variable y and the resultant value becomes the

new value of the variable x. However, the value of y remains unchanged.

If in an algorithm, more than one variables are required to store values of

the same type, notation of the form A [1..n] is used to denote n variables

A[1], A[2], …, A[n].

In general, for the integers m, n with m<n, A[m, n] is used to denote the variables

A[m], A[m+1], …., A[n]. However, we must note that another similar notation

A[m, n] is used to indicate the element of the matrix (or two – dimensional

array) A, which is in mth row and nth column.

1.3 Role and Notation for Comments

Comments help the human reader of the algorithm to better understand the

algorithm. In different programming languages, there are different notations

for incorporating comments in algorithms. We use the convention of putting

comments between pair of braces i.e., { }. The comments may be inserted at

any place within an algorithm. For example, if an algorithm finds roots of a

quadratic equation, then we may add the following comments, somewhere

in the beginning of the algorithm, to tell what the algorithm does.

{this algorithm finds the roots of a quadratic equation in which the coefficient

of x2 is assumed to be non – zero}.

Self Assessment Questions

1. Comments help the human reader of the algorithm to better understand

the ––––––––––––.

2. The symbol –––––––––––– is used for assignment.

1.4 Example of an Algorithm

Before going in to the detail of problem - solving with algorithms, just to have

an idea of what an algorithm is, we consider a well – known algorithm for

finding Greatest Common Divisor (G.C.D) of two natural numbers and also

mention some related historical facts. First, the algorithm is expressed in



English. Then, we express the algorithm in a notation resembling a

programming language.

Euclid‘s Algorithm for finding G.C.D of two Natural Numbers m and n

El: {Find remainder}. Divide m by n and let r be the (new) remainder

{we have 0 < r < n}

E2: {is r zero ?} If r = 0, the algorithm terminates and n is the answer.

Otherwise,

E3: {Interchange} Let the new value of m be the current value of n and the

new value of n be the current value of r. Go back to Step E1.

The termination of the above method is guaranteed, as m and n must

reduce in each iteration and r must become zero in finite number of

repetitions of steps E1, E2, and E3.

The great Greek mathematician Euclid sometimes between fourth and third

century BC, at least knew and may be the first to suggest, the above

algorithm. This algorithm is considered as the first among non – trivial

algorithms. However, the word ‗algorithm‘ itself came into usage quite late.

The word is derived form the name of the Persian mathematician

Mohammed al – Khwarizmi who lived during the ninth century A.D. The

word ‗al – khowarizmi‘ when written in Latin became ‗algorismus‘, from

which ‗algorithm‘ is a small step away.

In order to familiarize ourselves with the notation usually used to express

algorithms, next, we express the Euclid‘s algorithm in a pseudo – code

notation which is closer to a programming language.

Algorithm GCD – Euclid (m, n)

{This algorithm computes the greatest common divisor of two given positive

integer}

begin {of algorithm}

while n 0 do

begin {of while loop}

r m mod n;

{a new variable is used to store the remainder which is obtained by dividing

m by n, with 0 < r < m}



m n;

{the value of n is assigned as new value of m; but at this stage, value of n

remains unchanged}

n r,

{the value of r becomes the new value of n and the value of r remains

unchanged}

end {of while loop}

return (n)

end; {of algorithm}

Self Assessment Question

3. ―A new variable is used to store the remainder which is obtained by

dividing m by n, with 0 < r < m”. This is denoted by ––––––––––––– .

1.5 Problems and Instances

The difference between the two concepts viz., ‗problem‘ and ‗instance‘, can

be understood in terms of the following examples. An instance of a problem

is also called a question‘. We know that the roots of a general quadratic

equation.

0cbxax2 a 0 (1)

are given by the equation

a2

ac4bbx

2

(2)

Where a, b, c, may be any real numbers except the restriction that a 0.

Now, if we take a = 3, b = 4 and c = 1,

we get the particular equation

01x4x3 2 (3)

Using (2), the roots of (3) are given by

,6

24

32

13444 2

1or3

1x

With reference to the above discussion, the issue of finding roots of the

general quadratic equation ax2 + bx + c = 0 with a 0 is called a problem,



whereas the issue of finding the roots of the particular equation 3x2 + 4x + 1 = 0

is called a question or instance of the (general) problem.

In general, a problem may have a large, possibly infinite, number of

instances. The above mentioned problem of finding the roots of the

quadratic equation.

ax2 + bx + c = 0 with a 0, b and c as real numbers, has infinitely many

instances, each obtained by giving some specific real values to a, b and c,

taking care that the value assigned to a is not zero. However, all problems

may not be of generic nature. For some problems, there may be only one

instance / question corresponding to each of the problems. For example, the

problem of finding out the largest integer that can be stored or can be

arithmetically operated in a given computer is a single-instance problem.

Many of the interesting problems like the ones given below, are just single-

instance problems.

Problem (i): Crossing the river in a boat which can carry at one time, along

with the boatman only one wolf, a horse and a bundle of grass, in such a

way that neither wolf harms, nor horse eats grass. In the presence of the

boatman, neither wolf attacks horse, nor horse attempts to eat grass.

Problem (ii): The Four-Colour Problem which requires us to find out

whether a political map of the world can be drawn using only four colours,

so that no two adjacent countries get the same colour.

Coming back to the problem of finding the roots of a quadratic equation, it

can be easily seen that in finding roots of a quadratic equation, the only

operations that have been used are plus, minus, multiplication and division

of numbers along with the operation of finding out the square root of a

number Using only these operations, it is also possible through step by step

method, to find the roots of a cubic equation over the numbers, which, in

general is of the form

0dcxbxax 23

where, a 0, b, c and d are real numbers.

Further, using only the set of operations mentioned above, it is also

possible, through a step by step method to solve a biquadratic equation over

real numbers, which in general, is of the form



0edxcxbxax 234

Where, a 0, b, c, d and e are real numbers.

However, the problem of finding the roots of a general equation of degree

five or more, cannot be solved using only the operations mentioned above,

through a step by step method i.e., can not be solved algorithmically.

In such cases, we may attempt some non-algorithmic methods including

solutions based on numerical methods which may not be exact but will be

some good approximate solutions to such problems. Or we may just use

some hit and trial method, e.g. consisting of guessing a possible root and

then verifying the guess as a possible solution, by actually substituting the

guessed root in the given equation. A hit and trial method is not an

algorithm, because we cannot guarantee the termination of the method, as

discussed later. Termination is one of the characteristic properties of an

algorithm.

It may be noted that a (general) problem, like finding the roots of an

equation of degree 5 or more, may not be solvable algorithmically, i.e.,

through some step-by-step method, still it is possible for some (particular)

instances of the problem to have algorithmic solutions. For example, the

roots of the equation 032x5 are easily available through a step by step

method. Also, the roots of the equation 01x3x2 36 can be easily

found through a method, in which, to begin with, we may take 3xy .


4. Four-Colour Problem requires us to find out whether a political map of

the world, can be drawn using only ––––––––––––.

1.6 Characteristics of an Algorithm

Next, we consider the concept of algorithm in more detail. While designing

an algorithm as a solution to a given problem, we must take care of the

following five important characteristics of an algorithm.

Finiteness

An algorithm must terminate after a finite number of steps and further each

step must be executable in finite amount of time. In order to establish a



sequence of steps as an algorithm, it should be established that it

terminates (in finite number of steps) on all allowed inputs.

Definiteness (no ambiguity)

Each step of an algorithm must be precisely defined; the action to be carried

out must be rigorously and unambiguously specified for each case.

However, the method is not definite, as two different executions may yield

different outputs.

Inputs

An algorithm has zero or more but only finite, number of inputs.

Examples of algorithm requiring zero inputs:

i) Print the largest integer, say MAX, representable in the computer

system being used.

ii) Print the ASCII code of each of the letters in the alphabet of the

computer system being used.

iii) Find the sum S of the form I + 2 + 3 +……… Where S is the largest

integer less than or equal to MAX defined in example (i) above.

Outputs

An algorithm has one or more outputs. The requirement of at least one

output is obviously essential. Otherwise we can not know the answer/

solution provided by the algorithm.

The outputs have specific relation to the inputs, where the relation is by the

algorithm.

Effectiveness

An algorithm should be effective. This means that each of the operations to

be performed in an algorithm must be sufficiently basic that it can, in

principle, be done exactly and in a finite length of time, by a person using

pencil and paper. It may be noted that the ‗FINITENESS‘ condition is a

special case of ‗EFFECTIVENESS‘. If a sequence of steps is not finite, then

it cannot be effective also.

A method may be designed which is a definite sequence of actions but is

not finite (and hence not effective).

Example: If the following instruction is a part of an algorithm: Find exact

value of e using the following formula.



.....!3

1

!2

1

!1

11e and add it to x.

Then, the algorithm is not effective, because as per instruction, computation of e

requires computation of infinitely many terms of the form!n

1 for n =1, 2, 3, ……,

which is not possible/ effective.

However, the instruction is definite as it is easily seen that computation of

each of the term !n

1 is definite (at least for a given machine).


5. An algorithm has one or more ––––––––––––

1.7 Building Blocks of Algorithms

Next, we enumerate the basic actions and corresponding instructions used

in a computer system based on Von Neumann architecture. We may recall

that an instruction is a notation for an action and a sequence of instructions

defines a program whereas a sequence of actions constitutes a process. An

instruction is also called a statement.

The following three basic actions and corresponding instructions form the

basis of any imperative language. For the purpose of explanations, the

notation similar to that of a high — level programming language is used.

Basic Actions & Instructions

i) Assignment of a value to a variable is denoted by

Variable expression;

Where the expression is composed from variable and constant operands

using familiar operators like +, –, * etc.

Assignment action includes evaluation of the expression on the R.H.S.

An example of assignment instruction/ statement is

j 2 * i + j – r;

It is assumed that each of the variables occurring on R.H.S of the above

statement, has a value associated with it before the execution of the

above statement. The association of a value to a variable, whether

occurring on L.H.S or on R.H.S, is made according to the following rule.



For each variable name, say i, there is a unique location, say loc 1 (i), in

the main memory. Each location loc (i), at any point of time contains a

unique value say v (i). Thus the value v (i) is associated to variable i.

Using these values, the expression on R.H.S is evaluated. The value so

obtained is the new value of the variable on L.H.S. This value is then

stored as a new value of the variable (in this case, j) on L.H.S. It may be

noted that the variable on L.H.S (in this case, j) may also occur on R.H.S

of the assignment symbol.

In such cases, the value corresponding to the occurrence on R.H.S (of j,

in this case) is finally replaced by a new value obtained by evaluating

the expression on R.H.S (in this case, 2 * i + j – r)

The values of the other variables, viz., i and r remain unchanged due to

assignment statement.

ii) The next basic action is read values of variables i, j, etc. from some

secondary storage device, the identity of which is (implicitly) assumed here,

by a statement of the form read (i j, );

The values corresponding to variables i, j, ... in the read statement, are,

due to read statement, stored in the corresponding locations loc (i), loc(j)

...., in the main memory. The values are supplied either, by default,

through the keyboard by the user or from some secondary external

storage. In the latter case, the identity of the secondary external storage

is also specified in the read statement.

iii) The last of the three basic actions is to deliver/ write values of some

variables say i, j etc. to the monitor or to an external secondary storage

by a statement of the form

Write (i, j,….);

The values in the locations loc (i), loc(j),……. corresponding to the variables

i, j….., in the write statement are copied to the monitory or a secondary

storage. Generally, values are written to the monitor by default. In case, the

values are to be written to a secondary storage, then identity of the

secondary storage is also specified in the write statement. Further, if the

argument in a write statement is some sequence of characters enclosed

within quotes then the sequence of characters as such, but without quotes,

is given as output. For example, corresponding to the write statement.



Write (‗This equation has no real roots‘)

The algorithm gives the following output:

This equation has no real roots.

In addition to the types of instructions corresponding to the above

mentioned actions, there are other non – executable instructions which

include the ones that are used to define the structure of the data to be used

in an algorithm.

Control Mechanism and Control structures

In order to understand and to express an algorithm for solving a problem, it

is not enough to know just the basic actions viz., assignment, reads and

writes. In addition we must know and understand the control mechanisms.

These are the mechanisms by which the human beings and the executing

system become aware of the next instruction to be executed after finishing

the one currently in execution. The sequence of execution of instructions

need not be the same as the sequence in which the instruction occurs in

program text. First, we consider three basic control mechanisms or

structuring rules, before considering more complicated ones.

(i) Direct sequencing: When the sequence of execution of instructions is

to be the same as the sequence in which instruction are written in

program text, the control mechanism is called direct sequencing. Control

structure, (i.e.,) the notation for the control mechanism for direct

sequencing is obtained by writing off the instructions,

One after the other on successive lines, or even on the same line if

there is enough space on a line, and

Separated by some statement separator, say semi-colons, and in the

order of intended execution.

For example, the sequence of the next lines

A;

B;

C;

D;

denotes that the execution of A is to be followed by execution of B, to be

inturn followed by execution of C and finally by that of D.



When the composite action consisting of actions denoted by A, B, C and

D, in this order is to be treated as single component of some larger

structure, brackets such as ‗begin.... end‘ may be introduced i.e., in this

case we may use the structure

Begin A ; B ; C ; D end

Then the above is also called a (composite/ compound) statement

consisting of four (components) statement viz A, B, C and D.

(ii) Selection: In many situations, we intend to carry out some action A if

condition Q is satisfied and some other action B if condition Q is not

satisfied. This intention can be denoted by

If Q then do A else do B,

Where A and B are instructions, which may be even composite

instructions obtained by applying these structuring rules recursively to

the other instructions.

Further, in some situations B is null, i.e., if Q is false, then no action is

stated. This new situation may be denoted by

If Q then do A

In this case, if Q is true, A is executed. If Q is not true, then the

remaining part of the instruction is ignored, and the next instruction, if

any, in the program is considered for execution.

Also, there are situation when Q is not just a Boolean variable i.e., a

variable which can assume either a true or a false value only. Rather Q

is some variable capable of assuming some finite number of values say,

a, b, c, d, e, f. Further, suppose depending upon the value of Q, the

corresponding intended action is as given by following table:

Value Action

a A b A c B d NO ACTION e D f NO ACTION

The above intention can be expressed through the following notation:

Case Q of



a, b: A;

c: B;

e: D;

end;

Example: We are to write a program segment that converts % of marks

to grades as follows

% of marks (M) Grades (G)

M 80 A

60 M < 60 B

50 M < 60 C

40 M < 50 D M < 40 F

Then the corresponding notation may be:

Case M of

80 .. 100: ‘A’

60 .. 79 : ‘B’

50 .. 59 : ‘C’

40 .. 49 : ‘D’

0 .. 39 : ‘F’

Where M is an integer variable

(iii) Repetition: Iterative repetitive execution of a sequence of actions is the

basis of expressing long processes by comparatively small number of

instructions. The repeated execution of the sequence of actions has to

be terminated. The termination may be achieved either through some

condition Q or by stating in advance the number of times the sequence

is intended to be executed.

When we intend to execute a sequence S of actions repeatedly, while

condition Q holds, the following notation may be used for the purpose:

While (Q) do begins S end;

Example: We are required to find out of the sum (SUM) of first n natural

numbers. Let a variable x be used to store an integer less than or equal

to n, then the algorithm for the purpose may be of the form.



Algorithm Sum_First_N_1

begin

read (n); {assuming value of n is an integer 1}

x 1; SUM 1;

while (x<n) do ……………. ( 1)

begin

x x + 1;

SUM SUM + x

end; {of while Ioop} ………………………….. ( 1)

Write (‗The sum of the first‘, n, ‗natural numbers is‘ SUM)

end. {of algorithm}

Explanation of the algorithm Sum_First_N_1:

Initially, an integer value of the variable n is read. Just to simplify the

argument, we assume that the integer n 1. The next two statements

assign value 1 to each of the variables x and SUM. Next, we get the

execution of the while – loop. The while – loop extends from the statement

( 1) to ( 1) both inclusive. Whenever we enter the loop, the condition x < n

is (always) tested. If the condition x < n is true then the whole of the

remaining portion upto (inclusive) is executed. However, if the condition is

false then all the remaining statement of the while – loop, i.e., all statements

upto and including ( 1) are skipped.

Suppose we read 3 as value of n, and (initially) x equal 1, because of x 1.

Therefore, as 1< 3, therefore the condition x < n is true. Hence the following

portion of the while loop is executed:

begin

x x + 1;

SUM SUM + x;

end

and as a consequence of execution of this composite statement

The value of x becomes 1 and

the value of SUM becomes 3

As soon as the word end is encountered by the meaning of the while – loop,

the whole of the while – loop between ( 1) and ( 1),( including ( 1) and ( 1)

is again executed).



By our assumption n = 3 and it did not change since the initial assumption

about n; however, x has become 2. Therefore, x < n is again satisfied. Again

the rest of the while loop is executed. Because of the execution of the rest of

the loop, x becomes 3 and SUM becomes the algorithm comes to the

execution of first statement of while – loop, i.e., while (x < n) do, which tests

whether x < n. At this stage x = 3 and n = 3. Therefore, x < n is false.

Therefore, all statement upto, and including ( 1), are x < n and skipped.

Then the algorithm executes the next statement, viz, write (‗The sum of the

first‘ n, ‗natural number is‘ sum). As, at this stage, the value of SUM is 6, the

following statement is prompted, on the monitor:

The sum of the first 3 natural numbers is 6.

It may be noticed that in the statement write (‗ , ‗n‘ ‗ ‗ , SUM) the

variables n and SUM are not within the quotes and hence, the values of n

and SUM viz 3 and 6 just before the write statement are given as output.

Some variations of the ‗while ... do‘ notation are used in different

programming languages. For example, if S is to be executed at least once,

then the programming language C uses the following statement:

Do S while (Q)

Here S is called the body of the ‗do while‘ loop. It may be noted that here S

is not surrounded by the brackets, viz., begin and end. It is because of the

fact do and while enclose S.

Again consider the example given above, of finding out the sum of first n

natural numbers.

The above instruction is denoted in the programming language Pascal by

Repeat S until (not Q)

Example:

Algorithm Sum_First_N _2

Begin {of algorithm}

Read (n);

x 0; Sum 0 ……….( 2)

do

x x + 1

Sum Sum + x

While (x < n)…………… ( 2)

end {of algorithm}



If S is executed n number of times, the following notation may be used:

For v varying from I to (i + n – 1) do begin S end;

OR

For v I to (i + n – 1) do begin S end;

where v is some integer variable assuming initial value i and increasing by 1

after each execution of S and final execution of S takes place after

v assumes the value i + n – 1

Then the execution of the for-loop is terminated. Again begin S, do is called

the body of the for -loop. The variable x is called index variable of the for-loop.

Example: Again, consider the problem of finding the sum of first n natural

numbers, algorithm using ‗for ...‘ may be of the form.

algorithm Sum_First_N_3

begin

Read (n);

SUM 0

for x 1 to n do ( 3)

begin

SUM SUM + ( 3)

end:

write (‗The of the first‘, n, natural numbers‘, SUM) end {of the algorithm}

In the algorithm Sum First-N-3, there is only one statement in the body of

the for-loop. Therefore, the bracket words begin and end not will be used in

the for – loop. In this algorithm, also, it may be noted that only the variable

SUM is initialized. The variable x is not initialized explicitly. The variable x is

implicit initialized to 1 through the construct ‘for x varying from 1 to n do‘.

And, after each execution of the body of the ‗for – loop, x is implicitly

incremented by 1.

A noticeable feature of the structuring rules viz., sequencing, selection and

iteration, is that each defines a control structure with a single entry point and

single exit point. It is this property that makes them simple but powerful

building blocks for more complex control structures.


6. The variable x is called ––––––––––– of the for-loop



1.8 Procedure and Recursion

Though the above mentioned three control structure, viz., direct sequencing,

selection and repetition, are sufficient to express any algorithm, yet the

following two advanced control structures have proved to be quite useful in

facilitating the expression of complex algorithms viz.

i) Procedure

ii) Recursion

First we consider the advanced control structure procedure.

1.8.1 Procedure

Among a number of terms that are used, instead of procedure, are

subprogram and even function. These terms may have shades of

differences in their usage in different programming languages. However, the

basic idea behind these terms is the same.

It may happen that a sequence frequently occurs either in the same

algorithm or repeatedly in different parts of the algorithm or may occur in

different algorithms. In such cases, writing repeatedly the same sequence,

is a wasteful activity. Procedure is a mechanism that provides a method of

checking this wastage.

Under this mechanism, the sequence of instructions expected to be

repeatedly used in one or more algorithms, is written only once, outside and

independent of the algorithms of which the sequence could have been a

part otherwise. There may be many such sequences and hence, there is

need for an identification of each of such sequences. For this purpose, each

sequence is prefaced by statements in the following format.

Where < name >, parameter – list > and other expressions with in the

angular brackets as first and last symbols, are place – holders for suitable

values that are to be substituted in their places. For example, suppose

finding the sum of squares of two variables is a frequently required activity,

(1)

Procedure < Name > (< parameter — list >) [:< type >1]

<declarations>

<sequence of instructions expected to occur

repeatedly

end;



then we may write the code for this activity independent of the algorithm of

which it would otherwise have formed a part. And then, in (1), <name> may

be replaced by ‘sum — square’ and <parameter – list> by the two element

sequence x, y. The variables like x when used in the definition of an

algorithm are called formal parameters or simply parameters. Further,

whenever the code which now forms a part of a procedure, say sum –

square is required at any place in an algorithm, then in place of the indented

code, a statement of the form

sum – square (a, b); (2)

is written, where values of a and b are defined before the location of the

statement under (2) within the algorithm.

Further, the pair of brackets in [ : < type > ] indicates that ‘ < type >’ is

optional. If the procedure passes some value computed by it to the calling

program, then, ‘ ;< type>, is used and then <type> in (1) is replaced by the

type of the value to be passed, in this case integer.

In cases of procedure which pass a value to the calling program another

basic construct (In addition to assignment, read and write) viz., return (x) is

used, where x is a variable used for the value to be passed by the

procedure.

There are various mechanisms by which values of a and b are respectively

associated with or transferred to x and y. The variables like a and b, defined

in the calling algorithm to pass data to the procedure (i.e. the called

algorithm), which the procedure may use in solving the particular instance of

problem, are called actual parameters or arguments.

Also, there are different mechanisms by which statement of the form sum –

square (a, b) of an algorithm is associated with the code, of the procedure

for which the statement is used. However, all these mechanisms are named

as ‗calling the procedure‘. The main algorithm may be called as the ‗calling

algorithm‘ and the procedure may be called the ‗the called algorithm‘.

In order to explain the involved ideas, let us consider the following simple

examples of a procedure and a program that calls the procedure. In order to

simplify the discussion, in the following, we assume that the inputs etc., are

always of the required types only, and make other simplifying assumptions.



Example:

Procedure sum-square (a, b: integer) integer;

{denoted the inputs a and b integers and the output is also an integer}

S: integer;

{to store the required number}

begin

22 baS

Return (S)

end;

Program diagonal – Length

{the program finds lengths of diagonals of the sides of right – angled

triangles whose lengths are given as integers. The program terminates

when the length of any side is not positive integer}

L1, L2 integer; {given side lengths}

D: real;

{to store diagonal length}

read (L1, L2)

while (L1 > 0 and L2 > 0) do

begin

D square – root (sum – square (L1, L2))

Write (‗For sides of given length‘, L1, L2, ‗the required diagonal length is‘ D);

read (L1, L2)

end

In order to explain how diagonal length of a right-angled triangle is

computed by the program diagonal-Length using the procedure sum-square,

let us consider the side lengths being given as 4 and 5.

First Step: In program Diagonal-length through the statement read (L1, L2),

we read L1 as 4 and L2 as 5. As L1 > 0 and L2 > 0. Therefore, the program

enters the while-loop. Next the program, in order to compute the value of the

diagonal calls the procedure sum-square by associating with a the value of

L1 as 4 and with b the value of L2 as 5. After these associations, the

procedure sum -square takes control of the computations. The procedure

computes S as 41 = 16 + 25. The procedure returns 41 to the program. At

this point the program again takes control of further execution. The program

uses the value 41 in place of sum-square (L1, L2). The program calls the



procedure square-root, which is supposed to be built in the computer

system, which temporarily takes control of execution. The procedure

square-root returns value 41and also returns control of execution to the

program Diagonal-Length which in turn assigns this value to D and prints

the statement.

For sides of given lengths 4 and 5, the required diagonal length is 41 .

The program under while-loop again expects values of L1and L2 from the

user. If the values supplied by the user are positive integers, whole process

is repeated after entering the while loop. However, if either L1 0 (say –34)

or L2 0, then while loop is not entered and the program terminates.

We can summarize as

A procedure is a self-contained algorithm which is written for the purpose of

connecting into another algorithm, but is written independent of the

algorithm into which it may be connected.

1.8.2 Recursion

Next, we consider another important control structure namely recursion. In

order to facilitate the discussion, we recall from Mathematics, one of the

ways in which the factorial of a natural number n is defined.

factorial (1) = 1

factorial (n) = n * factorial (n – 1)

For those who are familiar with recursive definitions like the one given above

for factorial, it is easy to understand how the value of (n!) is obtained from

the above definition of factorial of a natural number. However, for those who

are not familiar with recursive definitions, let us compute 4 using the

above definition.

By definition

factorial (4) = 4 * factorial (3)

Again by the definition


Similarly,


And by definition

factorial (1) = 1



Substituting back values of factorial (1), factorial (2) etc, we get

Factorial (4) = 4.3.2.1 = 24, as desired.

This definition suggests the following procedure/ algorithm for computing the

factorial of a natural number n:

In the following procedure factorial (n), let ‗fac‘ be the variable which is used

to pass the value by the procedure factorial to a calling program. The

variable ‗fac‘ is initially assigned value 1, which is the value of factorial (1),

Procedure factorial (n)

fac; integer:

begin

fac 1

if n equals 1 then return fac

else begin

fac n * factorial (n – 1)

return (fac)

end;

end;

In order to compute factorial (n – 1), procedure factorial is called by itself,

but this time with (simpler) argument (n – 1). The repeated calls with simpler

arguments continue until factorial is called with argument 1. Successive

multiplications of partial results with 2, 3, …… up to n finally deliver the

desired result.

Definition: A procedure, which can call itself, is said to be recursive

procedure/ algorithm. For successful implementation of the concept of

recursive procedure, the following conditions should be satisfied.

i) There must be an in-built mechanism in the computer system that

supports the calling of a procedure by itself, eg., there may be an in-

built stack operations on a set of stack registers.

ii) There must be conditions within the definition of a recursive procedure

under which, after finite number of calls, the procedure is terminated.

iii) The arguments in successive calls should be simpler in the sense that

each succeeding argument takes us towards the conditions mentioned

in (ii).



In view of the significance of the concept of procedure, and specially of the

concept of recursive procedure, in solving some complex problems, we

discuss another recursive algorithm for the problem of finding the sum of

first n natural numbers, discussed earlier. For the discussion, we assume n

is a non-negative integer.

Procedure SUM (n : integer: integer)

s : integer:

If n = 0 then, return (0)

Begin s n + SUM (n – 1)

end;

end;


7. –––––––––––– is a self-contained algorithm which is written for the

purpose of connecting into another algorithm.

8. A procedure, which can call itself, is said to be –––––––––––––

1.9 Outline of Algorithmics

The problem, which has at least one algorithmic solution, is called

algorithmic or computable problem. Also, we should note that there is no

systematic method (i.e., algorithm) for designing algorithms even for

algorithmic problems. In fact, designing an algorithm for a general

algorithmic/ computable problem is a difficult intellectual exercise. It requires

creativity and insight and no general rules can be formulated in this respect.

As a consequence, a discipline called algorithmics has emerged that

comprises large literature about tools, techniques and discussion of various

issues like efficiency etc. related to the design of algorithms. In the rest of

our study, we shall be explaining and practising algorithms. We see below

some well-known techniques which have been found useful in designing

algorithms.

i) Divide and conquer

ii) Dynamic programming

iii) The greedy approach

iv) Bracketing

v) Branch and bound

vi) Searches and traversals.



In view of the difficulty of solving algorithmically even the computationally

solvable problems, some of the problem types, enumerated below, have

been more rigorously studied

i) Sorting problems

ii) Searching problems

iii) Linear - programming problems

iv) Number - theory problems

v) String processing problems

vi) Graph problems

vii) Geometric problems

viii) Numerical problems.

Understanding the problem

Understanding allows appropriate action. This step forms the basis of the

other steps to be discussed. For understanding the problem, we should read

the statement of the problem, if required, a number of times. Then we

should find out

i) The type of problem, so that if a method of solving problems of the type,

is already known, then the known method may be applied to solve the

problem under consideration.

ii) The type of inputs and the type of expected/ desired outputs, specially,

the illegal inputs, i.e., inputs which are not acceptable, are characterized

at this stage. For example, in a problem of calculating income-tax, the

income cannot be non-numeric character strings. The range of inputs,

for those inputs which are from ordered sets. For example, in the

problem of finding whether a large number is prime or not, we cannot

give as input a number greater than the maximum number that the

computer system used for the purpose can store and arithmetically

operate upon. For still larger numbers, some other representation

mechanism has to be adopted.

iii) Special cases of the problem, which may need different treatment for

solving the problem. For example, if for an expected quadratic equation

ax2 + bx + c = 0, a the coefficient of x2, happened to be zero then usual

method of solving quadratic equations, cannot be used for the purpose.



Analyzing the problem

This step is useful in determining the characteristics of the problem under

consideration, which may help in solving the problem. Some of the

characteristics in this regard are discussed below:

i) Whether the problem is decomposable into independent smaller or

easier sub problems, so that programming facilities like procedure and

recursion etc. may be used for designing a solution of the problem. For

example, the problem of evaluating

dxxcosxsinx5 222

can be decomposed into smaller and simpler problem viz.,

dxxcosxSinanddxx5 222

ii) Whether steps in a proposed solution or solution strategy of the

problem, may or may not be ignorable, recoverable or inherently

irrecoverable, i.e., irrecoverable by the (very) nature of the problem.

Depending upon the nature of the problem, the solution strategy has to

be decided or modified.

For example,

a) While proving a theorem, if an unrequired lemma is proved, we may

ignore it. The only loss is the loss of efforts in proving the lemma.

Such a problem is called ignorable step problem.

b) Suppose we are interested in solving 8 — puzzle problem of

reaching from some initial state, say,

2 8 7

1 3 5

6 4

To some final state, say,

1 2 3

8 4

7 6 5



By sliding, any one of the digits from a cell adjacent to the blank cell,

to the blank cell. Then a wrong cannot be ignored but has to be

recovered. By recoverable, we mean that we are allowed to move

back to the earlier state from which we came to the current state, if

current state seems to be less desirable than the earlier state. The 8-

puzzle problem has recoverable steps, we may say the problem is a

recoverable problem.

c) However if, we are playing chess, then a wrong step may not be

even recoverable. In other words, we may be in a position, because

of the adversary‘s move back to earlier state. Such a problem is

called an irrecoverable step problem.

Depending on the nature of the problem as ignorable-step,

recoverable-step or irrecoverable-step problem, we have to choose

out tools, techniques and strategies for solving the problem.

For example, for ignorable-step problem, simple control structures

for sequencing and iteration may be sufficient. However, if the

problem additionally has recoverable-step possibilities then facilities

like back-tracking, as are available in the programming language

PROLOG, may have to be used. Further, if the problem additionally

has irrecoverable-step possibilities then planning tools should be

analyzed in advance to find out where the sequence may lead to,

before the first step is actually taken.

There are many other characteristics of a problem viz,

Whether the problem has certain outcome or uncertain outcome

Whether a good solution is absolute or relative

Whether the solution is a state or a path

Whether the problem essentially requires interaction with the

user etc.

Which can be known through analyzing the problem under

consideration, and the knowledge of which, in turn, may help us in

determining or guessing a correct sequence of actions for solving the

problem under consideration.



Capabilities of the computer system

We know that because of change in computational capabilities, a totally

different algorithm has to be designed to solve the problem (eg. that of

multiplying two natural numbers)

Most of the computer systems used for educational purposes are PCs

based on Von-Neumann architecture. Algorithms, that are designed to be

executed on such machines are called sequential algorithms.

However, new powerful machines based on parallel distributed

architectures, are also increasingly becoming available. Algorithms, that use

such additional facilities, are called parallel distributed; such parallel

distributed algorithms, may not have much resemblance to the

corresponding sequential algorithms for the same problem.

Approximate vs Exact Solution

For some problems like finding the square root of a given natural number n,

it may not be possible to find exact value for all n‘s (eg., n = 3). We have to

determine in advance what approximation is acceptable, eg., in this case,

the acceptable error may be, say, less than 01.

Also, there are problems, for which finding the exact solutions may be

possible, but the cost may be too much.

In the case of such problems, unless it is absolutely essential, it is better to

use an alternative algorithm which gives reasonably approximate solution,

which otherwise may not be exact. For example, consider the Traveling

Salesperson Problem: A salesperson has a list of, say n cities, each of

which he must visit exactly once. There are direct roads between each pair

of the given cities. Find the shortest possible route that takes the

salesperson on the round trip starting and finishing in any one of the n cities

and visiting other cities exactly once.

In order to find the shortest paths, one should find the cost of covering each

of the n! different paths covering the n given cities. Even for a problem for

visiting 10 cities, n!, the number of possible distinct paths is more than 3

million. In a country like India, a traveling salesperson may be expected to

visit even more than 10 cities. To find out exact solution in such cases,

though possible, is very time consuming. In such case, a reasonably good

approximate solution may be more desirable.




9. In order to find the shortest paths, one should find the cost of covering

each of the ––––––––––– different paths covering the n given cities

1.10 Specification Methods for Algorithms

An algorithm is a description statement of a sequence of activities that

constitute a process of getting desired output from the given inputs. Such

description or statement needs to be specified in some notation or

language. We briefly mentioned about some possible notations for the

purpose. Three well — known notations/ languages used mostly for the

purpose, are enumerated below.

i) Natural Language (NL): An NL is highly expressive in the sense that it

can express algorithms of all types of computable problems. However,

main problem with an NL is the inherent ambiguity, i.e., a statement or

description in NL may have many meaning, which, may be unintended

and misleading.

ii) A Pseudo code notation is a sort of dialect obtained by mixing some

programming language constructs with natural language descriptions of

algorithms. The pseudo-code method of notation is the frequently used

one for expressing algorithms. However, there is no uniform/ standard

pseudo-code notation used for the purpose, though, most pseudo-code

notations are quite similar to each other.

iii) Flow chart is a method of expressing algorithms by a collection of

geometric shapes with imbedded descriptions of algorithmic steps.

However, the technique is found to be too cumbersome, specially, to

express complex algorithms.

Analyzing an Algorithm

Generally, the resources that are taken into consideration for analyzing

algorithms include

i) Time expected to be taken in executing the instances of the problem

generally as a function of the size of the instance.

ii) Memory space expected to be required by the computer system, in

executing the instances of the problem, generally as a function of the

size of the instances.



iii) Also sometimes, the man-hour or man-month taken by the team

developing the program, is also taken as a resources for the purpose.


10. A ––––––––––––– notation is a sort of dialect obtained by mixing some

programming language constructs with natural language descriptions of

algorithms

1.11 Summary

In this unit we have studied

If in an algorithm, more than one variables are required to store values

of the same type, notation of the form A [1..n] is used to denote n

variables A[1], A[2], …, A[n]

The comments may be inserted at any place within an algorithm

The difference between the two concepts viz., ‗problem‘ and ‗instance‘

The characteristics of an algorithm is very important to design it

The following two building blocks

i) Basic actions and instructions

ii) Control Mechanism and control structure

are very much necessary to frame an algorithm

The following two advanced control structures

i) Procedure

ii) Recursion

have proved to be quite useful in facilitating the expression of

complex algorithms

The different outline of Algorithms which is very much necessary to

solve a problem effectively

The different specification methods and the resources that are taken into

consideration for analyzing algorithms


1. What is the use of comments in an Algorithm?

2. Briefly explain the concept of ―Problems and Instances‖

3. Give the different characteristics of an algorithm

4. Explain the different control mechanisms and control structures in an

Algorithm



5. Explain the concept of procedure in a Complex Algorithm

6. Explain the concept of ―Analyzing the Problem‖ in Algorithms

7. Explain ―Specification Methods‖ in an Algorithm

8. What do you mean by ―Recursion in Algorithms‖? Explain?

1.13 Answers


1. Algorithm

2.

3. r m mod n

4. Four colours

5. Outputs

6. Index varialbe

7. Procedure

8. Recursive procedure/algorithm

9. n!

10. Pseudo code

Terminal Questions

1. Comments help the human reader of the algorithm to better understand

the algorithm. In different programming languages, there are different

notations for incorporating comments in algorithms. (For details refer

section 1.3)

2. A (general) problem, like finding the roots of an equation of degree 5 or

more, may not be solvable algorithmically, i.e., through some step-by-

step method, still it is possible for some (particular) instances of the

problem to have algorithmic solutions. (For details refer to section 1.5)

3. Refer to section 1.6








Unit 2 Mathematical Functions and Notations Structure

2.1 Introduction

Objectives

2.2 Functions & Notations

2.3 Modular Arithmetic/Mod Function

2.4 Mathematical Expectation in Average Case Analysis

2.5 Efficiency of an Algorithm

2.6 Well Known Asymptotic Functions & Notations

2.7 Analysis of Algorithms – Simple Examples

2.8 Well known Sorting Algorithms

Insertion sort

Bubble sort

Selection sort

Shell Sort

Heap Sort

2.9 Summary


2.11 Answers

2.1 Introduction

You know that number of mathematical and statistical tools, techniques and

notations form an essential part in the concept of analysis of algorithms. In

this unit, we will study a number of well known approximation functions.

These functions will calculate approximate values of quantities, and these

quantities are useful in many situations and among these some of the

quantities are calculated just for comparison with each other.

Objectives


use a number of Mathematical Notations like , , , , mod, log, e etc.

apply concepts of Mathematical Expectation in Algorithms

use of principle of Mathematical Induction in establishing truth of

infinitely many Statements

analyze the efficiency of an Algorithm

use Asymptotic Functions and Notations

apply different techniques in writing algorithms.



2.2 Functions & Notations

To start with, we recall the following notations and definitions.

Here,

N = {1, 2, 3, ….}

I = {…, –2, –1, 0, 1, 2, …..}

R = set of Real numbers.

Notation: If a1, a2, …. an are n real variables/ numbers then

(i) Summation

The expression a1 + a2 + …..+ ai + ….+ an may be denoted in shorthand

as n

1i

ia

(ii) Product

The expression a1 a2 …. x ai ….. an is denoted as n

1i

ia

Function

For two given sets A and B (which need not be different , i.e., A may be the

same as B) a rule f which associates with each element of A, a unique

element of B, is called a function from A to B. If f is a function from a set A to

a set B, then we denote it by f: A B. Also, for x A, f(x) is called image of

x in B. Then, A is called the domain of f and B is called the co domain of f.

Example:

Let f: I I be defined such that f(x) = x2 for all x I then

f maps 4 to 16

f maps 0 to 0

f maps 5 to 25

Remark

We may note the following:

i) If f: x y is a function, then there may be more than one element, say

x1 and x2 such that f(x1) = f(x2)

In the above example

f(2) = f(–2) = 4

By putting restriction that f(x) f(y) if x y, we get special functions,

called 1 – 1 or injective functions.



ii) Though for each element x X, there must be at least one element

y Y, such that f(x) = y. However it is not necessary that for each

element y Y there must be at least one element x X, such that

f(x) = y, For example, for y = – 3 Y there is no x X such that

f(x) = x2 = –3. By putting the restriction on the function f, such that for

each y Y, there must be at least one element x of X such that f(x) = y,

we get special functions called onto or surjective functions.

Definition of some functions:

1 – 1 or Injective Function: A function f: A B is said to be 1 – 1 or

injective function if for x, y A, if f(x) = f(y) then x = y

We have already seen that the function defined in above example is not

1 – 1. However, by changing the domain, though defined by the same rule,

f becomes a 1 – 1 function.

Example: In this particular case, if we change the domain to

N = {1, 2, 3 …..}, then we can easily check that function.

f: N N defined as f(x) = x2 for all x N is 1 – 1 because, in this case, for

each x N its negative –x N. Hence for f(x) = f(y) implies x = y. For

example, If f(x) = 4 then there is only one value of x = 2 such that f(2) = 4.

Onto/surjective function: A function f: X Y is said to be onto, or

subjective if to every element of, the co domain of f, there is an element

x X such that f(x) = y.

We have already seen that the function defined in above example is not

onto.

However, by changing the co domain y or changing the rule, (or both) we

can make f as onto.

Example: (Changing the domain)

Let X = I = {….. –3, –2, –1, 0, 1, 2, 3 ……} but, if we change Y as

Y = {0, 1, 4, 9 …..} = {y | y = n2 for n X }

Then it can be seen that f: X Y defined by f(x) = x2 for all x X is onto

Example: (changing the rule)

Here, we change the rule so that X = Y = { …….–3, –2, –1, 0, 1, 2, 3 …….}

But f: X Y is defined as f(x) = x + 3 for x X.



Then we apply the definition to show that f is onto

If y Y, then, by definition, for f to be onto, there must exist an x X such

that f(x) = y. So the problem is to find out x X such that f(x) = y .

Let us assume that x X exists such that f(x) = y

i.e., x + 3 = y

i.e., x = y – 3

But, as y is given, x is known through the above equation. Hence f is onto.

Monotonic Functions: For the definition of monotonic functions, we

consider only functions

f: R R where, R is the set of real numbers.

A function f: R R is said to be monotonically increasing if for x, y R and

x y we have f(x) f(y)

Further, f is said to be strictly monotonically increasing if x < y then

f(x) < f(y)

Example:

Let f: R R be defined as f(x) = x + 3, for x R

Then, for x1, x2 R, the domain, if x1 x2 then x1 + 3 x2 + 3, (by using

monotone property of addition), which implies f(x1) f(x2). Hence, f is

monotonically increasing.

A function f: R R is said to be monotonically decreasing if, for x, y R

and x y then

f(x) f(y)

In other words, as x increases, value of its image decreases.

Further, f is said to be strictly monotonically decreasing, if x < y then

f(x) > f(y).

Example:

Let f: R R be defined as

f(x) = – x + 3

If x1 x2 then –x1 –x2 implying – x1 + 3 –x2 + 3 which further implies

that f(x1) f(x2) hence, f is monotonically decreasing.



Suppose that a computer can execute one billion steps per second. Table

2.1 shows the time that the computer takes to execute f(n) steps. Note that

1 s = 10–6 Seconds and 1 ms = 10–3 seconds.

Table No. 2.1: Table showing the time

N F(n) = n f(n) = log2n f(n) = nlog2n f(n) = n2

f(n) = 2n

10 0.01 s 0.003 s 0.033 s 0.1 s 1 ms

20 0.02 s 0.004 s 0.086 s 0.4 s 1 ms

30 0.03 s 0.005 s 0.147 s 0.9 s 1 sec

50 0.05 s 0.006 s 0.282 s 2.5 s 13 days

100 0.10 s 0.007 s 0.664 s 10 s 4 – 1013

years

106 1 ms 0.020 s 19.93 s 16.7 min

Floor Function: maps each real number x to the integer, which is the

greatest of all integers less than or equal to x. Then the image of x is

denoted by x

For example: 2.5 = 2, –2.5 = –3, 6 = 6

Ceiling Function: maps each real number x to the integer, which is the

least of all integers greater than or equal to x. Then the image of x is

denoted by x .

For example: 2.5 =3 –2.5 = –2, 6 = 6

Note: For every real number x, we have

x – 1 < x x x < x + 1

Example:

Each of the floor function and ceiling function is a monotonically increasing

function but not strictly monotonically increasing function. Because, for real

numbers x and y.

If x y then y = x + k for some k 0.

y = x + k = integer part of (x + k) integer part of x = x



Similarly,

y = x + k = least integer greater than or equal to x + k least integer

greater than or equal to x = x

But, each of floor and ceiling function is not strictly increasing, because

2.5 = 2.7 = 2.9 = 2 and 2.5 = 2.7 = 2.9 = 3


1. A function f: R R is said to be monotonically increasing if for x, y R

and x y we have ––––––––––––.

2. –––––––––––––– maps each real number x to the integer, which is the

least of all integers greater than or equal to x.

2.3 Modular Arithmetic/ Mod Function

Consider the following

i) If, we are following 12 – hour clock, and if it is 11 O‘clock now then after

3 hours, it will be 2 O‘clock and not 14 O‘clock (whenever the number of

o‘clock exceeds 12, we subtract n = 12 from the number)

ii) If, it is 5th day (i.e., Friday) of a week, after 4 days, it will be 2nd day

(i.e., Tuesday) and not 9th day, of course of another week (whenever the

number of the day exceeds 7, we subtract n = 7 from the number, we

are taking here Sunday as 7th day. Instead of 0th day).

iii) If, it is 6th month (i.e., June) of a year, then after 8 months, it will be

2nd month (i.e., February) of course another year (whenever, the number

of the month exceeds 12, 23 subtract n = 12) Thus, we have

Definitions

b mod n: If n is a given positive integer and b is any integer, then

b mod n = r where 0 r < n and b = k * n + r

In other words, r is obtained by subtracting multiplies of n so that the

remainder r lies between 0 and (n – 1).

For example: If b = 42 and n = 11 then

b mod n = – 42 mod 11 = 2 (– 42 = (–4) 11 + 2)



Mod function can also expressed in terms of the floor function as follows:

b(mod n) = nn

bb

Factorial: For N = {1, 2, 3, ….}, the factorial function

factorial: 0N0N

given by n = n. n – 1 = n factorial (n – 1)

Exponentiation Function (Exp): is a function of two variables x and n

where x is any non – negative real number and n is an integer (though n can

be taken as non integer also, but we restrict to integers only)

Exp (x n) denoted by xn, is defined recursively as follows:

For n = 0

Exp (x, 0) = x0 = 1

For n > 0

Exp (x, n) = x. Exp (x, n – 1)

i.e.,

xn = x . xn–1

For n < 0, let n = –m where m > 0

m

mn

x

1xx

In xn, n is also called the exponent/ power of x.

For example: If x = 1.5, n = 3, then

Also, Exp (1.5, 3) = (1.5)3 = (1.5) [(1.5)2] = [(1.5) [1.5 (1.5)1]

= 1.5 [(1.5 (1.5 (1.5)0))]

= 1.5 [(1.5 (1.5 1))] = 1.5 [(1.5 1.5)]

1.5 [2.25] = 3.375

Exp (1.5, – 3) = (1.5)–3 = 375.3

1

5.1

13

Further, the following rules apply to exponential function.



mnnmbb

mnnm bb

nmnm bb.b

For b 1 and for all n, the function bn is monotonically increasing in n. In

other words, if n1 n2 then .1bifbb 2n1n

Polynomial: A polynomial in n of degree k, where k is a non – negative

integer, over R, the set of real numbers, denoted by P(n), is of the form

011k

1kk

kk ana......nananP

Where, 0ak and ,Rai i = 0, 1, …….,K

Using the summation notation

Ra,0ananP ik

k

0i

iik

Each of ii na is called a term.

Generally, the suffix k in Pk (n) is dropped and instead of Pk(n) we write P(n)

only.

We may note that P(n) = nk = 1. nk for any k, is a single term polynomial. If

k 0 then P(n) = nk is monotonically increasing. Further, if k 0 then

p(n) = nk is monotonically decreasing.

Result: Though 0 = 1. The following is a very useful result relating the

exponentials and polynomials.

For any constants b and c with b > 1

0b

nLim

n

c

n

The result, in non – mathematical terms, states that for any given constants

b and c, but with

b > 1, the terms in sequence ,........b

k,.....,

b

3,

b

2,

b

1k

c

3

c

2

c

1

c



gradually decrease and approaches zero. Which further means that for

constants b and c, and integer variable n, the exponential term bn, for b > 1,

increases at a much faster rate than the polynomial term nc.

Exponential Function: The letter e is used to denote the quantity

......!3

1

!2

1

!1

11

and is taken as the base of natural logarithm function, then for all real

numbers x, we define the exponential function

0i

i32x

!i

x.........

!3

x

!2

xx1e

For all real numbers, we have

Further, if | x | 1 then 2x xx1ex1

The following is another useful result, which we state without proof:

Result: xn

ne

n

x1Lim

Logarithm: The concept of logarithm is defined indirectly by the definition of

exponential defined earlier. If a > 0, b > 0 and c > 0 are three real numbers,

such that

c = ab

Then clogb a (read as log of c to the base a)

Then a is called the base of the logarithm.

For example: if 26 = 64, then log2 64 = 6,

i.e., 2 raised to power 6 gives 64.

Generally, two bases, viz, 2 and 3 are very common in scientific and

computing fields and hence, the following special notations for these bases

are used.

(i) Ig n denotes log2n n (base 2)

(ii) In n denoted logen (base e);

Where the letter n denotes logarithm and the letter n denotes natural.

Result: Given below are some important properties of logarithms without

proof.



For n, a natural number and real numbers a, b and c all greater than 0, the

following identities are true

i) clogblogbclog aaa

ii) blognblog an

a

iii) blogb

1log aa

iv) alog

1blog

ba

v) ablogcblog

ca


3. ––––––––––––– is a function of two variables x and n where x is any

non – negative real number and n is an integer.

4. 0i

i32

!i

x.........

!3

x

!2

xx1

2.4 Mathematical Expectation in Average Case Analysis

The concept of mathematical expectation is needed in average – case

analysis of algorithms. In order to understand the concept better, consider

the following example.

Example: Suppose, the students of MCA, who completed all the courses in

the year 2005, had the following distribution of marks:

Range of marks Percentage of students who

scored in the range

0% to 20% 08

20% to 40% 20

40% to 60% 57

60% to 80% 09

80% to 100% 06

If a student is picked up randomly from the set of students under

consideration, what is the % of marks expected of such a student? After

scanning the table given above, we intuitively expect the student to score



around the 40% to 60% class, because, more than half of the students have

scored marks in and around this class.

Assuming that marks within a class are uniformly scored by the students in

the class, the above table may be approximated by the following more

compact table.

% marks Percentage of students

scoring the marks

10* 08

30 20

50 57

70 09

90 06

As explained earlier, we expect a student picked up randomly, to score

around 50% because more than half of the students have scored marks

around 50%

This informal idea of expectation may be formalized by giving to each

percentage of marks, weight in proportion to the number of students scoring

the particular percentage of marks in the above table.

Thus, we assign weight 100

8 to the score 10% (8, out of 100 students

score on the average 10% marks); 100

20 to the score 30% and so on.

Thus,

Expected % of marks

47100

690

100

970

100

5750

100

2030

100

810

The final calculation of expected marks of 47 is roughly equal to our intuition

of the expected marks, according to our intuition, to be around 50.

We generalize and formalize these ideas in the form of the following

definition.



Mathematical Expectation

For a given set S of items, let to each item, one of the n values, say

v1, v2 ….., vn, be associated. Let the probability of occurrence of an item with

value vi be pi. If an item is picked up at random, then its expected value E(v)

is given by

nn2211i

n

1i

i vp.....vpvpvp)v(E


5. The concept of –––––––––– is needed in average – case analysis of

algorithms

2.5 Efficiency of an Algorithm

If a problem is algorithmically solvable then it may have more than one

algorithmic solutions. In order to choose the best out of the available

solutions, there are criteria for making such a choice. The

complexity/efficiency measures or criteria are based on requirement of

computer resources by each of the available solutions. The solution which

takes least resources is taken as the best solution and is generally chosen

for solving problem. However, it is very difficult to even enumerate all

possible computer resources e.g., time taken by the designer of the solution

and the time taken by the programmer to encode the algorithm.

Mainly, the two computer resources taken into consideration for efficiency

measures, are time and space requirements for executing the program

corresponding to the solution/algorithm. We will restrict to only time

complexities of algorithms of the problems.

In order to understand the complexity/efficiency of an algorithm, it is very

important to understand the notion of the size of an instance of the problem

under consideration and the role of size in determining complexity of the

solution.

It is easy to realize that given an algorithm for multiplying two n x n matrices,

the time required by the algorithm for finding the product of two

2 x 2 matrices, is expected to take much less time than the time taken by

the same algorithm for multiplying, say, two 100 x 100 matrices. This

explains intuitively the notion of the size and instances of a problem and



also the role of size in determining the (time) complexity of algorithm. If the

size of general instance is n then time complexity of the algorithm solving

the problem under consideration is some function of n.

In view of the above explanation, the notion of size of an instance of a

problem plays an important role in determining the complexity of an

algorithm for solving the problem under consideration. However, it is difficult

to define precisely the concept of size in general, for all problems that may

be attempted for algorithmic solutions.

However, for all types of problems, this does not serve properly the purpose

for which the notion of size is taken into consideration. Hence, different

measures of size of an instance of a problem are used for different types of

problem. For example,

i) In sorting and searching problems, the number of elements, which are to

be sorted or are considered for searching, is taken as the size of the

instance of the problem of sorting/ searching.

ii) In the case of solving polynomial equations i.e., dealing with the algebra

of polynomials, the degrees of polynomial instances, may be taken as

the sizes of the corresponding instances.

There are two approaches for determining complexity (or time required) for

executing an algorithm, viz.,

i) empirical (or a posterior)

ii) theoretical (or priori)

In the empirical approach (the programmed) algorithm is actually executed

on various instances of the problem and the size (s) and time (t) of

execution for each instance is noted. And then by some numerical or other

technique, t is determined as a function of s. This function then, is taken as

complexity of the algorithm under consideration.

In the theoretical approach, we mathematically determine the time needed

by the algorithm, for a general instance of size, say, n of the problem under

consideration. In this approach, generally, each of the basic instructions like

assignment, read and write and each of the basic operation like ‘+’,

comparison of pair of integers etc. is assumed to take one or more, but

some constant number of, (basic) units of time for execution. Time for



execution for each structuring rule is assumed to be some function of the

time required for constituent of the structure.

Thus starting from the basic instructions and operations and using

structuring rules, we can calculate the time complexity of a problem or an

algorithm.

The theoretical approach has a number of advantages over the empirical

approach as listed below:

i) The approach does not depend on the programming language in which

the algorithm is coded and on how it is coded in the language.

ii) The approach does not depend on the computer system used for

executing (a programmed version of) the algorithm.

iii) In case of a comparatively inefficient algorithm, which ultimately is to be

rejected, the computer resources and programming efforts which

otherwise would have been required and wasted, will be saved.

iv) Instead of applying the algorithm to many different – sized instances, the

approach can be applied for a general size, say, n of an arbitrary

instance of the problem under consideration. In the case of theoretical

approach, the size n may be arbitrary large. However, in empirical

approach, because of practical considerations, only the instances of

moderate sizes may be considered.


6. Starting from the basic instructions and operations and using structuring

rules, we can calculate the ––––––––– of a problem.

2.6 Well known Asymptotic Function & Notations

We often want to know a quantity only approximately and not necessarily

exactly, just to compare with another quantity. And, in many situations,

correct comparison may be possible even with approximate values of the

quantities. The advantage of the possibility of correct comparisons through

even approximate value of quantities is that the time required to find

approximate values may be much less than the time required to find exact

values. We will introduce five approximation functions and their notations.

The purpose of these asymptotic growth rate functions to be introduced, is

to facilitate the recognition of essential character of a complexity function

through some simpler functions delivered by these notations. For example a



complexity function f(n) = 5004 n3 + 83 n2 + 19 n + 408, has essentially

same behavior as that of g(n) = n3 as the problem size n becomes larger

and larger. But g(n) = n3 is much more understtandable and its value easier

to compute than the function f(n)

Given below are the five well-known approximation functions.

1) O : (O (n2) is pronounced as big-oh of n2, or sometimes just as oh of n2)

2) : ( (n2) is pronounced as ‗big-omega of n2 or sometimes just as

omega of n2)

3) : ( (n2) is pronounced as ‗theta of n2)

4) o : (o (n2) is pronounced as ‗little-oh h2)

5) : ( (n2) is pronounced as ‗little-omega of n2)

These approximations denote relations from functions to functions.

f, g: N N are given by

f(n) = n2 – 5n and g(n) = n2 then O(f(n)) = g(n) or O(n2 – 5n) = n2

To be more precise, each of these notations is a mapping that associates a

set of functions to each function under consideration. For example, if f(n) is

a polynomial of degree k then the set O(f(n)) includes all polynomials of

degree less than or equal to k.

Remarks

In the discussion of any one of the five notations, generally two functions,

say, f and g are involved. The functions have their domains and co domains

as N, the set of natural numbers, i.e., f : N N

g : N N

These functions may also be considered as having domain and co domain

as R.

1) The notation O

Provides asymptotic upper bound for a given function. Let f(x) and g(x) be

two functions each from the set of natural numbers or set of positive real

numbers.

Then f(x) is said to be O(g(x)) (pronounced as big-oh of g of x) if there exists

two positive integers/real number constants C and K such that

f(x) C g(x) for all x k ……….(A)



Note: (The restriction of being positive on integers/ real is justified as all

complexities are positive numbers)

Example: For the function defined by

f(x) = 2x3 + 3x2 + 1 show that

i) F(x) = O(x3)

ii) f(x) = O(x4)

iii) x3 = O (f(x))

iv) x4 O(f(x))

v) F(x) O(x2)

Solution:

i) Consider

f(x) = 2x2 + 3x2 + 1

2x3 + 3x3 + 1x3 = 6x3 for all x 1

(by replacing each term xi by the highest degree term x3)

there exist C = 6 and k = 1 such that

f(x) C, x3 for all x k

Thus we have found the required constants C and k. Hence f(x) is O(x3)

ii) As, above, we can show that

f(x) 6x4 for all x 1

However, we may also, by computing some value of f(x) and x4, find

C and k as follows

f(1) = 2 + 3 + 1 = 6 ; (1)4 = 1

f(2) = 2.23 + 3.22 + 1 = 29 ; (2)4 = 16

f(3) = 2.33 + 3.32 + 1 = 82 ; (3)4 = 81

for C = 2 and k = 3 we have

f(x) 2. x4 for all x k

Hence f(x) is O(x4)

iii) for C = 1 and k = 1 we get

x3 C (2x3 + 3x2 + 1) for all x k

iv) We prove the result by contradiction. Let there exist positive constants C

and k such that

x4 C (2x3 + 3x2 + 1) for all x k

x4 C (2x3 +3x3 + x3) = 6 Cx3 for x k

x4 6 Cx3 for all x k



Implying x 6C for all x k

But for x = max {6C + 1, k}, the previous statement is not true.

Hence the proof.

v) Again we establish the result by contradiction.

Let O(2x3 + 3x2 + 1) = x2

Then for some positive numbers C and k

2x3 + 3x2 + 1 Cx2 for all x k,

Implying

x3 C x2 for all x k (x3 2x3 + 3x2 + 1 for all x 1)

implying

x C for x k

Again for x = max {C + 1, k}

The last inequality does not hold. Hence the result.

2) The Notation

The function f(n)= (g(n)) (read as ―f of n is omega of g of n‖) if there exist

positive constants c and no such that f(n) =c g(n) for all n, n no.

Eg: The function 3n+2= (n) as 3n+2 3n for n 1

The function 3n+3= (n) as 3n+3 3n for n 1

The function 10n2+4n+2= (n2) as 10n2+4n+2 n2 for n 1

The statement f(n)= (g(n)) states that g(n) is only a lower bound on f(n).

3) The Notation

This notation provides simultaneously both asymptotic lower bound and

asymptotic upper bound for a given function.

Let f(x) and g(x) be two functions, each from the set of natural numbers or

positive real numbers to positive real numbers. Then, f(x) is said to be

(g(x)) (pronounced as big-theta of g of x) if, there exists positive constants

C1, C2 and k such that C2 g(x) f(x) C1g(x) for all x k.

(Note the last inequalities represent two conditions to be satisfied

simultaneously viz.,

C2 g(x) f(x) and f(x) C1g(x))

Now we state the following theorem without proof which relates the three

functions O

*



Theorem: For any two functions f(x) and g(x), f(x) = (g(x)) if and only if

f (x) = 0 (g(x)) and f(x) = (g(x)) where f(x) and g(x) are non-negative.

Example: Let f(n): 1 + 2 + .... + n, n 1. Show that f(n) = (n2).

Solution: First, we find an upper bound for the sum. For n 1, consider

1 + 2 + …… + n n + n + ….. + n = n. n = n2

This implies that f(n) = O(n2). Next, we obtain a lower bound for the sum.

We have

n.....2

n...21n...21

n.....2

n

2

n..

2

n

2

n

2

n.

2

n

.4

n2

This proves that f(n) = (n2). Thus by the above. Theorem f(n) = (n2).

4) The Notation

The asymptotic upper bound provided by big-oh notation may or may not be

tight in the sense that if f(x) = 2x3 + 3x2 + 1. Then for f(x) = O(x3), though

there exist C and k such that f(x) C(x3) for all x k yet there may also be

some values for which the following equality also holds

f(x) = C(x3) for x k However, if we consider f(x) = O(x4) then there can not

exist positive integer C such that f(x) = Cx4 for all x k.

The case of f(x) = O(x4) provides an example for the notation of small-oh

i.e., notation o

Let f(x) and g(x) be two functions, each from the set of natural numbers or

positive real numbers to positive real numbers.



Further, let C > 0 be any number, then f(x) = o (g(x)) (pronounced as little oh

of g of x) if there exists natural number k satisfying

f(x) < Cg (x) for all x k 1 ………..(B)

Here we get the following points

i) In the case of little-oh the constant C does not depend on the two

functions f(x) and g(x). instead, we can arbitrarily choose C > 0

ii) The inequality (B) is strict whereas the inequality (A) of big — oh is not

necessarily strict.

Examples:

For f(x) = 2x3 + 3x2 + 1, we have

i) f(x) = o(xn) for any n 4

ii) f(x) o(xn) for any n 3

Solutions:

i) Let C > 0 be given and to find out k satisfying the requirement of little-

oh.

Consider

2x3 + 3x2 + 1 < C xn 3n

3Cx

x

1

x

32

When n = 4

Then, above inequality becomes

Cxx

1

x

32

3

If we take 1,C

7maxk

Then,

2x3 + 3x2 + 1 < C x4 for n k. In general, as xn > x4 for n 4, therefore,

2x3 + 3x2 + 1 < C xn for n 4.For all x k with 1,C

7maxk

ii) We prove the result by contradiction Let, if possible, f(x) = 0 (x) for n 3.

Then there exist positive constants C and k such that 2x3 + 3x2 + I < C xn

for all x k.



Dividing by x3 throughout, we get

3n

2xC

x

1

x

32

As C is arbitrary, we take C = 1, then the above inequality reduces to

3n

2xC

x

1

x

32 for n 3 and x k 1

Also, it can be easily seen that xn–3 1 for n 3 and x k 1.

1x

1

x

32

2 for n 3

However, the last inequality is not true. Therefore, the proof by

contradiction. Generalizing the above example, we get the results

f(x) is a polynomial of degree m and g(x) is a polynomial of degree n. Then

f(x) = o(g(x)) if and only if n > m

We state below the two results which can be useful in finding small-oh upper

bound for a given function (without proof).

More generally, we have

Theorem: Let f(x) and g (x) be functions in definition of small — oh notation.

Then f(x) = o(g (x)) if and only if

0xg

xfLimx

Next, we introduce the last asymptotic notation, namely, small-omega. The

relation of small -omega to big-omega is similar to what is the relation of

small-oh to big oh.

5) The Notation

Again the asymptotic lower bound may or may not be tight. However, the

asymptotic bound cannot be tight. The definition of is as follows:

Let f(x) and g(x) be two functions each from the set of natural numbers or

the set of positive real numbers to set of positive real numbers.

Further, let C > 0 be any number, then, f(x) = (g(x)) if there exists a

positive integer k Such that f(x) > C g(x) for all x k.



Example:

Lf f(x) = 2x3 + 3x2 + 1 then f(x) = and also f(x) = (x2)

Solution:

Let C be any positive constant

Consider 2x3 + 3x2 + 1 > Cx

To find out k 1 satisfying the Conditions of the bound .

Cx

1x3x2 2 (dividing through out by x)

Let k be integer with k C + 1 then for all x k

1CkCC3C2K3k2x3x2x

1x3x2 12222

f(x) = (x)

Again, consider for any C > 0, 2x3 + 3x2 + I > C > x2 then

Cx

13x2

2 Let k be integer with k C + 1

Then, for x k we have C3C23k23x2x

13x2

2

Hence

f(x) = (x2)

In general, we have the following two theorems

Theorem: If f(x) is a Polynomial of degree n, and g(x) is a polynomial of

degree n, then

f(x) = (g(x)) if and only if m >n.

Theorem: Let f(x) and g(x) be functions in the definitions of little-omega.

Then f(x) = (g(x)) if and only if

xg

xflimx

0xf

xglimx




7. The ––––––––– provides asymptotic upper bound for a given function.

8. Let f(x) and g (x) be functions in definition of small-oh notation. Then f(x)

= o(g (x)) if and only if ––––––––––––.

2.7 Analysis of Algorithms — Simple Examples

Now we study the different types of analysis of Algorithms with simple

illustrations.

Compute Prefix Average: For a given array A[1 …n] of numbers, the

problem is concerned with finding an array B[1…n] such that

B [1] = A[1]

B [2] = average of first two entries = 2

2A1A

B [3]=average of first 3 entries=3

3A2A1Aand in general for

1 i n,

B[ i ] = average of first i entries in the array A [1…n] = i

iA....2A1A

Algorithm first-prefix-Average (A [1…n])

begin (of algorithm)

for i 1 to n do

begin (first for-loop)

Sum 0;

{sum stores the sum of first i terms, obtained in different iterations of

for-loop}

for j 1 to i do

begin {of second for-loop}

Sum Sum + A [ j ];

end {of second for loop}

B [ i ] Sum / i

End {of the first for — loop}

end {of algorithm}



Analysis of First-Averages

The different steps involved in the analysis of first averages is discussed

below

Step 1: Initialization step for setting up of the array A [1. .n] takes constant

time, say, C1, in view of the fact that for the purpose, only address of A (or of

A[1] is to be passed. Also after all the values of B [1...n] are computed, then

returning the array B [1…n] also takes constant time say C2, again for the

same reason.

Step 2: The body of the algorithm has two nested for-loops, the outer one,

called the first for loop is controlled by i and is executed n times. Hence, the

second for-loop along with its body, which form a part of the first for-loop, is

executed n times. Further, each construct within second for-loop controlled

by j, is executed i times just because of the iteration of the second for-loop.

However, the second for-loop itself is being executed n times because of the

first for-loop. Hence, each instruction within the second for-loop is executed

(n, i) times for each values of

i = 1, 2, . . .,n.

Step 3: In addition, each controlling variable i and j is incremented by 1 after

each iteration of i or j as the case may be. Also, after each increment in the

control variable, it is compared with the (upper limit + 1) of the loop to stop

the further execution of the for-loop.

Thus, the first-for-loop makes n additions (to reach (n + 1) and n

comparisons with (n + 1)).

The second for-loop makes, for each value of i = 1, 2,……n, one addition

and one comparison. Thus, total number of each of additions and

comparisons done just for controlling variable j

2

1nnn......21

Step 4: Using the explanation of steps 2, we count below the number of

times the various operations are executed.

i) Sum 0 is executed n times, once for each value of i from 1 to n.

ii) On the similar lines of how we counted the number of additions and

comparisons for the control variable j, it can be seen that the number of



additions (sum sum + A [ j ] and divisions (B [ i ] sum / i) is

2

1nn


9. 1 + 2 + ……….. + n = ––––––––

2.8 Well known Sorting Algorithms

The following are the well-known Sorting Algorithms:

1) Insertion sort

2) Bubble sort

3) Selection sort

4) Shell sort

5) Heap sort

6) Merge sort

7) Quick sort

In this section, we will discuss some of the well-known algorithms for sorting

a given list of numbers:

Ordered set: Any set S with a relation, say, , is said to be ordered if for

any two elements x and y of S, either x y or y x is true. Then, we may

also say that (S, ) is an ordered set.

2.8.1 Insertion sort

The insertion sort, algorithm for sorting a list L of n numbers represented by

an array A [1... n] proceeds by picking up the numbers in the array from left

one by one and each newly picked up number is placed at its relative

position, w.r.t. the sorting order, among the earlier ordered ones. The

process is repeated till each element of the list is placed at its correct

relative position i.e., when the list is sorted.

Example:

A[1] A[2] A[3] A[4] A[5] A[6]

80 32 31 110 50 41 { given list, initially}

{Pick up the left — most number 80 from the list, we get the sorted sublist}

80 {pick up the next number 32 from the list and place it at correct position

relative to 80, so that the sublist considered so far is sorted}

32 80



{We may note in respect of the above sorted sublist, that in order to insert

32 before 80, we have to shift 80 from first position to second and then

insert 32 in first position.

The task can be accomplished as follows:

i) First 32 is copied in a location, say, m

ii) 80 is copied in the location A [2] = 32 so that we have

A[1] A[2] m

80 80 32

32 is copied in A [1] from m so that we have

A[1] A[2] m

32 80 32

{thus we get the sorted sublist given above}

32 80

{next number 31 is picked up, compared first with 80 and then (if required)

with 32, in order to insert 31 before 32 and 80, we have to shift 80 to third

position and then 32 to second position and then 31 is placed in the first

position}


1) First 31 is copied in a location, say, m

2) 80 is copied in the location A[3] = 31 so that we have

A[1] A[2] A[3] m

32 80 80 31

3) 32 is copied in A[2] from A[1] so that we have

A[1] A[2] A[3] m

32 32 80 31

4) the value 31 from m is copied in A[1] so that we get

A[1] A[2] A[3] m

31 32 80 31

Thus, we get the sorted sublist

31 32 80



{Next, 110 is picked up, compared with 80. As 110 > 80, therefore,

no shifting and no more comparisons. 110 is placed in the first position

after 80}

31 32 80 110

{next, number 50 is picked up, First compared with 110, found less; next

compared with 80, again found less, again compared with 32. The

correct position for 50 between 32 and 80 in the sublist given above.

Thus, each of 110 and 80 is shifted one place to make space for 50 and

then 50 is placed over there}


1) First 50 is copied in a location say m

2) 110 is copied in the location A [5] = 50 so that we have

A [1] A [2] A [3] A [4] A [5] m

31 32 80 110 110 50

3) 80 is copied in A [4] from A [3] so that we have

A [1] A [2] A [3] A [4] A [5] m

31 32 80 80 110 50

4) the value 50 from m is copied in A [3] so that we get

A [1] A [2] A [3] A [4] A [5] m

31 32 50 80 110 50

Thus, we get the following sorted sublist}

31 32 50 80 110

{next in order to place 40 after 32 and before 50, each of the values 50, 80

and 110 need to be shifted one place to the right as explained above.}

{However, values 31 and 32 are not to be shifted. The process of inserting

40 at correct place is similar to the ones explained earlier}

31 32 40 50 80 110

Algorithm: The insertion sort:



The idea of insertion sort as explained above, may be implemented through

procedure insertion -sort given below. It is assumed that the numbers to be

sorted are stored in an array A [1...n].

procedure insertion-Sort (A [1…..n]: real)

begin {of procedure}

if n = 1

then

write (‗A list of one element is already sorted‘)

else

begin {of the case when n 2}

for j 2 to n do

{to find out the correct relative position for A [j] and insert it there among the

already sorted elements A [1] to A [j — 1]}

begin {of for loop}

If A [ j ] < A [j — 1] then begin

{we shift entries only if A [j] < A [j — 1])

i j – 1; m A [ j ]

{In order to find correct relative position we store A [ j ] in m and start with

the last element

A [j – 1] of the already sorted part. If m is less than A [j – 1], then we move

towards left, compare again with the new element of the array or we reach

the left-most element A [1] }

While (I > 0 and m < A [ i ] ) do

begin {of while loop}

A [ i + 1] A [ i ]

i i – 1

end



{after finding the correct relative position, we move all the elements of the

array found to be greater than m = A [ j ], one place to the right so as to

make a vacancy at correct relative position for A [ j ] }

end; {of while loop}

A [i + 1) m

(i.e., m = A [ j ] is stored at the correct relative position}

end {if}

end ; { of for loop }

end ; (of else part)

end; {of procedure}

2.8.2 Bubble sort

The Bubble Sort algorithm for sorting of n numbers, represented by an array

A [1..n], proceeds by scanning the array from left to right. At each stage,

compares adjacent pairs of numbers at positions A [ i ] and A [i + 1] and

whenever a pair of adjacent numbers is found to be out of order, then the

positions of the numbers are exchanged. The algorithm repeats the process

for numbers at positions A [i + 1] and A [i + 2]

Thus, in the first pass after scanning once all the numbers in the given list,

the largest number will reach its destination, but other numbers in the array,

may not be in order. In each subsequent pass, one more number reaches

its destination.

Example:

In the following, in each line, pairs of adjacent numbers, shown in bold, are

compared. And if the pair of numbers are not found in proper order, then the

positions of these numbers are exchanged.



The list to be sorted has n = 6 as shown in the first row below:

Iteration number i = 1

80 32 31 110 50 40 (j = 1)

32 80 31 110 50 40 (j = 2)

32 31 80 110 50 40

32 31 80 110 50 40

32 31 80 50 110 40 (j = 5)

32 31 81 50 40 110 (j = 1)

remove from further

consideration

In the first pass traced above, the maximum number 110 of the list reaches

the rightmost position (i.e., 6th position). In the next pass, only the list of

remaining (n – 1) = 5 elements, as shown in the first row below, is taken into

consideration. Again pairs of numbers in bold, in a row, are compared and

exchanged, if required.

Iteration number i = 2

31 32 81 50 40 (j = 2)

31 32 81 50 40 (j = 3)

31 32 50 81 40 (j = 4)

31 32 50 40 81 (j = 1)

remove from further

consideration

In the second pass, the next to maximum element of the list viz., 81,

reaches the 5th position from left. In the next pass, the list of remaining

(n – 2) = 4 elements are taken into consideration.

Iteration Number i = 3

31 32 50 40 (j = 2)

31 32 50 40 (j = 3)

31 32 40 50 (j = 1)

remove from further

consideration



In the next iteration, only (n – 3) = 3 elements are taken into consideration

31 32 40

31 32 40

In the next iteration, only n – 4 = 2 elements are considered

31 32

These elements are compared and found in proper order. The process

terminates.

Procedure bubblesort (A[1... n])

begin

for i 1 to n-1 do

for j 1 to (n-i)

{In each new iteration, as explained above, one less number of elements is

taken into consideration. This is why j varies upto only (n- i)}

If A[ j ] > A[j + 1] then interchange A[ j ] and A [j + 1]

end

{A [1.. n] is in increasing order}

2.8.3 Selection sort

Selection Sort for sorting a list L of n numbers, represented by an array

A [1.. n], proceeds by finding the maximum element of the array and placing

it in the last position of the array representing the list. Then repeat the

process on the sub array representing the sublist obtained from the list

excluding the current maximum element.

The difference between Bubbles sort and selection sort is that, in selection

sort to find the maximum number in the array, a new variable MAX is used

to keep maximum of all the values scanned upto a particular stage. On the

other hand, in Bubble sort, the maximum number in the array under

consideration is found by comparing adjacent pairs of numbers and by

keeping larger of the two in the position at the right. Thus, after scanning the

whole array once, the maximum number reaches the right-most position of

the array under consideration.



The following steps constitute the selection sort algorithm:

Step 1: Create a variable MAX to store the maximum of the values scanned

upto a particular stage. Also create another variable say Max-POS which

keeps track of the position of such maximum values.

Step 2: In each iteration, the whole list / array under consideration is

scanned once find out the position of the current maximum through the

variable MAX and we have to find out the position of the current maximum

through MAX-POS.

Step 3: At the end of iteration, the value in last position in the current array

and the (maximum) value in the position Max-POS are exchanged.

Step 4: For further consideration, replace the list L by L ~ {MAX} {and the

array A by the corresponding sub array} and go to step 1.

Example

80 32 31 110 50 40 ( given initially)

Initially, MAX 80 MAX-POS 1

After one iteration, finally, MAX 110, MAX-POS = 4

Numbers 110 and 40 are exchanged to get

80 32 31 40 50 110

New list, after one iteration, to be sorted is given by:

80 32 31 40 50

Initially, MAX 80, MAX-POS 1

Finally, also Max 80, MAX-POS 1

entries 80 and 50 are exchanged to get

50 32 31 40 80

New List, after one iteration, to be sorted is given by:

50 32 31 40

Initially, MAX 50, MAX0-POS 1

Finally, also Max 50, MAX-POS 1

entries 50 and 40 are exchanged to get

40 32 31 50



New List, after third iteration, to be sorted is given by:

40 32 31

Initially & finally

Max 40; MAX-POS 1

Therefore, entries 40 and 31 are exchanged to get

31 32 40

New List, after fourth iteration, to be sorted is given by:

31 32

Max 32 and MAX-POS 2

no exchange, as 32 is the last entry

New List after fifth iteration, to be sorted given by: 31

This is a single element list. Hence, no more iterations. The algorithm

terminates. This completes the sorting of the given list.

Next, we formalize the method used in Sorting the list

Procedure Selection Sort: (A [1.. n])


for i 1to (n – 1)do

begin {of i for loop}

MAX A[ i ];

MAX-POS i

for j i + 1 to n do

begin {j for-loop}

if MAX < A[j] then

begin

MAX-POS j

MAX A[j]

end {of if}

end {of j for-loop}

A[MAX-POS] A[n – i + 1]

A[n – i + 1] MAX

{the ith maximum value is stored in the ith position from right or equivalent in

(n – i + 1) the position of the array



end {of i loop}

end {of procedure}.

2.8.4 Shell Sort

The sorting algorithm is named so in honour of D.L Short (1959), who

suggested the algorithm. Shell sort is also called diminishing-increment sort.

The essential idea behind Shell-Sort is to apply any of the other sorting

algorithm (generally Insertion Sort) to each of the several interleaving

sublists of the given list of numbers to be sorted. In successive iteration, the

sublists are formed by stepping through the file with an increment INCi taken

from some pre-defined decreasing sequence of step-sizes INC1 > INC2 >

………> INCi >…………...>1, which must terminate in 1.

Examples: Let the list of numbers to be sorted, be represented by the next

row.

13 3 4 12 14 10 5 1 8 2 7 9 11 6 (n = 14)

Initially, we take INC as 5, i.e.,

Taking sublist of elements at 1st, 6th and 11th positions, viz., sublsit of values

13, 10 and 7. After sorting these values we get the sorted sublist

7 10 13

Taking sublist of elements at 2nd, 7th and 12th positions, viz, sublist of values

3, 5 and 9. After sorting these values we get the sorted sublist.

3 5 9

Taking sublist of elements at 3rd, 8th and 13th positions, viz, sublist of values

4, 1 and 11. After sorting these values we get the sorted sublist

1 4 11

Similarly, we get sorted sublist

6 8 12

Similarly, we get sorted sublist

2 14

{Note that, in this case, sublist has only two elements it is 5th sublist and

n – 14 is less than 5INCINC

14 where INC = 5}



After merging or interleaving the entries from the sublists, while maintaining

the initial relative positions, we get the New List:

7 3 1 6 2 10 5 4 8 14 13 9 11 12

Next, take INC = 3 and repeat the process, we get sorted sublists:

5 6 7 11 14,

2 3 4 12 13 and

1 8 9 10

After merging the entries from the sublists, while maintaining the initial

relative positions, we get the New List

New List

5 2 1 6 3 8 7 4 9 11 12 10 14 13

Taking INC = 1 and repeat the process, we get the sorted list

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Note: Sublists should not be chosen at distances which are multiples of

each other e.g., 8, 4, 2 etc. Otherwise, same elements may be compared

again and again.

Procedure Shell — Sort (A [1..n] : real)

K: integer

{to store value of the number of increments}

INC [1..k] :integer

{to store the values of various increments}


read (k)

for i 1 to (k — 1) do

read (INC [ i ]

INC [k] 1

{last increment must be 1}

For i 1 to k do

{this i has no relation with previous occurrence of i)



Begin

j INC [ i ]

r [n / j];

for t 1 to k do

{for selection of sublist starting at position t}

begin

if n < r * j + t

then s r — 1

else s r

Insertio-Sort (A[ t… (t + s*j)])

end {of t-for-loop}

end {i for-loop}

end { of procedure}

2.8.5 Heap Sort

In order to discuss Heap-Sort algorithms we recall the following definitions:

Binary Tree: A tree is called a binary tree, if it is either empty, or it consists of a

node called the root together with two binary trees called the left subtree

and a right subtree. Now , we make the following observations.

1. It may be noted that the above definition is a recursive definition, in the

sense that definition of binary tree is given in its own terms (i.e. binary tree).

2. The following are all distinct and the only binary trees are having two

nodes.



The following are all distinct and only binary trees are having three

nodes

Heap: is defined as a binary tree with keys assigned to its nodes (one key

per node) such that the following conditions are satisfied.

i) The binary tree is essentially complete (or simply complete), i.e., all its

levels are full except possibly the last where only some rightmost leaves

may be missing.

ii) The key at each node is greater than or equal to the key at its children.

The following binary tree is Heap

10

5 7

4 2 1



However, the following is not a heap because the value 6 in a child node is

more than the value 5 in the parent node.

Also, the following is not a heap, because some leaves (e.g., right child

of 5), in between two other leaves (viz 4 and 1), are missing.

Alternative definition of Heap: Heap is an array H [1.. n] in which every

element in position i (the parent) in the first half of the array is greater than

or equal to elements in positions 2i and 2i + 1 (the children).

HEAP SORT is a three-step algorithm as discussed below:

i) Heap Construction for a given array.

ii) (Maximum deletion) Copy the root value (which is maximum of all values

in the Heap) to right-most yet-to-be occupied location of the array used

to store the sorted values and copy the value in the last node of the tree

(or of the corresponding array) to the root.

iii) Consider the binary tree (which is not necessarily a Heap now) obtained

from the Heap through the modification through step (ii) above and by

removing currently the last node from further consideration. Convert the

binary tree into a Heap by suitable modifications.

10

5 7

6 2 1

10

5 7

4 1



Example:

Let us consider applying Heap Sort of the list 80 32 31 110 50 40

120 represented by an array A [1... 7]

Step 1: Construction of a Heap for the given list

First, we create the tree having the root as the only node with node-value 80

as shown below:

Next, value 32 is attached as left child of the root, as shown below

As 32 < 80, therefore, heap property is satisfied. Hence, no modification of

the tree. Next, value 31 is attached as right child of the node 80, as shown

below

Again, as 31 < 80, heap property is not disturbed. Therefore, no modification

of the tree.

80

32

80

32 31

80



Next, value 110 is attached as left child of 32; as shown below:

However, 110 > 32, the value in child node is more than the value in the

parent node. Hence the tree is modified by exchanging the values in the two

nodes so that, we get the following tree.

Again, as 110 > 80, the value in child node is more than the value in the

parent node. Hence, the tree is modified by exchanging the values in the

two nodes so that, we get the following tree

80

32 31

110

110

80 31

32



Next, number 50 is attached as right child of 80, so that the new tree is as

given below

As the tree satisfies all the conditions of a Heap, we insert the next number

40 as left child of 31 to get the tree

As the new insertion violates the condition of Heap, the values 40 and 31

are exchanged to get the tree which is a heap

110

80 31

32 50

110

80 31

32 50 40

110

80 40

32 50 31



Next, we insert the last value 120 as right child of 40 to get the tree

The last insertion violates the conditions for a Heap. Hence, 40 and 120 are

exchanged to get the tree.

Again, due to movement of 120 upwards, the Heap property is disturbed at

nodes 110 and 120. Again, 120 is moved up to get the following tree which

is a heap

After having constructed the Heap through Step 1 above, we consider

110

80 40

32 50 31 120

110

80 120

32 50 31 40

120

80 110

32 50 31 40



Step 2 & 3: Consists of a sequence of actions viz, (i) deleting the value of

the root ii) moving the last entry to the root and (iii) then readjusting the

heap. The root of a heap is always the maximum of all the values in the

nodes of the tree. The value 120 currently in the root is saved in the last

location B [n] in our case B [7] of the sorted array, say, B[1. .n] in which the

values are to be stored after sorting in increasing order.

Next, value 40 is moved to the following binary tree, which is not a heap

In order to restore the above tree as a heap, the value 40 is exchanged with

the maximum of the values of its two children. Thus, 40 and 110 are

exchanged to get the tree which is a heap.

40

80 110

32 50 31

110

80 40

32 50 31



Again, 110 copied to B[6] and 31, the last value of the tree is shifted to the

root and last node is removed from further consideration to get the following

tree, which is not a heap

Again, the root value is exchanged with the value which is maximum of its

children‘s value i.e., exchanged with value 80 to get the following tree, which

again is not a Heap.

Again, the value 31 is exchanged with the maximum of the values of its

children, i.e., with 50, to get the tree which is a heap.

80

50 40

32 31

31

80 40

32 50

80

31 40

32 50



Again, 80 is copied in B[5] and 31, the value of the node replaces 80 in the

root and the last node is removed from further consideration to get the tree

which is not a heap.

Again, 50, the maximum of the two children‘s values is exchanged with the

value of the root 31 to get tree, which is not a heap.

Again, 31 and 32 are exchanged to get the heap

Next, 50 is copied in B [4]

50

32 40

31

50

31 40

32

31

50 40

32



The entry 31 in the last node replaces the value in the root and the last node

is deleted to get the following tree which is not a Heap

Again, 40, the maximum of the values of children is exchanged with 31, the

value in the root. We get the Heap

Again, 40 is copied in B [3]. The value in the last node of the tree viz 31,

replaces the value in the root and the last node is removed from further

consideration to get the tree, which is not a heap.

Again, 32, the value of its only child is exchanged with the value of the root

to get the Heap

32

31

40

32 31

31

32 40

31

32



Next, 32 is copied in B [2] and 31, the value in the last node is copied in the

root and the last node is deleted, to get the tree which is a heap.

This value is copied in s and the Heap Sort algorithm terminates.

The list L of n numbers to be sorted is represented by an array A [1..n]

The following procedure reads one by one the values from the given to-be-

sorted array A[1...n] and gradually builds a heap. For this purpose, it calls

the procedure Build-Heap. For building the heap, an array H [1..n] is used

for storing the elements of the heap. Once the Heap is built, the elements of

A [1..n] are already sorted in H [1..n] and hence A may be used for sorting

the elements of finally sorted list from which we used the array B. Then, the

following three steps are repeated n times (n, the number of elements in the

array), in the ith iteration.

i) The root element H [1] is copied in A [n — i + 1] location of the given

array A. The first time, root element is stored in A [n]. The next time, root

element is stored in A [n – 1] and so on.

ii) The last element of the array H [n – i + 1] is copied in the root of the

heap, i.e., in H [1] and H [n – i + 1] is removed from further

consideration. That is in the next iteration, only the array H [1 ... (n – i)]

(which may not be a heap) is taken into consideration.

iii) The procedure Build-Heap is called to build the array H [1.. (n – i)] into a

heap.

Procedure Heap-Sort (A [ 1..n]: real)

begin {of the procedure}

H[1..n] Build – Heap (A [1..n])

For i 1 to n do

begin {for loop}

A[n – i + 1] H [1]

H[1] H [n – i + 1]

Build – Heap (H[1 ... (n — i)])

end {for – loop}

end {procedure}

31



The following procedure takes an array B[1...m] of size m, which is to be

sorted, and builds it into a heap

Procedure Build-Heap (B [1.. m] : real)


for j = 2 to m do

begin {of for-loop}

location j

while (location > 1) do

begin

parent {location / 2]

If A [location] A [parent] then return {i.e., quite while loop]

else

(i.e., of A [location] > A [parent] then}

begin {to exchange A [parent] and A [location]}

temp A [parent]

A [parent] A [location]

A[location] temp

end {of if .. then ..else}

location parent

end {while loop}

end {for loop}

end { of procedure}


10. The –––––––––– algorithm for sorting of n numbers, represented by an

array A [1..n], proceeds by scanning the array from left to right.

11. The essential idea behind ––––––––– is to apply any of the other sorting

algorithm to each of the several interleaving sublists of the given list of

numbers to be sorted.

2.9 Summary

We Studied the different types of functions and Notations with well

illustrated example.

The concept of Modular arithmetic/ Mod function is well defined with

simple examples.



The concept of Mathematical Expectation and its use in average case

analysis of algorithms is well illustrated.

In this unit, we discussed that if a problem is algorithmically solvable

then it may have more than one algorithmic solutions. In order to choose

the best out of the available solutions, there are criteria for making such

a choice. The complexity/efficiency measures or criteria are based on

requirement of computer resources by each of the available solutions.

In this unit, we also studied the different types of Asymptotic functions

and their Notations.

The different methods to analyse the algorithm is studied step by step in

this unit .


1. Define an injective function

2. Define monotonic functions

3. Define exponential function with an example

4. Write the advantages of theoretical approach over the empirical

approach

5. For the function f(x) = 2x3 + 3x2 + 1,

show that

i) f(x) = (x3)

ii) f(x) (x2)

iii) f(x) (x4)

2.11 Answers


1. f(x) f(y)

2. Ceiling function

3. Exponentiation function

4. ex

5. Mathematical Expectation

6. time complexity

7. Notation O



8. 0)x(g

)x(fLimx

9. 2

)1n(n

10. Bubble Sort

11. Shell-Sort

Terminal Questions

1. Refer Section 2.2.2

2. Refer Section 2.2.4.

3. Refer Section 2:3.5

4. Refer Section 2.6

5. i) for C1 =3, C2 = 1 and k = 4 i.e. 1. C2 x3 f(x) C1 x

3 for all x k

ii) We can show by contradiction that no C1 exists. Let, if possible for

some positive integers k and C1, we have 2x3 + 3x2 + 1 C1. x2 for

all x k hence x3 C1 x2 for all x k

i.e.,x C1 for all x k

But for x = max {C1 + 1, k}

The last inequality is not true.

Therefore, x = max {C1 + 1, k}

iii) f(x) (x4)

we can show by contradiction that there does not exist C2

such that

C2 x4 (2x3 + 3x2 + 1)

If such a C2 exists for some k then C2 x4 2x3 + 3x2 + 1 6x3 for all

x 1, k 1, implying C2 x 6 for all x k

But for 16

2Cx

The above inequality is false. Hence, proof by contradiction.



Unit 3 Divide and Conquer Structure

3.1 Introduction

Objectives

3.2 Divide and Conquer Strategy

3.3 Binary Search

3.4 Max. and Min.

3.5 Merge Sort

3.6 Quick Sort

3.7 Summary

3.8 Terminal Question

3.9 Answers

3.1 Introduction

After studying the different concepts in mathematical functions and notaions,

we will discuss the concept of divide and conquer.

Given a function to compute n inputs, the divide-and-conquer strategy

suggests splitting the inputs into K distinct subsets, 1< K n yielding K

subproblems. These subproblems must be solved and then a method must

be found to combine subsolutions into a solution as the whole. If the

subproblems are still relatively large, then the divide-and-conquer strategy

can be possibly re-applied.

Often, the subproblems resulting from a divide-and-conquer design are of

the same type as the original problem. For those cases, the reapplication of

the divide-and-conquer principle is naturally expressed by a recursive

algorithm. Now, smaller and smaller subproblems of the same kind are

generated until eventually subproblems that are small enough to be solved

without splitting are produced.

Objectives


apply divide and conquer strategy to practical problems

analyze the binary search algorithm and compute the time for successful

and unsuccessful search

analyze Max and Min algorithm using divide-and-conquer method

analyze the time for the Merge Sort Algorithm

use the working of Quick Sort Algorithm.



3.2 Divide and Conquer Strategy

Given a function to compute on n inputs, the divide-and-conquer strategy

suggests splitting the inputs into K distinct subsets, 1<K<n, yielding K

subproblems. These subproblems must be solved and then a method must

be found to combine subsolutions into a solution as a whole. If the

subproblems are still relatively large, then the divide-and-conquer strategy

can be possibly reapplied.

We can write a control abstraction that mirrors the way an algorithm based

on divide-and-conquer will look. By a control abstractions we mean a

procedure whose flow of control is clear but whose primary operations are

specified by other procedures whose precise meanings are left undefined.

Algorithm 3.1

1.procedure DANDC (p,q)

2.global integer n, A(1:n)

3.integer p,q; //1 p q n//

4.local integer m;

5.if small (p,q)

6.then return (G(p,q))

7.else m divide (p,q) //p < m < q

8.return (combine(DANDC(p,m), DANDC(m + 1,q)))

9.endif

10.end DANDC

In Algorithm 3.1, procedure is called by DANDC (1,n)

– Small (p,q) is a Boolean function and will return true or false. True if

input size is small enough that the answer can be computed without

splitting. If so, function G is invoked.

– Divide (p,q), divides into (p, m) and (m+1, q)

– Combine function combines the results of p,m and m+1, q. Combine is a

function that determines the solution to p using the solutions to the K

sub problems.

If the size of p is n and the sizes of the K subproblems are n1, n2…….., nk

respectively, then the computing time of DANDC is described by the

recurrence relation.



otherwise

smallisnif

nfnT.....nTnT

ngnT

k21

otherwise

smallnif

nfn/22T

ngnTe.

i.

Where, T(n) is the time for DANDC on any input of size n & g (n) is the time

to compute the answer directly for small inputs. The function f(n) is the time

for dividing P and combining the solutions to subproblems.

The complexity of many divide-and-conquer algorithms is given by

recurrences of the form.

1nnfb/naT

1n1TnT

Where, a and b are known constants. We assume that T(1) is known and n

is a power of b (i.e. n = bk)


1. The complexity of many divide-and-conquer algorithms is given by

recurrences of the form –––––––––––––.

3.3 Binary Search

Let ai, 1 < i < n be a list of elements that are sorted in non-decreasing order.

Consider the problem of determining whether a given element x is present in

the list. If x is present we are to determine a value j such that aj = x. If x is

not present in the list, then j is to be set to zero.

Let P = (n, ai, ….., a1, x) denote an arbitrary instance of this search problem,

where,

n is the number of element in the list, ai ……….. a1 is the list of elements

and

x is the element searched for.

We use divide-and-conquer strategy to solve this problem. Let small (p, q)

be true for n = 1.

In this case, G(p, q) will take the value i if x = ai, otherwise it will take the

value 0.



Then, g(1) = (1), if P has more than one element, it can be divided (or

reduced) into a new subproblem as follows:

Pick an index q in the range [i, I] and compare x with aq. There are 3

possibilities.

1) x = aq: In this case, the problem P is immediately stored.

2) x < aq: In this case, x has to be searched in the sublist (ai, ai+1, ….., aq–1)

P reduces to (q – i, a1, aq–1, x)

3) x > aq: In this case, x has to be searched in the sublist (aq+1, ……….. , ai)

P reduces to (i – q,aq+1, …………..,ai, x)

Any given problem, P gets divided (reduced) into one new subproblem. This

division takes only (1) time. After a comparison with aq, the instance

remaining to be solved (if any) can be solved by using divide-and-conquer

rule.

If q is always chosen such that aq is the middle element (i.e. q = (n+1)/ 2),

then, the resulting search algorithm is known as binary search.

Algorithm 3.2 shows how binary search works for n elements where n 0.

Algorithm 3.2

1. procedure BINSRCH (x, j)

2. integer low, mid, high, x,j;

3. low 1, high n;

4. while low high do

5. mid (low + high)/2

6. case

7. :x<A(mid):high mid — 1

8. :x>A(mid): low mid + 1

9. :else:j mid; return

10. end case

11. repeat

12. j o

13. endBINSRCH



Suppose there are n elements, then, for successful search there is n

comparisons and for unsuccessful search there is n + 1 comparisons.

To determine the average behaviour, we need to look more closely at the

binary decision tree and equate its size to the number of element

comparison in the algorithm.

The distance of a node from the root is one less than its level.

The internal path length l is the sum of the distances of all internal nodes

from the root.

The external path length E is the sum of the distances of all external

nodes from the root.

Let s(n) = average number of comparisons in a successful search.

u(n) = average number of comparisons in an unsuccessful search.

For successful search

Total distance of all node to the root = l

Average distance of a node from root = n

l

Average number of element comparison for successful search = 1n

l

n

l1nS ––––––––––– (1)

For unsuccessful search

Total distance from node = E

Average distance of any node from root

1n

Elnu

––––––––––– (2)

E (n + 1)

In other words, E n(n + 1)

Substituting in (2) nlognu


2. Average number of element comparison for successful search –––––––.



3.4 Max. and Min.

The problem is to find the maximum and minimum items in a set of n

elements.

Algorithm 3.3

1. procedure SMAXMIN(max, min)

2. integer i, max, min;

3. max min A(1);

4. for I 2 to n do

5. if A(i)>max then max A(i)

6. else if A(i)<min then min A(i)

7. endif

8. endif

9. repeat

10. end SMAXMIN

Best Case: If we have n element, (n – 1) element comparison.

Worst Case: If n elements are in descending order, 2 comparison will take

place.

We have 2(n – 1) element comparison.

Average Case: 1n2

3 element comparison

Algorithm 3.4 MaxMin is a recursive algorithm that finds the maximum and

the minimum of set of elements {a(i),a(i+1)….,a(j)}. The situation of set sizes

one (i = j) and two (i = j – 1) are handled separately. For sets containing

more than two elements, the midpoint is determined (just as in binary

search) and two new subprograms are generated. When the maxima and

minima of these subproblems are determined, the two maxima are

compared and two minima are compared to achieve the solution for the

entire set.



Algorithm 3.4

1. Algorithm MaxMin (i,j, max, min)

2. // a[1:n] is a global array Parameter i and j are integers,

3. // 1 i j < n. The effect is to set max and min to the

4. // largest and smallest values in a [i, j], respectively.

5. {

6. if (i=j) then max:=min:=a[i]; //small(P)

7. else if (i = j – 1) then //Another case of Small (P)

8. {

9. if a [i] < a[j] then

10. {

11. max:=a[j]; min:=a[i];

12. }

13. else

14. {

15. max:=a[i]; min:=a[j]

16. }

17. }

18. else

19. {

20. // If P is not small, divide P into subproblems

21. // Find where to split the set

22. mid:= (i+j)/2;

23. //Solve the subproblems

24. Maxmin (i, mid, max, min);

25. MaxMin (mid+i, j, max1, min1);

26. // Combine the solutions

27. if (max<max 1) then max:=max1;

28. if(min>min1) then min:=min1;

29. }

30. }

Algorithm 3.4 Recursively finds the maximum and minimum

The procedure is initially invoked by the statement MaxMin(1,n,x,y)

Suppose, we simulate MaxMin on the following nine elements:

A: [1] [2] [3] [4] [5] [6] [7] [8] [9]

22 13 –5 –8 15 60 17 31 47



A good way of keeping track of recursive calls is to build a tree by adding a

node each time a new call is made. For this algorithm, each node has four

items of information: i,j, max and min. On the array a [ ] above, the tree of

Fig. 3.1 is produced.

Fig. 3.1: Trees of recursive calls of MaxMin

Examining Fig 3.1, we see that the root node contains 1 and 9 as the values

of i and j corresponding to the initial call to MaxMin. This execution produces

two new calls to MaxMin, where i and j have the values 1, 5 and 6, 9

respectively and thus split the set into two subsets of approximately the

same size. From the tree we can immediately see that the maximum depth

of recursion is four (including the first call). The circled numbers in the upper

left corner of each node represent the orders in which max and min are

assigned values.

If (n) represents this number, then the resulting recurrence relation is

1n0

2n1

2n22/nT2/nT

nT

When n is a power of two, n=2k for some positive integer k, then

22

nT2)n(T



224

nT22

244

nT4

…

….

1ki1

i1k 2)2(T2

22

n3222 k1k

Note that 22

n3 is the best, average and worst-case number of

comparisons when n is a power of two.


3. A good way of keeping track of –––––––– is to build a tree by adding a

node each time a new call is made.

3.5 Merge Sort

We assume throughout that the elements are to be sorted in non-decreasing

order. Given a sequence of n elements (also called keys) a[1], ……., a[n],

the general idea is to imagine them split into two sets a[1], …….. a[n/2]

and a[n/2+1],………..,a[n]. Each set is individually sorted and the resulting

sorted sequences are merged to produce a single sorted sequence of n

elements.

Thus, we have another ideal example of the divide-and-conquer strategy in

which the splitting is into two equal-sized sets and the combining operation

is the merging of two sorted sets into one.

Mergesort (Algorithm 3.5) describes this process very succinctly using

recursion and a function Merge (Algorithm 3.6) which merges two sorted

sets. Before executing MergeSort, the n elements should be placed in a

[1:n]. Then MergeSort (1, n) causes the keys to be rearranged into the non-

decreasing order in a



Algorithm 3.5

1. Algorithm MergeSort (low, high)

2. //a[low:high] is a global array to be sorted.

3. //small (P) is true if there is only one element

4. // to sort. In this case the list is already sorted.

5. {

6. if (low<high> then //If there are more than one element

7. {

8. //Divide P into subproblems

9. //Find where to split the set

10. mid:= (low+high)/2

11. //Solve the subproblems

12. MergeSort (low,mid);

13. MergeSort(mid+1, high);

14. //combine the solutions

15. Merge(low, mid, high)

16. }

17. }

Algorithms 3.6

Algorithm Merge (low, mid, high)

//a[low:high] is a global array containing two sorted

//subsets in a [low:mid} and in a [mid+1 :high]. The

//goal is to merge these two sets into a single set residing

// in a [low:high],b[ ] is an auxiliary global array.

{

h:=low; i:=low; j:=mid+1;

while ((h mid) and (j high)) do

{

If (a[h] a[j] then

{

b[i]: a[h]; h:=h+1;



}

Else

{

b[i]:=a[j]; j:=j+1;

}

i:i+1;

}

If (h>mid) then

for k = j to high do

{

b[i]:=a[k]; i:=i+1;

}

else

For k:=h to mid do

{

b[i]:=a[k]; i:=i+1;

}

for k:=low to high do a[k]:=b[k];

}

Fig. 3.2: Tree of calls of MergeSort (1, 10)



Fig. 3.3: Tree of calls of Merge

Fig. 3.2 is a tree that represents the sequence of recursive calls that are

produced by MergeSort when it is applied to ten elements. The pair of values in

each node are the values of the parameters low and high. Notice how the

Splitting continues until sets containing a single element are produced. Fig. 3.3

is a tree representing the calls to produce Merge by MergeSort.

For example, the node containing 1, 2 and 3 represent the merging of [1:2]

with a[3].

If the time for the merging operation is proportional to ‘n’, then the

computing for Mergesort is described by the recurrence relation.

ttanconsaisc,1ncn2/naT

ttanconsisa,1nanT

When n is a power of 2, n=2k, we can solve this equation by successive

substitutions

cn2

nc

4

nT22)n(T

cn24

nT4

cn24

cn

8

nT24



……

…….

kcn)1(T2k

nlogcnan

It is easy to see that if 2k < n 2k+1, then

T(n) T(2k+1)

Therefore, nlognOnT


4. If the time for the merging operation is proportional to ‘n’, then the

computing for Mergesort is described by the recurrence relation ––––––.

3.6 Quick Sort

In merge sort, the file a[1:n] was divided at its midpoint into subarrays which

were independently sorted and later merged.

In quick sort, the division into two subarrays is made so that the sorted

subarrays do not need to be merged later. This is accomplished by

rearranging the elements in a [1:n] such that a[i] a[j] for all i between 1 and

m and all j between m+1 and n for some m, 1 m n. Thus, the elements in

a [1:m] and a[m+1:n] can be independently sorted. No merge is needed.

The rearrangement of the elements is accomplished by picking some

element of a[ ], say, t=a[s] and then reordering the other elements so that all

elements appearing before t in a[1:n] are less than or equal to t and all

elements appearing after t are greater than or equal to t. This rearranging is

referred to as partitioning.

Function Partition of Algorithm 3.7 accomplishes an in-place partitioning of

the elements of a[m:p – 1]. It is assumed that a[p] a[m] and that a[m] is the

partitioning element. If m = 1 and p – 1 = n, then a[n + 1] must be defined

and must be greater than or equal to all elements in a[1:n]. The assumption

that a[m] is the partition element is merely for convenience; other choices for

the partitioning element than the first item in the set are better in practice.

The function, interchange (a, i, j) exchanges a[i] with a [j].



Algorithm 3.7

1. Algorithm Partition (a,m,p)

2. //within a[m], a[m+1], …., a[p-1] the elements are

3. //rearranged in such a manner that if initially t=a[m],

4. //then after completion a[q]=t for some q between m

5. // and p-1, a[k] t for m k q, and a[k] t

6. //for q<k<p. q is returned. Set a[p]=

7. {

8. V:=a[m]; i:=m; j:=p;

9. repeat

10. {

11. repeat

12. i:=i+1

13. until (a[i]v);

14. repeat

15. j:=j-1;

16. until (a[j]v);

17. if (i<j) then Interchange (a, i, j);

18. } until (ij);

19. a[m]:=a[j]; a[j]:=v; return j;

20. }

1. Algorithm Interchange (a,i,j)

2. //Exchange a[i] with a[j]

3. {

4. p:=a[i];

5. a[i]:=a[j]; a[j]:= p;

6. }



Algorithm 3.8

1. Algorithm QuickSort(p,q)

2. //Sorts the elements a[p],……,a[q] which reside in

3. //global array a[1:n] into ascending order; a[n+1] is

4. //considered to be defined and must be all the

5. // elements in a [1:n]

6. {

7. if (p<q) then //If there are more than one element

8. {

9. // divide p into two subproblems

10. j:=Partition(a, p, q+1);

11. //j is the position of the partitioning element

12. // Solve the subproblems

13. QuickSort(p, j-1);

14. QuickSort(j+1, q);

15. // There is no need for combining solutions.

16. }

17. }

In analyzing QuickSort, we count only the number of element comparisons

C(n). It is easy to see that the frequency count of other operations is of the

same order as C(n). We make the following assumptions: the n elements to

be sorted are distinct, and the input distribution is such that the partition

element v=a[m] in the call to Partition (a,m,p) has an equal probability of

being the ith smallest element, 1 i p-m, in a[m:p-1].

First, let us obtain the worst-case value Cw(n) of C(n). The number of

element comparisons in each call of Partition is at most p – m + 1. Let r be

the total number of elements in all the calls to Partition at any level of

recursion. At level one only one call, Partition (a, 1, n+1), is made and

r = n; at level two at most two calls are made and r = n – 1; and so on. At

each level of recursion, O(r) element comparisons are made by Partition. At

each level, r is atleast one less than the r at the previous level, as the

partitioning elements of the previous level are eliminated. Hence, Cw(n) is

the sum on r, as r varies from 2 to n, or O(n2).

The average value CA(n) of C(n) is much less than Cw(n).



Even though the worst-case time is O(n2), the average time is only O(n log

n). Let us now look at the stack space needed by the recursion. In the worst

case the maximum depth of recursion may be n – 1. This happens, for

example, when the partition element on each call to Partition is the smallest

value in a[m:p-1]. The amount of stack space needed can be reduced to

O(log n) by using an iterative version of Quick Sort in which the smaller of

the two subarrays a[p : j – 1] and a[j+1:q] is always sorted first. Also, the

second recursive call can be replaced by some assignment statements and

a jump to the beginning of the algorithm.

Algorithm 3.9

1. Algorithm QuickSort 2(p, q)

2. //Sorts the elements in a [p: q]

3. {

4. //stack is a stack of size 2 log (n)

5. repeat

6. {

7. while (p<q) do

8. {

9. j:=Partition (a,p,q+1):

10. if (j-p)<(q-j) then

11. {

12. Add (j+1); //Add j+1 to stack

13. Add(q); q:=j-1; // Add q to stack

14. }

15. else

16. {

17. Add(p; //Add p to stack)

18. Add (j-p); p:=j+1; //Add j-1 to stack

19. }

20. }// Sort the smaller subfile

21. if stack is empty then return:

22. Delete (q); Delete (p); //Delete q and p from stack

23. } until (false);

24. }



We can now verify that the maximum stack space needed is O(log n)

Let S(n) be the maximum stack space needed. Then it follows that

1n0

1n2

1nS2

nS which is less than 2 log n


5. The average value CA(n) of C(n) is much less than ––––––––––––.

3.7 Summary

In this unit, we discussed that the complexity of many divide-and-

conquer algorithms is given by recurrences of the form

1n)n(f

b

naT

1n1T

nT (where a and b are known constants).

For successful binary search, n

l1nS and for unsuccessful binary

search, 1n

EnU

.

For Max and Min, 22

n3

is the best, average and worst-case number

of comparisons where n is a power of two.

For Merge sort , we have T(n) = O(n logn)

For quicksort, the worst case time is O(n2), the average time is only

O(n logn).


1. Briefly explain the Binary search Method.

2. Write the Algorithm to find the maximum and minimum items in a set of n

element.

3. Explain the concept of merge sort.

4. Sketch the Algorithm of Quick sort.

5. Write the Algorithm for sorting by Partitioning.



3.9 Answers


1.

1nnfb/naT

1n1TnT

2. 1n

l

3. Recursive calls

4.

ttanconsaisc,1ncn2/naT

ttanconsisa,1nanT

5. Cw(n).

Terminal Questions

1. Let ai, 1 < i < n be a list of elements that are sorted in non-decreasing

order. Consider the problem of determining whether a given element x is

present in the list. If x is present, we are to determine a value j such that

aj = x. If x is not present in the list, then j is to be set to zero.

Let P = (n, ai, ….., a1, x) denote an arbitrary instance of this search

problem, where n is the number of element in the list, ai ……….. a1 is

the list of elements and x is the element searched for. (Refer section

3.3)

2. Refer section 3.4






Unit 4 Greedy Method

Structure

4.1 Introduction

Objectives

4.2 Greedy Method Strategy

4.3 Optimistic Storage on Tapes

4.4 Knapsack Problem

4.5 Job Sequencing with Deadlines

4.6 Optimal Merge Pattern

4.7 Single Source Shortlist Paths

4.8 Summary


4.10 Answers

4.1 Introduction

In the previous unit, we have discussed the different techniques of

algorithms. Now we will study a well known method known as the greedy

method. The greedy method is the most straightforward design technique,

and it can be applied to a wide variety of problems. Most of these problems

have n inputs and require us to obtain a subset that satisfies some

constraints. Any subset that satisfies these constraints is called a feasible

solution. We need to find a feasible solution that either maximizes or

minimizes a given objective function. A feasible solution that does this is

called an optimal solution. There is usually a simple way to determine a

feasible solution but not necessarily an optimal solution.

Objectives


describe the greedy method approach

apply the greedy method for optimistic storage on tapes, knapsack

problem

define the job sequencing problem using greedy method.

4.2 Greedy Method Strategy

The greedy method suggests that one can device an algorithm that works in

stages, considering one input at a time. At each stage, a decision is made



regarding whether a particular input is in an optimal solution. This is done by

considering the inputs in an order determined by some selection procedure.

If the inclusion of the next input into the partially constructed optimal solution

will result in an infeasible solution, then this input is not added to the partial

solution. Otherwise, it is added. The selection procedure itself is based on

some optimization measure. This measure may be the objective function.

This version of the greedy technique is called the subset paradigm.

We can describe the subset paradigm abstractly, but more precisely than

above, by considering the control abstraction in Algorithm 4.1

Algorithm 4.1

1. Algorithm Greedy (a,n)

2. //a[1:n] contains the n inputs

3. {

4. Solution: = ;//initialize the solution

5. for i:=1 to n do

6. {

7. x: =select (a);

8. if feasible (solution, x) then

9. solution:=Union(solution, x);

10. }

11. return solution.

12. }

The function Select selects an input from a [ ] and removes it. The selected

input’s value is assigned to x. Feasible is a Boolean-valued function that

determines whether x can be included into the solution vector. The function

union combines x with the solution and updates the objective function. The

function Greedy describes the essential way that a greedy algorithm will

look, once a particular problem is chosen and the functions select, feasible

and union are properly implemented.

Problems that do not call for the selection of an optimal subset, in the

greedy method we make decisions by considering the inputs in some order.

Each decision is made using an optimization criterion that can be computed

using decisions already made. This version of the greedy method is called

the ordering paradigm.




1. ––––––––– is a Boolean-valued function that determines whether x can

be included into the solution vector.

4.3 Optimistic Storage on Tapes

There are n programs that are to be stored on a computer tape of length l.

Associated with each program i is a length li, 1 i n. Clearly, all programs

can be stored on the tape if and only if the sum of the length of the program

is at most l.

We assume that whenever a program is to be retrieved from this tape, the

tape is initially positioned at the front. Hence, if the programs are stored in

the order I=i1, i2 -----in, the time tj needed to retrieve program ij is proportional

to .Ijk1

ik If all programs are retrieved equally often, then the expected or

mean retrieval time (MRT) is nj1

jtn

1. In the optimal storage on tape

problem, we are required to find a permutation for the n programs so that

when they are sorted on the tape in this order the MRT is minimized. This

problem fits the ordering paradigm.

Minimizing the MRT is equivalent to minimizing ik

jk1nj1

lld

A greedy approach to building the required permutation would choose the

next program on the basis of some optimization measure. One possible

measure would be the d value of the permutation constructed so far. The

next program to be stored on the tape would be one that minimizes the

increase in d. If we have already constructed the permutation i1, i2, ….. ir,

then appending program j gives the permutation i1, i2,….., ir, ir+1=j. This

increases the d value by nj1

jik .ll Since jk1

ikl is fixed and

independent of j, we trivially observe that the increase in d is minimized if

the next program chosen is the one with the least length form among the

remaining programs.



The greedy method simply requires us to store the programs in non-

decreasing order of their lengths. This ordering can be carried out in

O(n log n).

Theorem: If l1 l2 …… ln, then the ordering ij=j, 1 j n, minimizes

k

1j

ij

n

1k

l over all possible permutations of the ij.

Proof: Let l=i1,i2,…..,in be any permutation of the index set {1, 2, …..,n}.

Then

n

1k

jk

k

1j

ij

n

1k

l1knlId

If there exist a and b such that a<b and lia > Iib, then interchanging ia and ib

results in a permutation I with

iaib

bk

ik l1bnl1anl1knId

akk

Subtracting Id from Id ,we obtain

iaibibia ll1bnll1anIdId

ibia llab

> 0

Hence, no permutation that is not in non-decreasing order of the li’s can

have minimum d. It is easy to see that all permutations in non-decreasing

order of the li’s have the same d value. Hence, the ordering defined by ij=j,

1 j n minimizes the d value.

The tape storage problem can be extended to several tapes. If there are

m>1, tapes, T0, ….., Tm-1, then the programs are to be distributed over these

tapes. For each tape a storage permutation is to be provided If lj is the

storage permutation for the subset of programs on tape j, then d(lj) is as

defined earlier. The total time 1mj0

jldisTD . The objective is to store the

programs in such a way as to minimize TD.



The obvious generalization of the solution for the one-tape case is to

consider the program in non-decreasing order of li’s. The program currently

being considered is placed on the tape that results in the minimum increase

in TD. This tape will be the one with the least amount of tape used so far. If

there is more than one tape with this property, then the one with the

smallest index can be used. If the jobs are initially ordered so that

,l.....ll n21 then the first m programs are assigned to tapes T0,…..,Tm-1

respectively. The next m programs will be assigned to tapes T0,…….., Tm-1

respectively.

The General rule is that program i is stored on tape T i mod m• On any given

tape the programs are stored in non-decreasing order of their lengths.

Algorithm 4.2 presents this rule in pseudocode. It assumes that the

programs are ordered as above. It has a computing time of (n) and does

not need to know the program lengths.

Algorithm 4.2

1. Algorithm Store (n,m)

2. //n is the number of programs and m the number

3. // of tapes

4. {

5. j:=0; //Next tape to store on

6. for i:=1 to n do

7. {

8. write(”append program”, i,”

9. to permutation for tape”,j);

10. j:= (j+1)mod m,

11. }

12. }


2. A greedy approach to building the required permutation would choose

the next program on the basis of some–––––––––––––.

3. The ordering defined by ij=j, 1 j n, –––––––––– the d value.




Given n objects and a knapsack or bag. Object i has a weight wi, and the

knapsack has a capacity m. If a fraction xi, 0 xi 1, of object i is placed into

the knapsack, then a profit of Pi xi is earned. The objective is to obtain a filling of

the knapsack that maximizes the total profit earned. Since the knapsack

capacity is m, we require the total weight of all chosen objects to be atmost m.

Formally, the problem can be stated as

Maxmise ni1

ii xP -------------------- 1

Subject to ni1

ii mxw ------------- 2

And ni1,1x0 i ------------------ 3

The profits and weights are positive numbers. A feasible solution (or filling)

is any set (x1, …., xn) satisfying 2 and 3 above. An optimal solution is a

feasible solution for which 1 is maximized.

The Greedy knapsack algorithm requires O(n) time. The Greedy knapsack

algorithm is given below.

Algorithm 4.3

Algorithm Greedyknapsack (m,n)

//p[1:n} and w[1:n] contain the profits and weights respectively

//of the n objects ordered such that p[i]/w[i] p[i+1]/w[i+1]

//m is the knapsack size and x[1:n] is the solution vector

{

for i:=1 to n do x[i]:=0.0;//Initialize x

u:=m;

for i:=1 to n do

{

if (w[i]>u) then break;

x[i]:=1.0; U:=U-w[i];

}

if (i n) then x[i]:=u/w[i];

}


4. The profits and weights are –––––––– numbers.



4.5 Job Sequencing with Deadlines

We are given a set of n jobs. Associated with job i is an integer deadline

di 0 and a profit pi > 0. For any job i the profit pi is earned iff the job in

completed by its deadline. To complete a job, one has to process the job on

a machine for one unit of time. Only one machine is available for processing

jobs. A feasible solution for this problem is a subset J of jobs such that each

job in this subset can be completed by its deadline. The value of a feasible

solution J is the sum of the profits of the jobs in J, or

Ii

ip . An optimal

solution is a feasible solution with maximum value. Since the problem

involves the identification of a subset, it fits the subset paradigm.

Algorithm 4.4 demonstrates the greedy algorithm for sequencing unit time

jobs with deadlines and profits

Algorithm 4.4 1. Algorithm JS (d,j,n)

2. //d[i] 1, 1 i n are the deadlines, n 1. The jobs

3. // are ordered such that p[1] p[2] .... p[n].

4. //J[i] is the ith job in the optimal solution, 1 i k.

5. //Also, at termination d [J[i]] d[J[i+1]], 1 i k.

6. {

7. d[0]:=J[0]:=0;//lnitialize

8. J[1]:=1 ;// include job 1

9. K:=1;

10. for i:=2 to n do

11. {

12. //Consider jobs in non-increasing order of p[i]

13. //Find position for i and check feasibility for insertion

14. r:=k;

15. While((d[J[r]]>d[i]) and (d[J[r]] r)) dp r:= r – 1;

16. if ((d[J[r]] d[i]) and (d[i]>r)) then

17. {

18. // Insert i into J[ ].

19. for q:=k to (r+1) step – 1 do J[q+1]:=J[q];

20. J[r+1]:=i; k:=k+1;

21. }

22. }

23. return k;

24. }



Theorem: If p1/w1 p2/w2 ….. pn/wn. Then, Greedy knapsack algorithm

generates an optimal solution to the given instance of the knapsack

problem.

Proof: Let x = (x1, …….,xn) be the solution generated by Greedy knapsack

algorithm. If all the xi equal one, then clearly the solution is optimal. So, let j

be the least index such that xj 1. From the algorithm it follows that xi=1 for

1 i<j, xi = 0 for j<i n and 0 xj<1. Let y=(y1, ……, yn) be an optimal solution

we can assume that wjyi = m. Let k be the least index such that yk xk.

Clearly, such a k must exist. It also follows that yk<xk. To see this, consider

the three possibilities k<j, k=j or k>j.

1. If k<j, then xk=1, But, yk xk, and so yk < xk.

2. k=j, then since wiyi = m and yi = xi for 1 i<j. it follow that either yk<xk

or wiyi = m

3. k > j, then wiyi = m, and this is not possible.

Now, suppose we increase yk to xk and decrease as many of (yk+1,……., yn)

as necessary so that the total capacity used is still on. This results in a new

solution z=(z1, …….., zn) with zi=xi, 1 i k and k<i nwi (yi - zi) = wk(zk-yk),

Then, for z we have

ni1 ni1

iik

k

nik

iikkkii

nik i

iiiikkk

ni1

ii

ni1

ii

ypw

pzywyzyp

w

pwzywYzypzp

If iiii ypzp then y could not have been an optimal solution. If these

sums are equal, then either z=x and x is optimal, or z x. In the latter case,

repeated use of the above argument will either show that y is not optimal or

transform y into x and thus show that x too is optimal.

Greedy algorithm for sequencing unit time jobs with deadlines and profits

For JS, there are two possible parameters in terms of which its complexity

can be measured. We can use n, number of jobs and s, the number of jobs

included in the solution J. The while loop of line 15 in Algorithm 4.4 is

iterated at most K times. Each iteration takes (1) time. If the condition; of

line 16 is true, then lines 19 and 20 are executed. These lines require (k-r)



time to insert job i. Hence, the total time for each iteration of the for loop of

line 10 is (k). This loop is iterated n-1 times. If s is the final value of k, that

is, S is the number of jobs in the final solution, then the total time needed by

algorithm JS is (Sn). Since S n, the worst-case time, as a function of n

alone is (n2). If we consider the job set pi=di=n — i + 1, 1 i n, then

algorithm JS takes (n2) time to determine J. Hence. The worst-case

computing time for JS is (n2) In addition to the space needed for d, JS

needs (S) amount of space for J.


5. If––––––––––, then Greedy knapsack algorithm generates an optimal

solution to the given instance of the knapsack problem.

4.6 Optimal Merge Pattern

When more than two sorted files are to be merged together, the merge can

be accomplished by repeatedly merging sorted files in pairs. Thus, if files x1,

x2, x3, and x4 are to be merged, we could first merge x1 and x2 to get a file yi.

then we could merge yi and x3 to get y2. Finally, we could merge y2 and x4 to

get the desired sorted file. Alternatively, we could first merge x1 and x2

getting yi, then merge x3 and x4 and get y2, and finally merge y1 ad y2 and

get the desired sorted file. Given n sorted files, there are many ways in

which we can pairwise merge them into a single sorted file. Different

pairings require differing amounts of computing time. The problem we

address ourselves to now is that of determining of an optimal way to

pairwise merge n sorted files. Since this problem calls for an ordering

among the pairs to be merged, it fits the ordering paradigm.

A greedy attempt to obtain an optimal merge path. Since merging of an n-

record file and an m-record file requires possibly n + m record moves, the

obvious choice for a selection criterion is at each step merge the two

smallest size files together. Thus, if we have files (x1,…….,x5) with sizes (20,

30, 10, 5, 30), our greedy rule would generate the following merge pattern:

merge x4 and x3 to get z1(lz1I=15), merge z1 and x1 to get z2(|z2| = 35) merge

x2 and x5 to get z3 (|z3|=60) and merge z2 and z3 to get the answer z4. The

total number of record moves is 205. One can verify that this is an optimal

merge pattern for the given problem instance.



The merge pattern such as the one just described will be referred to as a

two way merge pattern (each merge step involves the merging of two files).

The two-way merge pattern can be represented by binary merge trees. Fig.

4.1 shows a binary merge tree representing the optimal merge pattern

obtained for the above five files.

Fig. 4.1: Binary merge tree representing a merge pattern

The leaf nodes are drawn as squares and represent the given five files.

These nodes are called external nodes. The remaining nodes are drawn as

circles and are called internal nodes. Each internal node has exactly two

children, and it represents the file obtained by merging the files represented

by its two children. The number in each nodes is length (no. of records) of

the file represented by that node.

The external node x4 is at a distance of 3 from the root z4 (a node at level i is

at a distance of i – 1 from the root) Hence, the records of files x4 are moved

3 times, once to get z1, once again to get z2, and finally, one more time to

get z4. If di is distance from the root to the external node for file xi and qi, the

length of xi is then the total number of record moves, for this binary merge

tree it is n

1i

ii qd

This sum is called the weighted external path length of the tree.

tree node = record{

treenode*lchild; treenode * rchild;

integer weight;



In the algorithm below we discuss how to generate a two-way merge tree

Algorithm 4.5

1. Algorithm Tree (n)

2. //list is a global list of n single node

3. //binary trees as described above

4. {

5. for i:=1 to n–1 do

6. {

7. pt:=new treenode; //Get a new tree node

8. (pt /child):=Least(list);//Merge two trees with

9. (pt. /child) :=Least(list) ;//smallest lengths

10. (pt weight):=((pt lchild) weight)

11. +((pt rchild) weight);

12. Insert (list,pt):

13. }

14. return Least (list); //Tree left in list is the merge tree

15. }

An optimal two-way merge pattern corresponds to a binary merge tree with

minimum weighted external path length. The function Tree of Algorithm 4.5

uses the greedy rule stated earlier to obtain a two-way merge tree for n files.

The algorithm has as input a list of n trees.

Each node in a tree has three fields, /child, rchild and weight. Initially, each

tree in list has exactly one node. This node is an external node and has

lchild and rchild fields zero whereas weight is the length of one of n files to

be merged. During the course of the algorithm, for any tree in list with root

node t, t weight is the length of the merged file it represents (t weight

equals the sum of the lengths of the external nodes in tree t). Function Tree

uses two functions, Least (list) and Insert (list, t). Least (list) finds a tree in

list whose root has least weight and returns a pointer to these trees. These

trees are removed from list. Insert (list, t) inserts the tree with root t into list.

The main for loop in Algorithm 4.5 is executed n – 1 times. If list is kept in

non-decreasing order according to the weight value in the roots, then Least

(list) requires only O(1) time and Insert (list, t) can be done in O(n) time.

Hence, the total time taken is O(n2).



Computing time for Tree is O(n log n). Some speedup may be obtained by

combining the Insert of line 12 with the Least of line 9.


6. Each node in a tree has three fields ––––, ––––– and ––––

4.7 Single Source Shortlist Paths

We are given a directed graph G = (V, E), a weighting function cost for the

edges in a source vertex v0. The problem is to determine the shortest paths

from v0 to all the remaining vertices of G. It is assumed that all the weights are

positive. The shortest path between v0 and some other node v is an ordering

among a subset of the edges. Hence, this problem fits the ordering paradigm.

To formulate a greedy-based algorithm to generate the shortest paths, we

must conceive of a multistage solution to the problem and also an

optimization measure. One possibility is to build the shortest paths one by

one. As an optimization measure, we can use the sum of the lengths of all

paths so far generated. For this measure to be minimized, each individual

path must be of minimum length. If we have already constructed i shortest

path, then using this optimization measure, the next path to be constructed

should be the next shortest minimum length path. The greedy way to

generate the shortest paths from v0 to the remaining vertices is to generate

these paths in non-decreasing order of path length. First, a shortest path to

the nearest vertex is generated. Then a shortest path to the second nearest

vertex is generated and so on.

Fig. 4.2: Graph & Shortest paths from vertex 1 to all destinations

For the graph of fig 4.2 the nearest vertex to v0 = 1 is 4 (cost [1,4] =10). The

path 1, 4 is the first path generated. The second nearest vertex to node 1 is



5 and the distance between 1 and 5 is 25. The path 1, 4, 5 is the next path

generated. In order to generate the shortest paths in this order, we need to

be able to determine.

1) The next vertex to which a shortest path must be generated and

2) A shortest path to this vertex

Let S denote the set of vertices (including v0) to which the shortest paths

have already been generated. For w not in S, let dist[w] be the length of the

shortest path starting from v0, going through only those vertices that are in S

and ending at w. We observe that:

1. If the next shortest path is to vertex u, then the path begins at v0, and

goes through only those vertices that are in S. To prove this, we must

show that all the intermediate vertices on the shortest path to u are in S.

Assume there is a vertex w on this path that is not in S. Then, the v0 to u

path also contains a path from v0 to w that is of length less than the v0 to

u path. By assumption, the shortest paths are being generated in non

decreasing order of path length, and so the shorter path v0 to w must

already have been generated. Hence, there can be no intermediate

vertex that is not in S.

2. The destination of the next path generated must be that of vertex u

which has the minimum distance, dist [u], among all vertices not in S.

This follows from the definition of dist and observation 1. In case there

are several vertices not in S with the same dist, then any of these may

be selected.

3. Having selected a vertex u in observation 2 and generated the shortest

v0 to u path, vertex u becomes a member of S. At this point, the length of

the shortest paths starting at v0, going through vertices only in S, and

ending at a vertex w not in S may decrease; that is, the value of dist [w]

may change. If it does change, then it must be due to a shorter path

starting at v0 and going to u and then to w. The intermediate vertices on

the v0 to u path and the u to w path must all be in S. Further, the v0 to u

path must be the shortest such path; otherwise dist [w] is not defined

properly. Also, the u to w path can be chosen so as not to contain any

intermediate vertices. Therefore, we can conclude that if dist [w] is to

change, then it is because of a path from v0 to u to w, where, the path



from v0 to u is the shortest path and the path from u to w is the edge

(u, w). The length of this path is dist [u] + cost [u, w].

The above observations lead to a simple algorithm 4.6 for the single source

shortest path problem. These algorithms (known as Dijkstra’s algorithm)

only determines the lengths of the shortest path from v0 to all other vertices

in G.

Algorithm 4.6

1. Algorithm shortest paths (v, cost dist, n)

2. // dist [j], 1 j n, is set to the length of the shortest

3. // path from vertex v to vertex in a digraph G with n

4. // vertices dist [v] is set to zero. G is represented by its

5. // cost adjacency matrix cost [1:n, 1:n]

6. {

7. for i: = 1 to n do

8. { // Initialise S

9. S [i] : = false; dist [i] : = cost [v, i];

10. }

11. S[v]: = true; dist[v] : = 0.0; //put v in s

12. for num:=2 to n – 1 do

13. {

14. // Determine n –1 paths from v

15. choose u from among those vertices not

16. in S such that dist [u] is minimum;

17. S[u] : = true; //put u in S

18. for (each w adjacent to u with S[w] = false) do

19. // update distances

20. if (dist [w] > (dist [u] + cost [u, w])) then

21. dist [W]: = dist [u] + cost [u, W];

22. }

23. }

Above algorithm is also known as Greedy algorithm to generate shortest

paths.

In the function shortest paths (Algorithm 4.6) it is assumed that the n

vertices of G are numbered 1 through n. The set S is maintained as a bit



array with S [i] = 0 if vertex I is not in S and S [i] = 1. It is assumed that the

graph itself is represented by its cost adjacency matrix with cost [i, j]’s being

the weight of the edge <i, j>. The weight cost [i, I] is set to some large

number, , in case the edge <i, j> is not in E (G). For i = j, cost [i, j] can be

set to any non negative number without affecting the outcome of the

algorithm.

The time taken by the algorithm on graph with n vertices is O(n2). To see

this, note that the for loop of line 7 in Algorithm 4.6 takes (n) time. The for

loop of line 12 is executed n – 2 times. Each execution of this loop requires

O(n) time at lines 15 and 16 to select the next vertex and again at the for

loop at line 18 to update dist. So the total time for this loop is O(n2). In case

a list t of vertices currently not in S is maintained, then the number of nodes

on this list would at any time be n-num. This would speed up lines 15 and 16

and the for loop of line 18, but the asymptotic time would remain O(n2).


7. The shortest path between v0 and some other node v is an –––––––

among a subset of the edges.

8. The time taken by the algorithm on graph with n vertices is –––––––––.

4.8 Summary

In this unit we studied that the greedy method for optimally storage on

tapes algorithm takes (n) computing time.

The greedy method for knapsack problem takes (n2) time to determine

the solution.

The greedy method for optimal merge pattern has O(nlogn) computing

time for tree

The greedy method for single source shortlist path problem has (n2)

asymptotic time.


1. Explain the Greedy Method

2. Explain the concept of Optimal Storage on Tapes

3. Briefly write in your own words the concept of Knapsack Problem

4. Explain the technique involved in Optimal Merge Pattern



5. Do you agree that the single source shortlist paths problem fits the

ordering paradigm ? Justify your answer

6. Write an algorithm for single source shortest path problem

7. Write the Greedy-Knapsack algorithm

4.10 Answers


1. Feasible

2. Optimization measure

3. Minimizes

4. Positive

5. p1/w1 p2/w2 ….. pn/wn

6. lchild, rchild and weight.

7. ordering

8. O(n2)

Terminal Questions

1. The greedy method suggests that one can device an algorithm that

works in stages, considering one input at a time. At each stage, a

decision is made regarding whether a particular input is in an optimal

solution. This is done by considering the inputs in an order determined

by some selection procedure. (Refer Section 4.2)

2. We assume that whenever a program is to be retrieved from this tape,

the tape is initially positioned at the front. Hence, if the programs are

stored in the order I=i1, i2 -----in, the time tj needed to retrieve program ij is

proportional to .Ijk1

ik If all programs are retrieved equally often, then

the expected or mean retrieval time (MRT) is nj1

jtn

1. (Refer

Section 4.3)








Unit 5 Dynamic Programming Structure

5.1 Introduction

Objectives

5.2 Dynamic Programming Strategy

5.3 Multistage Graphs

5.4 All Pair Shortest Paths

5.5 Traveling Salesman Problems

5.6 Summary


5.8 Answers

5.1 Introduction

In the previous unit, we discussed the concept of greedy method but when

the optimal solutions to subproblems are to be retained in order to avoid

recomputing their values, we use dynamic programming. Dynamic

programming is an algorithm design method that can be used when the

solution to problem can be viewed as the result of sequence of decisions.

The principle of optimality states that an optimal sequence of decisions has

the property that whatever the initial state and decision are, the remaining

decisions must constitute an optimal decision sequence with regard to the

state resulting from the first decision.

The essential difference between the greedy method and dynamic

programming is that in the greedy method only one decision sequence is

ever generated. In dynamic programming, many decision sequences may

be generated. However, sequences containing suboptimal subsequences

cannot be optimal and so not be generated.

Objectives


apply Dynamic Programming in Multistage graphs

solve all pair shortlist paths using Dynamic Programming

solve Traveling Salesman Problems using Dynamic Programming.



5.2 Dynamic Programming Strategy

Because of the use of the principle of optimality, decision sequences

containing subsequences that are suboptimal are not considered. Although

the total number of different decision sequences is exponential to the

number of decisions (if there are d choices for each of the n decisions to be

made then there are dn possible decision sequences). Dynamic

programming algorithms often have a polynomial complexity.

Another important feature of the dynamic programming approach is that

optimal solutions to subproblems are retained so as to avoid recomputing

their values. The use of these tabulated values makes it natural to recast the

recursive equations into an iterative algorithm.


1. Dynamic programming algorithms often have a ––––––––––––

5.3 Multistage Graphs

A multistage graph G = (V,E) is a directed graph in which the vertices are

partitioned into K 2 disjoint sets Vi, 1 i k. In addition, if <u, v> is an

edge in E, then u vi, and v vi+1 for some i, 1 i < k. The sets V1 and Vk

are such that iVV k1 Let s and t, respectively, be the vertices in V1

and Vk. The vertex s is the source, and t the link. Let c(i, j) be the cost of

edge <i, j>. The cost of a path from s to t is the sum of the costs of the

edges on the path. The multistage graph problem is to find a minimum-cost

path from s to t. Each set Vi defines a stage in the graph. Because of the

constraints, on E, every path from s to t starts in stage 1, goes to, stage 2,

then to stage 3, then to stage 4, and so on, and eventually terminates in

stage K.



Fig. 5.1: A minimum-cost s to t path is indicated by the broken edges.

Many problems can be formulated as multistage graph problems. We give

only one example. Consider a resource allocation problem in which n units

of resource are to be allocated to r projects. If j 0 j n units of the resource

are allocated to project i, then the resulting net profit is N(i,j). The problem is

to allocate the resource to the r projects in such a way as to maximize total

net profit. This problem can be formulated as an r+1 stage graph problem as

follows:

Stage i, 1 i r, represents project i. There are n+1 vertices V(i, j), 0 j n,

associated with stage i, 2 i r. Stages 1 and r+1 each have one vertex,

V(1,0) = s and V(r+1,n)=t, respectively. Vertex V(i,j), 2 i r, represents the

state in which a total of j units of resource have been allocated to projects

1,2, ….. ,i –1. The edges in G are of the form <V(i,j),V(i+1, l)> for all j l and

1 i < r. The edge <V(i,j, V(i+1, l)>, j<l, is assigned a weight or cost of N(i, l-j)

and corresponds to allocating l-j units of resource to project i, 1 i < r. In

addition, G has edges of the type <v(r,j), v(r+1,n)>. Each such edge is

assigned a weight of )}p,r(N{maxjnp0

. It should be easy to see that an

optimal allocation of resources is defined by a maximum cost s to t path.

This is easily converted into a minimum-cost problem by changing the sign

of all the edge costs.

A dynamic programming formulation for a k-stage graph problem is obtained

by first noticing that every s to t path is the result of a sequence of k – 2



decisions. The ith decision involves determining which vertex in Vi+1, 1 i k

– 2, is to be on the path. It is easy to see that the principle of optimality

holds. Let p(i,j) be a minimum-cost path from vertex j in Vi of vertex t. Let

cost(i,j) be the cost of this path. Then, using the forward approach, we

obtain

)}l,1i(tcos)l,j(c{min)j,i(tcos

El,j1iVl

…………(1)

Before writing an algorithm for a general k-stage graph, let us impose an

ordering on the vertices in V. This ordering makes it easier to write the

algorithm. We require that the n vertices in V are indexed 1 through n.

Indices are assigned in order of stages. First, s is assigned index 1, then

vertices in V2 are assigned indices, then vertices from V3, and so on. Vertex

t has index n. Hence, indices assigned to vertices in Vi. As a result of this

indexing scheme, cost and d can be computed in the order n-1, n-2..., 1.

The first subscript in cost, p and d only identifies the stage number and is

omitted in the algorithm. The resulting algorithm in pseudocode is F Graph,

given in Algorithm 5.1

Algorithm 5.1

1. Algorithm FGraph (G,k,n,p)

2. //The input is a K-stage graph G=(V,E) with n vertices

3. //indexed in order of stages. E is a set of edges and c[i,j]

4. //is the cost of <i, j> p[1:k] is a minimum-cost path.

5. {

6. cost[n] :=0.0;

7. for j:=n-1 to 1 step – 1do

8. {//compute cost[j]

9. Let r be a vertex such that <j,r> is an edge

10. of G and c[j,r]+cost[r] is minimum:

11. cost[j] :=c[j, r]+cost[r]:

12. d[j]:r;

13. }

14. //Find a minimum-cost path

15. p[i]:=1; p[k]:=n;

16. for j:=2 to k-1 do p[j]:=d [p-1]];

17. }



In the above algorithm, we study multistage graph pseudocode

corresponding for the forward approach.

The complexity analysis of the function F Graph is fairly straightforward. If G

is represented by its adjacency lists, then r in line 9 of Algorithm 5.1 can be

found in time proportional to the degree of vertex j, Hence, if G has IEl

edges, then time for the for loop of line 7 is (lVl+lEI). The time for the loop

of line 16 is (K). Hence, the total time is (IVI+lEl). In addition to the space

needed for input, space is needed for cost[ ], d[ ], and p[ ].

The multistage graph problem can also be solved using the backward

approach. Let bp(i,j) be a minimum-cost path from vertex s to a vertex j in Vi.

Let bcost (I,j) be the cost bp(i,j). From the backward approach we obtain.

)}j,l(c)l,1i(tcosb{min)j,i(tcosb

Ej,l1iVl

………….5.2

Algorithm 5.2

1. Algorithm Bgraph (G, k, n, p)

2. // same function as Fgraph

3. {

4. bcost[1]:= 0.0;

5. for j:=2 to n do

6. {// compute bcost [j]

7. let r be such that <r,j> is an edge of

8. G and bcost [r] + c [r,j] is minimum;

9. bcost [j]: = bcost [r] + c[r,j];

10. d[j]:=r;

11. }

12. // Find a minimum-cost path

13. p[1]:=1;p[k]:=n;

14. for j:=k-1 to 2 do p[j]:=d[p [j+1]];

15. }

In the above algorithm we study, the multistage graph pseudocode

complexity to backwards approach



This algorithm has the same complexity as F graph provided G is now

represented by its inverse adjacency lists (i.e., for each vertex v, we have a

list of vertices w such that <w,v> E).

It should be easy to see that both F graph and B graph work correctly even

on a more generalized version of multistage graphs. In this generalization,

the graph is permitted to have edges <u.v> such that u Vi, V V1 and i<j.


2. A multistage graph G = (V,E) is a directed graph in which the vertices

are ––––––––––––into K 2 disjoint sets Vi, 1 i k.

5.4 All Pair Shortest Paths

Let G = (V, E) be a directed graph with n vertices and let cost be a cost

adjacency matrix for G such that cost (i, i) = 0, 1 i n. Then cost (I,j) is the

length (or cost) of edge <i,j> if <i,j> E(G) and cost (i,j) = if i j and <i,j>

E(G). The all pairs shortest path problem is to determine a matrix A such

that A (i,j) is the length of a shortest path from i to j.

Let us examine a shortest i to j path in Gi j. This path originates at vertex i

and goes through some intermediate vertices (possibly none) and

terminates at vertex j. We can assume that this path contains no cycles for if

there is a cycle, then this can be deleted without increasing the path length.

If k is an intermediate vertex on this shortest path, then the subpaths from i

to k and from k to j must be shortest paths from i to k and k to j respectively.

Otherwise, the i to j path is not of minimum length. So the principle of

optimality holds. This alerts us to the prospect of using dynamic

programming. If k is the intermediate vertex with highest index, then the i to

k path is a shortest i to k path in G going through no vertex with index

greater than k–1. Similarly, the k to j path is a shortest k to j path in G going

through no vertex of index greater than k–1. We can regard the construction

of a shortest i to j path as first requiring a decision as to which is the highest

indexed intermediate vertex k. once this decision has been made, we need

to find two shortest paths, one from i to k and the other from k to j. Neither of

these may go through a vertex with index greater than k–1.Using Ak(i,j) to

represent the length of a shortest path from i to j going through no vertex of

index greater than k, we obtain



)j,i(tcos),j,k(A)k,i(A{minmin)j,i(A 1k1k

nk1……….5.3

Clearly, A°(i,j) = cost (i, j), 1 i n, 1 j n. We can obtain a recurrence for

Ak(j,j) using argument similar to that used before. A shortest path from i to j

going through no vertex higher than k either goes through vertex k or it does

not. If it does, Ak (i,j) = Ak–1 (i,k) + Ak–1 (k, j). If it does not, then no

intermediate vertex has index greater than k-1. Hence,

Ak(i,j)=Ak–1 (i, j). Combining, we get

A(i,j)=min{A k-1(i, j), Ak-1 (i,k)+Ak-1(k,,j)},k>1 --------------- (5.4)

Recurrence (5,4) can be solved for An by first computing A1, then A2, then A3

and so on. Since there is no vertex in G with index greater than n,

A (i,j) = A(i,j)

Algorithm Allpaths (cost, A, n)

Algorithm 5.3

1. //cost[1:n, 1:n] is the cost adjacency matrix of a graph

2. //with n vertices; A[I, j] is the cost of a shortest path from

3. //vertex i to vertex j. cost [I, j] = 0.0, for 1 i n.

4. {

5. for i:=1 to n do

6. for j:=1 to n do

7. A[i, j]:=cost [i, j]; //copy cost into A.

8. for k:=1 to n do

9. for i:=1 to n do

10. for j:=1 to n do

11. A[i, j]:=min(A[i, j], A[i, k]+A[k, j];)

12. }

Above algorithm functions to compute the length of shortest paths

Function All paths computer An (i.j). The computation is done in place so the

superscript on A is not needed. The reason this computation can be carried

out in-place is that Ak (i,k) = Ak-1 (i, j) and Ak (k, j) = Ak-1 (k, j). Hence, when

Ak is formed, the kth column and row do not change. Consequently, when

Ak(i,j) is computed in line II of Algorithm 5.3, A (i,k) = Ak-l (i,k) = Ak (i, k) and

A(k,j) = Ak-l (kJ) = Ak (k,j). So, the old values on which the new values are

based do not change on this iteration.



Let M = max {cost (i,j)l<i,j> E (G)}. It is easy to see that An (i,j) (n–1)M.

From the working of all paths, it is clear that if <i,j> E (G) and i j, then we

can initialize cost (i,,j) > (n-I) M,. If, at termination, A(i,j)>(n-1), then there is

no directed path from i to j in G.

The time needed by All paths (Algorithm 5.3) is especially easy to determine

because the looping is independent of the data in the matrix A. Line 11 is

iterated n3 times, and so the time for all paths is (n3).


3. The time needed by All paths is easy to determine, because the looping

is –––––––––––– of the data in the matrix .

5.5 Travelling Salesman Problem

A tour of G is a directed simple cycle that includes every vertex in V. The

cost of the tour is the sum of the cost of the edges on the tour. The travelling

salesperson problem is to find a tour of minimum cost.

Let G=(V,E) be a directed graph with edges cost cij. The variable cij is

defined such that cij = 0, cij> o for all i and j and cij = if < i,j> E. Let lVl = n

and assume n>1. A tour is a simple path that starts and ends at vertex 1.

Every tour consists of an edge < 1, k> for some k V — {1} and a path from

vertex k to vertex 1. The path from vertex k to vertex 1 goes through each

vertex in V-{1,k} exactly once. It easy to see that if the tour is optimal, then

the path from k to 1 must be a shortest k to 1 path going through all vertices

in V– {1,k). Hence, the principle of optimality holds. Let g (i,s) be the length

of a shortest path starting at vertex i, going through all vertices in S, and

terminating at vertex 1. The function g(1, V-{1}) is the length of an optimal

salesperson tour. From the principal of optimality if follows that

})}k,j{V,k(gc{min})1{V,1(g k1nk2

……….5.5

Generalizing (5.5), we obtain (for i s)

})}j{S,j(gc{min)S,i(g ijSj

…………..5.6

Equation 5.5 can be solved for g (1, V- {1}) if we know g (k, V-{1,k}) for all

choices of k. The g values can be obtained by using (5.6). Clearly, g (i, ) =

Ci1, i 1 n.



Hence, we can use (5.6) to obtain g(i,S) for all S of size 1. Then, we can

obtain g (i, s) for S with |S|=2, and so on. When |S|< n-1, the values of i and

S for which (i, S) is needed are such that

i 1, 1 S, and i S.

Let N be the number of g (i, S)’s that have to be computed before (5.6) can

be used to compute g (i,v-{i}). For each value of |S| there are n-1 choices for

i. The number of distinct sets S of size k not including l and i is (kn-2). Hence,

2n

0k

2n2n 21nk1nN

An algorithm that proceeds to find an optimal tour by using (5.5) and (5.6)

will require (n22n) time by using computation of g(i,s) with |S|=k ,require k-1

comparisons when solving (5.6). This is better than enumerating all n!

different tours to find best one. The most serious drawback of this dynamic

programming solution is, O(n2n). This is too large even for modest values

of n.


4. A ––––––– of G is a directed simple cycle that includes every vertex in

V.

5.6 Summary

In this unit, we studied that Dynamic programming is an algorithm

design method that can be used when the solution to a problem can be

viewed as the result of a sequence decisions.

Optimal sequence of decisions has the property that whatever the initial

state and decision are, the remaining decisions must constitute an

optimal decision sequence with regard to the state resulting from the first

decision, this is called as the principle of optimality.

We also studied the concept of travelling salesmen problems using the

concept of graphs.


1. Explain the concept of Multistage Graphs

2. Explain the Concept of travelling Salesman Problem

3. Write an Algorithm to obtain a minimum cost s-t path



5.8 Answers


1. Polynomial complexity

2. partitioned

3. independent

4. tour

Terminal Questions

1. A multistage graph G = (V,E) is a directed graph in which the vertices

are partitioned into K 2 disjoint sets Vi, 1 i k. In addition, if <u, v> is

an edge in E, then u vi, and v vi+1 for some i, 1 i < k. The sets V1

and Vk are such that iVV k1 Let s and t, respectively, be the

vertices in V1 and Vk. The vertex s is the source, and t the link. Let c(i, j)

be the cost of edge <i, j>. The cost of a path from s to t is the sum of the

costs of the edges on the path. The multistage graph problem is to find a

minimum-cost path from s to t. Each set Vi defines a stage in the graph.

(Refer Section 5.3)





Unit 6 Backtracking Structure

6.1 Introduction

Objectives

6.2 Backtracking Strategy

6.3 8-Queens Problem

6.4 Sum of Subsets


6.6 Summary


6.8 Answers

6.1 Introduction

In the previous unit, we discussed the concept of dynamic programming and

now in the search for fundamental principles of algorithm design, we will

discuss about backtracking, which represents one of the most general

techniques. Many problems which deal with searching for a set of solutions

or which ask for an operational solution satisfying some constraints can be

solved using the backtracking formulation. The name backtrack was first

coined by D.H Lehmer in the 1950s. Early workers who studied the process

were R.J. Woler, who gave an algorithmic account of it in 1960, and S.

Golomb and L.Barmert who presented a very general description of it in the

form of applications.

Objectives


analyze the 8-queen problem

solve the problem of the Sum of subsets

use the applications of the Knapsack problem.

6.2 Backtracking Strategy

In many applications of the backtrack method, the desired solution is

expressible as an ntuple (x1,……..,xn) where the xis are chosen from some

finite set si often the problem to be solved calls for finding one vector that

maximizes a criterion function p(x1,....xn). Sometimes it seeks all vectors that

satisfy P. For example, sorting the array of integers in a [1:n] is a problem



whose solution is expressible by an n-tuple, were xi is the index in a of ith

smallest element. The criterion function P is the inequality a[xi] a[xi+1] for

1 i < n. The set Si is finite and includes the integer 1-through n. Through

sorting is not usually one of the problems solved by backtracking, it is one

example of a familiar problem whose solution can be formulated as an n-

tuple. In this unit we study a collection of problems whose solution are best

done using backtracking.

Suppose mi is the size of set Si. Then there are m=m1 m2 ……. mn n-tuples

that are possible candidates for satisfying the function P. The brute force

approach would be to form all these n-tuples, evaluate each one with P and

save those which yield the optimum. The backtrack algorithm has its virtue,

the ability to yield the same answer, for fewer in trials. Its basic idea is to

build up the solution vector on component at a time and to use modified

criterion functions Pi{x1, ….. xi) (some times called bound functions) to test

whether the vector being formed has any chance of success. The major

advantage of this method is that if it is realized that the partial vector

(x1, x2, …. xi) can in no way lead to an optimal solution, then mi+1…..mn

possible test vectors can be ignored entirely. Many of the problems we solve

using backtracking requires that all the solution satisfy a complex set of

constraints. For any problem, these constraints can be divided into two

categories; explicit and implicit.

Definition

Explicit constraints: Are rules that restrict each xi, to take a value only

from a given set,

common examples of explicit constraints

are xi > 0 or Si = {all non negative real no}

xi = 0 or 1 or Si = {0, 1}

di xi < ui or Si = {a: li a ui}

The explicit constraints depend on the particular instance I of the problem

being solved. All tuples that satisfy the explicit constraints define a possible

solution space for I.

Definition

The implicit constraints are tuples that determine which of the tuples in the

solution space of I satisfy the criterion function. Thus implicit constraints

describe the way in which the xi must relate to each other.




1. –––––––––––– are rules that restrict each xi, to take a value only from a

given set.

6.3 8-Queens Problem

A classic combinational problem is to place eight queens on 8x8 chessboard

so that no two “attack” that is, so that no two of them are on the same row,

colours or diagonal. Let us number the rows and columns of the chessboard

1 through 8 (Fig 6.1). The queens can also be numbered 1 through 8. Since

each queen must be on a different row, we can without loss of generality

assume that queen i is to be placed on row i. All solutions to the 8-queens

problem can therefore be represented as 8-tuples (x1…… x8), where xi is the

column on which queen i is placed. The explicit constraints using this

formulation are Si = {1, 2, 3, 4, 5, 6, 7, 8}; 1 i 8. Therefore, the solution

space consists of 88-tuples. The implicit constraints for this problem are that

no two queens can be on the same diagonal. The first of these two

constraints implies that all solutions are permutations of the 8-tuple

(1, 2, 3, 4, 5, 6, 7, 8). This realization reduces the size of the solution space

from 88 tuples to 8! Tuples. We see later how to formulate the second

constraint in terms of the xi. Expressed as an 8-tuple, the solution in Fig. 6.1

is (4, 6, 8, 2, 7, 1, 3, 5).

1 2 3 4 5 6 7 8

1 Q

2 Q

3 Q

4 Q

5 Q

6 Q

7 Q

8 Q

Fig. 6.1: 8-Queens Problem



Algorithm 6.1

Algorithm Place (k, i)

// Returns true if a queen can be placed

// in kth row and ith columns otherwise

// it returns false, x[ ] is a global array

// whose first (k – 1) values have been set

// Abs (r) returns the absolute value of r

{

For (i:=1) to k – l) do

if ((x [j] – l // Two in the same column

or (Abs (x [j] = Abs (j – k)))

// or in the same diagonal

Then return false.

Return true:

}

Algorithm NQueens (k, n)

// Using backtracking this procedure prints all

// possible placements of n queens on an

// n x n chessboard so that they are

// non attacking

{

for I : = 1 to n do

{

if Place (k,j) then

{

x [ k] : = i;

if (k=n) then write (x[1:n])

else Nqueen (k+1, n);

}

}

}

At this point, we can see how effective function Nqueen is over the brute

force approach. For an 8x8 chessboard, there are 8

64 possible ways to

place 8 pieces or approximately 4.4 billion 8 tuples to examine. However, by



allowing only placements of queens on distinct rows and columns, we

require the examination of at most 8! or only 40,3208-tuples.

We can estimate the number of nodes that will be generated by NQueens.

The bounding function is static and no change is made to the function as the

search proceeds. In addition, all nodes on the same level, of the separate

space level, state space tree have the same degree.

As required placement of each queen on the chessboard was chosen

randomly with each choice we kept track of the number of columns a queen

could legitimately be placed on. These numbers are listed beneath each

chessboard. The number following the vector represents the value that

function estimate would produce from these sizes. The coverage of these

five trials is 1625.

So that estimated no. of unbounded nodes is only 2.34% of the total no of

nodes in the 8 queen state space tree.


2. The total no of nodes in the 8-queen state space is –––––––––––.

3. For an 8x8 chessboard there are ––––––––––possible ways to place 8

pieces.

6.4 Sum of Subsets

Given positive numbers wi, 1 i n, and m, this problem calls for finding all

subsets of the wi whose sums are m. For example, if n = 4, (w1, w2, w3, w4) =

(11, 13, 24, 7), and m = 31 then the desired subsets are (11, 13, 7) and

(24, 7) rather than represent the solution vector by the wi which sum to m,

we could represent the solution as described by the vector (1,2,4) and (3, 4).

In general, all solutions are k-tuples (x1, x2…….. xk), 1 k n, and different

solutions may have different sized tuples. The explicit constraints xj {jlj is

an integer and 1 j n}. The implicit requires that no two be the same and

that the sum of the corresponding will be m. Since we wish to avoid

generating multiple instance of the same subject (e.g (1, 2, 4) and (1, 4, 2)

represent the same subject), another implicit constraints that is imposed is

xi<xi+1, I i < k.



In another formulation of the sum of subsets problem, each solution subject

is represented by n-tuple (x1, x2, ….. xn) such that xi {0,1}, 1 i n. Then,

xi = 0, if wi is not chosen then the above instance are (1, 1, 0, 1) and

(0, 0, 1, 1). The formulation express all solutions using fixed sized tuple.

Thus, we conclude that there may be several ways to formulate a problem

so that all solutions are tuples that satisfy some constraints. One can verify

that for both of the above formulations, the solutions space consists of 2n

distinct tuples.

Backtracking algorithms determine problem solutions by systematically

searching the solution space for the given problem instance. This search is

facilitated by using a tree organization for the solution space. For a given

solution space, many tree organizations may be possible. The next two

examples examine some of the ways to organize a solution into a tree.

We have two possible formulations of the solutions space for the sum of

subsets problem. Fig 6.2 and Fig 6.3 show a possible tree organization for

each of these formulations for the case n=4. The tree of Fig 6.2 corresponds

to the variable tuple size formulation. The edges are labeled such that an

edge from a level i node to a level i + 1 node represents a value for xi. At

each node the solution space is partitioned into solution space. The solution

space is defined by all paths from the root node to any node in the tree,

since any such path corresponds to a subset satisfying the explicit

constraints. The possible paths are ( ) (this corresponds to the empty path

from the root to itself), (1), (1,2), (1,2,3), (1,2,3,4), (1,2,4), (1,3,4), (2), (2,3)

and so on. Thus, left most subtree defines all subsets containing wi, the next

subtree defines all subsets containing w2 but not w1, and so on.

The tree of Fig 6.3 corresponds to the fixed tuples size formulation. Edges

from level i nodes to level i + I nodes are labeled with the value of xi, which

is either zero or one. All paths from the root to a leaf node define the

solution space. The left subtree of the root defines all subsets containing wl,

the right subtree defines all subsets not containing wl and so on. Now there

are 24 leaf nodes which represent 16 possible tuples.



Fig. 6.2: A possible space organization for the sum of subsets problem.

Nodes are numbered as in breath first search.

At this point, it is useful to develop some terminology regarding tree

organization of solution spaces. Each node in this tree defines a problem

state. All paths from the root to other nodes define the state space of the

problem. Solution states are those problem states s for which the path from

the root to s defines a tuple in the solution space. In the tree Fig 6.2, all

nodes are solutions states whereas in the tree Fig 6.3, only leaf nodes are

solution states. Answer states are those solution states s for which the path

from the root to s defines a tuple that is a member of the set of solutions (i.e.

it satisfies the implicit constraints) of the problem.



The tree organization of the solution space is referred to as the state space

tree.

Fig. 6.3: Corresponds to the fixed tuples size formulation.

Suppose we are given distinct positive number and we desire to find all

combinations of these numbers whose sums are m. This is called the sum

of subsets problem. The example in above shows how we could formulate

this problem using either fixed or variable sized tuples. We consider a back

tracking solution using the fixed tuple size strategy. In this case, the element

xi of the solution vector is either one or zero depending on whether the

weight wi, is included or not.

The children of any node in Fig 6.3 are easily generated. For a node at level

i, the left child corresponds to xi = 1 and the right to xi = 0.

A simple choice for the bounding functions Bk (x1, ……. xi) = true if n

ki

i

k

i

ii mwxw11

Clearly, x1, ……..,xk cannot lead to an answer node if this condition is not

satisfied. The bounding functions can be strengthened if we assure that the

wi’s are initially in non-decreasing order. In this case, x1…… xk cannot lead

to an answer node if



mwxw k

k

i

ii 1

1

The bounding functions we use are therefore Bk (x1, …………….,xk) = true iff

mWxwandmwxw k

k

i

ii

n

ki

i

k

i

ii 1

111

Since our algorithms will not make use of Bn, we need not be concerned by

the appearance of Wn+1 in this function. Although we have now specified all

that is needed is to directly use either of the backtracking scheme, a simpler

algorithms results if we tailor either of these schemes to the problem at

hand. This simplification results from the realization that if xk = 1, then

n

1ki

i

k

1i

ii mwxw

For simplicity, we redefine the recursive scheme. The resulting algorithms is

SumOfSub (Algorithm 6.2)

Algorithm 6.2

Algorithm SumOfSub (s,k,r)

// Find all subsets of w(k....n) that sum to m

// The values of x[j], 1 j < k, have already

// been determined 1k

1j

]j[X]j[wS

//and n

kj

]j[wr . The w[j]’s are in

// non decreasing order It is assumed that

// w[l] m and n

1i

m]i[w

{

// Generate left child. Note: S+tw[j] m

// since Bk–1 is true.



x [k] : = 1;

If (s + w[k] = m) then write (x [1: k]);

// sub set found there is no recursive call

// here as w[j] > 0, 1 j n

else if (s + w[k] + w [k + 1] m)

then SumOfSub (s + w[k], k + 1, r – w[k]);

// Generate right child and evaluate Bk

if(s + r – w [k] m) and (s + w [k + 1] m ) then

{

x[k]:=0

SumOfSub (s, k + 1, r — w [k]):

}

}

Algorithm SumOfSub avoids computing n

1ki

i

k

1i

ii wandkw each time by

keeping those values in variables s and r respectively. The algorithm

assumes n

1i

ii .mwandmw The initial call is Sum Of Sub n

1i

iw,1,0 .

It is interesting to note that the algorithm does not explicitly use the test k> n

to terminate the recursion. This test is not needed as an entry to the

algorithm s m and s+r m. Hence, r 0 and so k can be no greater than n.

Also note that in the else if statement, since s + wk < m and s + r m, it

follows that r wk and hence k + 1 n, observe also that if s + wk = m, then

xk+1,……………….xn must be zero. These zeros are omitted from the output.

In next statement we do not test for n

1ki

i

k

1i

ii mwxw as we already

know s + r m and xk + 1.


4. A simple choice for the bounding functions Bk (x1, ……. xi) is true

if ––––––––––

5. Given positive numbers wi, 1 i n, and m, the problem calls for finding

all –––––––––––of the wi whose sums are m




Given n positive weights wi, n positive profits Pi, and a positive number m

that is the knapsack capacity, this problem calls for choosing a subset of the

weights such that

ni1

ii

ni1

ii xPandmxw is minimized.

The xi’s constitute a zero-one-valued vector.

The solution space for this problem consists of the 2n distinct ways to assign

zero or one values to the xi’s. Thus, the solution source is the same as that

for the sum of subset problem. Two possible tree organizations are possible,

One corresponds to the fixed tuple size formulation and the other to the

variable tuple size formulation. Backtracking algorithms for the knapsack

problem can be arrived at using either of these two state space trees.

Regardless of which is used, bounding functions are needed to help kill

some live nodes without expanding them. A good bounding function for this

problem is obtained by using an upper bound on the value of the best

feasible solution.

Obtainable by expanding the given live node and any of its descendants, if

this upper bound is not higher than the value of the best solution determined

so far, then that live node can be killed.

We continued the discussion using the fixed tuple size formulation, If at

node z, the values of xi, 1 i k have already been determined, then an

upper bound for z can be obtained by relaxing the requirement xi = 0 on 1 to

0 xi 1 for k + 1 i n and using the greedy algorithm to solve the

relaxed problem. Function Bound (cp, cw, k) determines an upper bound on

the best solution obtained by expanding any zone z at level of the state

space tree. The object weights and profits are w[i] and p[i]. It is assumed

that ni1;)1i(w

)1i(p

]i[w

)i(p



Algorithm 6.3

Algorithm Bound (cp, cw, k)

// cp is the current profit, cw is the current

// weight total k is the index of the last removed

// item : and m is the knapsack size

{

b:=cp; c:= cw;

for i = k +1 to n do

{

c: = c + w [i]

if (c < m) then b:= b + p [i]

else return b + (cI – (c-m)) / w[i] * p [i];

} return b:

{

From bound it follows that the bound for a feasible left child of a node z is

the same as that for z. Hence, the bounding function need not be used

whenever the backtracking algorithm makes a move to the left child of a

node. The resulting algorithm is Bknap. It was obtained from the recursive

backtracking scheme, Initially set fp: = -1: This algorithm is invoked as

Bknap (1, 0, 0):

When fp —1, x[i], 1 i n is such that n

1i

p .fixip The path y [i], 1 i

k, is the path to the current node. The current weight

1k

1i

p

1k

1i

w iyipcandiyiwc

Algorithm 6.4

Algorithm Bknap (k, cp, cw)

// m is the size of the knapsack; is the

// no of weights and profits w[ ] and p [ ]

// are the weights and profits

// p [i] /w [i] p[i +1] fw wf]1i[piw

ipis the final

// weight of knapsack: fp is the final

// maximum profit x[k] = 0 if w[k] is not



// the knapsack else x [k] = 1

{

// Generate left child

If (cw + w [k] m ) then

{

y [k] = 1:

if (k<n) then Bknap (k+1, cp+p [k], cw + w [k]

if (cp + p[k] > fp) and (k= n) then

{

fp: = cp + p[k]; fw: = cw + w[k]

for j: = 1 to k do x [j]: = y[j]:

}

}

// Generate right child

if (Bound (cp, cw, k) fp) then

{

y[k]: = 0; if (k < n) then Bknop (k + 1, cp, cw):

If ((cp > fp) and (k = n) then

{

fp : = cp : fw = cw

for j : = 1 to k do

x[j] : = y[j]:

}

}

}


6. If the object weights and profits are w[i] and p[i], It is assumed

that ––––––

6.6 Summary

In this unit, we studied that any problem can be divided into two

categories explicit and implicit.

We also studied in this unit a classic combinational problem called as

8-queens problem.

The concept of sum of subsets was discussed with suitable examples.



The Knapsack problem and its algorithm was also discussed with

suitable explanation


1. Write briefly the concept of 8-Queens Problem

2. Write the Algorithm of sum of Subsets

3. Briefly explain the Knapsack Problem

6.8 Answers


1. Explicit constraints

2. 281,69)8(17

0 0j

j

i

i

3. 8

64

4. n

ki

i

k

i

ii mwxw11

5. subsets

6. ni1;)1i(w

)1i(p

]i[w

)i(p

Terminal Questions

1. A classic combinational problem is to place eight queens on 8x8

chessboard so that no two “attack” that is, not two of them are on the

same row, colours or diagonal. Let us number the rows and columns of

the chessboard 1 through 8 (Fig 6.1). The queens can also be

numbered 1 through 8. Since each queen must be on a different row, we

can without loss of generality assume that queen i is to be placed on row

i. All solutions to the 8-queens problem can therefore be represented as

8-tuples (x1…… x8), where xi is the column on which queen i is placed.

The explicit constraints using this formulation are Si = {1, 2, 3, 4, 5, 6, 7,

8}; 1 i 8. Therefore the solution space consists of 88-tuples. The implicit

constraints for this problem are that no two queens can be on the same

diagonal. The first of these two constraints implies that all solutions are



permutations of the 8-tuple (1, 2, 3, 4, 5, 6, 7, 8). This realization

reduces the size of the solution space from 88 tuples to 8! Tuples. (Refer

Section 6.3)





Unit 7 Branch and Bound Structure

7.1 Introduction

Objectives

7.2 Branch and Bound Strategy

7.3 O/I Knapsack Problem


7.5 Summary


7.7 Answers

7.1 Introduction

In the previous units we discussed about the concept of backtracking and

now we will study a new concept known as branch and bound. The term

branch-and-bound refers to all state space search methods in which all

children of the E-node are generated before any other live node can

become the E-node.

As in the case of backtracking, bounding functions are used to help avoid

the generation of subtrees that do not contain an answer node.

Objectives


solve the O/I Knapsack Problem

find the solution using Traveling Salesman Problem.

7.2 Branch and Bound Strategy

The term branch-and-bound refers to all state space search methods in

which all children of the E-node are generated before any other live node

can become the E-node.

In branch-and-bound terminology, a BFS-like state space search will be

called FIFO (First In First Out) search as the list of live nodes is first-in-first-

out list (or queue). A D-search-like state space search will be called LIFO

(Last In First Out) search as the list of live nodes is a last-in-first-out list

(or stack). As in the case of backtracking, bounding functions are used to

help avoid the generation of subtrees that do not contain an answer node.




1. The term branch-and-bound refers to all state space search methods in

which all children of the E-node are generated before any other live

node can become the––––––––––––.

7.3 O/l Knapsack Problem

To use the branch-and-bound technique to solve any problem, it is first

necessary to conceive of a state space tree for the problem.

The objective function ii xp is replaced by the function .xp ii Clearly,

ii xp is maximized iff – ii xp is minimized. This modified knapsack

problem is stated as given below

n

1i

ii xpMinimize

n

1i

ii mxwtoSubject

xi = 0 or 1, 1 i n ------- (7.1)

We continue the discussion assuming a fixed tuple size formulation for the

solution space. The discussion is easily extended to the variable tuple size

formulation. Every leaf node in the state space tree represents an

assignment for which ni1

ii mxw is an answer (or solution) node. All

other leaf nodes are infeasible. For a minimum-cost answer node to

correspond to any optional solution, we need to define ni1

ii xpxc for

every answer node x. The cost c(x) = for infeasible leaf nodes. For non-

leaf nodes, c(x) is recursively defined to be min{c(Ichild(x)), c(rchild(x))}



Algorithm 7.1

1. Algorithm U Bound(cp, cw, k, m)

2. // cp is the current profit total, cw is the current

3. // weight total; k is the index of the last removed

4. // item; and m is the knapsack size

5. // w[i] and p[i] are respectively the weight and profit

6. // of the ith object

7. {

8. b:=cp; c:=cw;

9. for I:=k+1 to n do

10. {

11. if (c+w[i] m) then

12. {

13. c:c+w[i]; b:=b-p[i];

14. }

15. }

16. return b;

17. }

We now need two functions c (x) and u(x) such that c (x) c(x) u(x) for

every node x. The cost c (.) and u(.) satisfying this requirement may be

obtained as follows. Let x be a node at level j, 1 j n+1. At node x,

assignments have already been made to xi, 1 i < j. The cost of these

assignments is ji1

ii xp . So, ji1

ii xpxc and we may use

ji1

ii .xpxu If ji1

ii xpq then an improved upper bound function u(x) is

u(x) = Ubound ,m,1j,xw,qji1

ii where Ubond is defined in Algorithm


2. To use the branch-and-bound technique to solve any problem, it is first

necessary to conceive of a –––– ––––– –––––for the problem.




An O(n22n) dynamic programming algorithm for the travelling sales person

was arrived at in Unit 5. We now investigate branch-and-bound algorithms

for this problem. Although the worst-case complexity of these algorithms will

not be any better than O(n22n), the use of the good bounding functions will

enable these branch and bound algorithms to solve some problem instances

in much time less than that required by the dynamic programming algorithm.

Let G=(V,E) be a directed graph defining an instance of the travelling sales

person problem. Let cij equal the cost of edge <i, j>,cij = if < i, j > E, and

let |V|=n. Without loss of generality, we can assume that every tour starts

and ends at vertex 1. So, the solution space S is given by S = {1, , 1| is a

permutation of (2,3 ,n)}. Then, | s |= (n — 1)!. The size of S can be reduced

by restricting S so that (1, i1,i2,……in–1,1) S iff < ij, ij+1 > E, 0 j n — 1,

and i0 = in = 1.

To search the travelling salesperson state space tree, we need to define a

cost function c (.) and two other functions c (.) and u(.) such that c (r) c(r)

u(r) for all nodes r. The cost c(.) is such that the solution node with least

c(.) corresponds to a shortest tour in G. One choice for c(.) is

c(A) = length of tour defined by the path from the root to A, if A is a leaf

and

c(A) = cost of minimum-cost leaf in the subtrees A if A is not a leaf

l(i) = length of the path defined at node i

Fig. 7.1: State space tree for the travelling sales person problem with n=4 and

i0 = i4 = 1



120011

012315

2030

021112

101710

167416

318619

4253

241615

11103020

a) Lost matrix b) Reduced cost matrix L=25

Fig. 7.2: An example of cost matrix and reduced cost matrix

A simple c (.) such that c (A) c(A) for all A is obtained by defining c (A) to

be the length of the path defined at node A. A better c (.) can be obtained

by using the reduced cost matrix corresponding to G. A row (column) is said

to be reduced if it contains atleast one zero and all remaining entries are

non-negative. A matrix is reduced iff every row and column is reduced. As

an example of how to reduce the cost matrix of a given graph G, consider

the matrix of fig. 7.2(a). This corresponds to a graph with five vertices. Since

every tour on this graph includes exactly one edge <i, j> with i=k, 1 k 5,

and exactly one edge <i, j> with j=k, 1 k 5, subtracting a constant t from

every entry in one column or one row of the cost matrix reduces the length

of every tour by exactly t. A minimum-cost tour remains a minimum-cost tour

following this subtraction operation. If t is chosen to be the minimum entry in

row i (column j), then subtracting it from all entries in row i (column j)

introduces a zero into row i (column j) Repeating this as often as needed,

the cost matrix can be reduced. The total amount subtracted from the

columns and rows is a lower bound on the length of a minimum-cost tour

and can be used as the c value for the root of the state space tree.

Subtracting 10, 2, 2, 3, 4, 1 and 3 from rows 1, 2, 3, 4 and 5 and Column 1

and 3 respectively of the matrix of fig 7.2(a), yields the reduced matrix of fig.

7.2(b). The total amount subtracted is 25. Hence, all tours in the original

graph have a length of at least 25.

We can associate a reduced cost matrix with every node in the travelling

salesperson state space tree. Let A be the reduced cost matrix for node R.

Let S be a child of R such that the tree edge (R, S) Corresponds to including

edge <i,j> in the tour. If S is not a leaf, then the reduced cost matrix for S

may be obtained as follows:



1) Change all entries in row i and column j and A to . This prevents the

use of any more edge leaving vertex i or entering vertex j.

2) Set A(j,1) to . This prevents the use of edges <j,1>.

3) Reduce all rows and columns in the resulting matrix except for rows and

columns containing only .

Let the resulting matrix be B.

Step (1) and (2) are valid as no tour in the subtree S can contain edges of

the type <i,k> or <k,j> or <j,1> (except for edge <i,j>). If r is the total amount

subtracted in step (3) then,

c (S) = c (R) c + A(i, j)+r. For leaf nodes, (.)= c( c ) is easily computed as

each leaf defines a unique tour. For the upper bound function u, we can use

u( R)= for all nodes R.


3. We can associate a –––– –––– ––––– with every node in the travelling

salesperson state space tree

7.5 Summary

The term branch-and-bound refers to all state space search method in

which all children of the E-node are generated before any other live

node can become the E-node.

The Knapsack problem using branch-and-bound technique states as

n

1i

ii xpMinimize

n

1i

ii mxwtoSubject

xi = 0 or 1, 1 i n.

In travelling salesperson problem, one choice for c(.) is

leafanotisAifA,subtreestheinleafcostminimumofcost

leafaisA,torootthefrompaththebydefinedtourlengthAc




1. Briefly explain the O/I Knapsack Problem

2. Explain the travelling Salesman Problem

3. Write the Algorithm for Knapsack Problem

4. Write the general method for Branch and Bound

5. What do you mean by Bounding?

7.7 Answers


1. E-node

2. state space tree

3. reduced cost matrix

Terminal Questions



The objective function ii xp is replaced by the function .xp ii Clearly

ii xp is maximized iff – ii xp is minimized. This modified knapsack

problem is stated as given below

n

1i

ii xpMinimize

n

1i

ii mxwtoSubject

xi = 0 or 1, 1 i n. (Refer Section 7.3)

2. Let G=(V,E) be a directed graph defining an instance of the traveling

sales person problem. Let cij equal the cost of edge <i, j>,cij = if < i, j >

E, and let |V|=n. Without loss of generality, we can assume that every

tour starts and ends at vertex 1. So, the solution space S is given by S =

{1, , 1| is a permutation of (2,3 ,n)}. Then | s |= (n — 1)!. The size of S

can be reduced by restricting S so that (1, i1,i2,……in–1,1) S iff < ij, ij+1 >

E, 0 j n — 1, and i0 = in = 1.



To search the traveling salesperson state space tree, we need to define

a cost function c (.) and two other functions c (.) and u(.) such that c (r)

c(r) u(r) for all nodes r. The cost c(.) is such that the solution node

with least c(.) corresponds to a shortest tour in G. One choice for c(.) is

c(A) = length of tour defined by the path from the root to A, if A is a leaf

and c(A) = cost of minimum-cost leaf in the subtrees A and if A is not a

leaf l(i) = length of the path defined at node i. (Refer Section 7.4).



The objective function ii xp is replaced by the function .xp ii

Clearly, ii xp is maximized iff – ii xp is minimized. This modified

knapsack problem is stated as given below

n

1i

ii xpMinimize

n

1i

ii mxwtoSubject

xi = 0 or 1, 1 i n ------- (7.1)

We continue the discussion assuming a fixed tuple size formulation for

the solution space. The discussion is easily extended to the variable

tuple size formulation. Every leaf node in the state space tree represents

an assignment for which ni1

ii mxw is an answer (or solution) node.

All other leaf nodes are infeasible. For a minimum-cost answer node to

correspond to any optional solution, we need to define ni1

ii xpxc

for every answer node x. The cost c(x) = for infeasible leaf nodes. For

non-leaf nodes, c(x) is recursively defined to be min{c(Ichild(x)),

c(rchild(x))}



Algorithm 7.1

18. Algorithm U Bound(cp, cw, k, m)

19. // cp is the current profit total, cw is the current

20. // weight total; k is the index of the last removed

21. // item; and m is the knapsack size

22. // w[i] and p[i] are respectively the weight and profit

23. // of the ith object

24. {

25. b:=cp; c:=cw;

26. for I:=k+1 to n do

27. {

28. if (c+w[i] m) then

29. {

30. c:c+w[i]; b:=b-p[i];

31. }

32. }

33. return b;

34. }

Now we need two functions c (x) and u(x) such that c (x) c(x) u(x)

for every node x. The cost c (.) and u(.) satisfying this requirement may

be obtained as follows. Let x be a node at level j, 1 j n+1. At node x,

assignments have already been made to xi, 1 i < j. The cost of these

assignments is ji1

ii xp . So, ji1

ii xpxc and we may use

ji1

ii .xpxu If ji1

ii xpq then, an improved upper bound function

u(x) is u(x) = Ubound ,m,1j,xw,qji1

ii where Ubond is defined in

Algorithm.

4. The general method is

1. In branch and bound method a state space tree is built and all the

children of E nodes ( a live node whose children are currently being



generated) are generated before any other node can become a live

node.

2. For exploring new nodes either a BFS or D-Search technique can be

used.

3. In Branch and bound technique, BFS-like state space search will be

called FIFO (First in First Out) search. This is because the list of live

node is First In First Out list (queue). On the other hand, the D-

search like state space search will be called LIFO search because

the list of live node is Last in First out list (stack).

4. In this method, a space tree of possible solutions is generated. Then

partitioning (called as branching) is done at each node of the tree.

We compute lower bound and upper bound at each node. This

computation leads to selection of answer node.

5. Bounding functions are used to avoid the generation of subtrees that

do not contain an answer node.

5. Bounding is having the following characteristics

1. As we know that the bounding functions are used to avoid the

generation of sub trees that do not contain the answer nodes. In

bounding lower bounds and upper bounds are generated at each

node.

2. A cost function c (x) is such that c (x) c (x) is used to provide the

lower bounds on solution obtained from node x.

3. Let upper is an upper bound on cost of minimum-cost solution. In

that case, all the live nodes with c (x) upper can be killed.

4. At the start, the upper is usually set to . After generating the

children of current E-node, upper can be updated by minimum cost

answer node. Each time a new answer node can be obtained.



Unit 8 Additional Features Structure

8.1 Introduction

Objectives

8.2 Lower Boundary Theory

8.3 NP Hard and NP Complete Problems

8.4 Approximate Algorithms for NP Hard Problems

Absolute approximation

- Approximation

Polynomial Time Approximation Schemes

8.5 Summary


8.7 Answers

8.1 Introduction

We have studied in the previous units the different techniques of algorithms.

Now, the purpose of this unit is to expose you to some techniques that have

been used to establish that a given algorithm is the most efficient possible.

The way this is done is by discovering a function g(n) that is a lower bound

on the time that any algorithm must take to solve the given problem. If we

have an algorithm whose computing time is the same order as g(n), then we

know that asymptotically we can do no better.

Objectives


apply the Lower Boundary theory

use Comparison tree for searching and sorting

apply Approximate Algorithms for NP hard problems

use Approximation in Bin Packing

analyse Polynomial time approximation schemes.

8.2 Lower Boundary Theory

If f(n) is the time for some algorithm, then we write f(n)= (g(n)) to mean that

g(n) is a lower bound for f(n). Formally, this equation can be written if there

exist positive constants C and no such that |f(n)| C|g (n)| for all n>no. In



addition to developing lower bounds to within a constant factor, we are also

concerned with determining more exact bounds whenever this is possible.

For many problems it is possible to easily observe that a lower bound

identical to n exists where n is the number of input to the problem. Consider

all algorithms that find the maximum of an unordered set of n integers.

Clearly, every integer must be examined at least once, so (n) is a lower

bound for any algorithm that solves this problem.

Comparison Trees for Searching and Sorting

Below, we will study some of the comparison trees used for searching and

sorting

Ordered Searching:

In obtaining the lower bound for the ordered searching problem, we consider

only those comparison based algorithms in which every comparison

between two elements of S is of the type “compare x and A [i]”. The

progress of any searching algorithm that satisfies this restriction can be

described by a path in a binary tree. Each internal node in this tree

represents a comparison: between x and A[i], There are three possible

outcome of this comparison:

x<A[i],x=A[i], or x>A[i]. We can assume that if x=A[i], the algorithm

terminates. The left branch is taken if x<A[i], and the right branch is taken

x>A[i]. If the algorithm terminates, following a left or right branch, then no i

has been such that x=A[i] and the algorithm must declare the search

unsuccessful.

Theorem: Let a[1:n],n 1, contain n distinct elements ordered so that

a[1]<……<a[n]. Let FIND (n) be the minimum number of comparisons

needed, in the worst case, by any comparison-based algorithm to recognize

whether x A[1:n]. Then FIND (n) [log (n+1)].

Proof: Consider all possible comparison trees that model algorithm to solve

the searching problem. The value of FIND (n) is bound below by the

distance of the longest path from the root to a leaf in such a tree. There

must be n internal nodes in all these trees corresponding to the n possible

successful occurrences of x in A. If all internal nodes of a binary tree are at

levels less than or equal to k, then there are at most 2k–1 internal modes.

Thus n<2k–1 and FIND (n)=k [log (n+1)].



Sorting:

Now, let’s consider the sorting problem. We can describe any sorting

algorithm that satisfies the restrictions of the comparison tree model by a

binary tree. Consider the case in which the n numbers A [1:n] to be sorted

are distinct. Now, any comparison between A[i] and A[j] must result in one

of two possibilities:

Either A [i]<A [j] or A [i] > A[j]. So, the comparison tree is a binary tree in

which each internal node is labeled by the pair i : j which represents the

comparison of A[i] with A[j]. If A[i] is less than A[j], then the algorithm

proceeds down the left branch of the tree; otherwise, it proceeds down the

right branch. The external nodes represent termination of the algorithm

associated with every path from the root to an external node is a unique

permutation. To see that this permutation is unique, note that the algorithms

we allow are only permitted to move data and make comparisons. The data

movement on any path from the root to an external node is the same no

matter what the initial input values are. As there are n! different possible

permutations of n items, and any one of these might legitimately be the only

correct answer for the sorting problem on a given instance, the comparison

tree must have at least n! external nodes.

Fig 8.1 shows a comparison tree for sorting three items. The first

comparison is A[1]:A[2]. If A[1] is less than A[2], then the next comparison

is A[2] with A[3]. If A[2] is less than A[3], then the left branch leads to an

external nodes containing left branch leads to an external node containing

1,2,3. This implies that the original set was already sorted for

A[1]<A[2]<A[3]. The other five external nodes correspond to the other

possible orderings that could yield a sorted set.



Fig. 8.1: A Comparison Tree for sorting three items

We consider the worst case for all comparison based sorting algorithms. Let

T (n) be the minimum number of comparisons that are sufficient to sort n

items in the worst case. Using our knowledge of binary trees once again, if

all internal nodes are at levels less than k, then there are at most 2k external

nodes. Therefore, if we let k = T (n)

n! 2T(n)

Since T(n) is an integer, we get the lower bound

T (n) [log n!]


1. In obtaining the lower bound for the ordered searching problem, we

consider only those comparison based algorithms in which every

comparison between two elements of S is of the type ––––––––––– .

8.3 NP Hard and NP Complete Problems

Basic Concepts

In this unit, we are concerned with the distinction between problems that can

be solved by a polynomial time algorithm and problems for which no

polynomial time algorithm is known. It is an unexplained phenomenon that

for many of the problems we know and study, the best algorithms for their

solutions have computing times that cluster into two groups. The first group

consists of problems whose solution are bounded by polynomials of small

degree.

1:2

2:3 2:3

1:3 1:3 1, 2, 3

3, 2, 1

1, 3, 2 3, 1, 2 2, 1, 3 2, 3, 1



The second group is made up to problems whose best-known algorithms

are non-polynomial. Examples we have seen include the traveling

salesperson and the knapsack problems for which the best algorithms given

in this text have complexities O (n22n) and O (2n/2) respectively.

In the quest to develop efficient algorithms, no one has been able to develop

a polynomial time algorithm for any problem in the second group. This is

very important because algorithms whose computing times are greater then,

polynomial very quickly require such vast amounts of time to execute that

even moderate-size problems cannot be solved.

The theory of NP-completeness, which we present here, does not provide a

method of obtaining polynomial time algorithms for problems in the second

group. Nor does it say that algorithms of this complexity do not exist.

Instead, what we do is show that many of the problems for which there are

no known polynomial time algorithms are computationally related. In fact, we

establish two classes of problems. These are given names, NP-hard and

NP-Complete.

A problem that is NP-Complete has the property that it can be solved in

polynomial time if and only if all other NP-Complete problems can also be

solved in polynomial time.

If an NP-hard problem can be solved in polynomial time, then all NP-

Complete problems can be solved in polynomial time. All NP complete

problems are NP-hard, but some NP-hard problems are not known to be

NP-complete.

Fig. 8.2: Venn diagram

NP Complete

NP

P

NP hard



Although one can define many distinct problem classes having the

properties stated above for the NP-hard and NP-complete classes, the

classes we study are related to non-deterministic computations. The

relationship of these classes to non deterministic computations together with

the apparent power of non- determinism lead to the intuitive conclusion that

no NP- complete or NP- hard problem is polynomially solvable.

Cook’s Theorem

Cook’s theorem states that satisfiability is in P if and only if P=NP. We now

prove this important theorem. We know that satisfiability in NP. Hence, if P

= NP, then satisfiability is in P. It remains to be shown that if satisfiability is

in P, then P = NP. To do this, we show how to obtain from any polynomial

time non- deterministic decision algorithm A and input I a formula Q (A, I)

such that Q is satisfiable if A has a successful termination with input I. If the

length of I – is n and the time complexity of A is p (n) for some polynomial

p(), then the length of Q is O (p3(n)log n) = O(p4(n)). The time needed to

construct Q is also O (p3(n) log n). A deterministic algorithm Z to determine

the out come of A on any input I can be easily obtained. Algorithm Z simply

computes Q and then uses a deterministic algorithm for the satisfiability

problem to determine whether Q is satisfiable. If O (q (m)) is the time

needed to determine whether a formula of length m is satisfiable, then the

complexity of Z is O (p3(n) log n+q (p3(n)log n)). If satisfiability is in P, then q

(m) is a polynomial function of m and the complexity of Z becomes O(r (n))

for some polynomial r(). Hence, if satisfiability is in P, then for every non-

deterministic algorithm A in NP we can obtain a deterministic Z in P. So, the

above construction shows that if satisfiability is in P, then P=NP.

Some Theorems

Definition: A K- clique in a graph G is a complete sub graph of G with K-

vertices.

CNF product of sums

Theorem 1: CNF satisfiability is polynomially transformable to the clique

problem. Therefore, the clique problem is NP complete.

Proof: Let F=F1 F2…….Fq be an expression in CNF, where the Fi’ s are the

factors, each Fi is of the form



(xi1+xi2+…..xik) where xij is a literal. Construct an undirected graph G=(V,E)

whose vertices are pairs of integers [i,j] for 1 i q and 1 j k.

First component represents a factor and the 2nd component represents a

literal within the factor since there are 2n combinations. It is a polynomial

And it is NP-complete.

Definition: Let G=(V,E) be an undirected graph. A vertex cover of G is a

subset SÇV such that each edge of G is incident upon some vertex in S.

Theorem 2: The clique problem is polynomially transformable to the vertex

cover problem. Therefore, the vertex problem is NP-Complete.

Proof: Given an undirected graph G=(V,E), consider the complement graph

G=(V,E)

Where E {(v, w)|v,w V,v w and (v,w) E}

A set S V is a clique in G iff V-S is a vertex cover of G. If S is a clique in

G, no edge in G connects 2 vertices in S. Thus, every edge in G is incident

upon atleast one vertex in V-S, implying V-S is a vertex cover of G.

Similarly, if V-S is a vertex cover of G, then every edge of G is incident upon

atleast one vertex of V-S. Thus, no edge of G connects 2 vertices in S.

Therefore, every pair of vertices of S is connected in G, and S is a clique in

G.

To find whether there exists a clique of size k, we construct G and

determine whether it has a vertex cover of size ||v||-k. Given a standard

representation of G=(V,E) and K, we find a representation of G and ||v||-k in

time which is a polynomial in the length of the representation for G and K.


2. first group consists of problems whose solution times are bounded by

polynomials of ––––––––––.

3. A K- clique in a graph G is a complete sub graph of G with ––––––––.

8.4 Aproximate Algorithms for NP Hard Problems

Many NP-hard optimization problems have great practical importance and it

is desirable to solve large instances of these problems in a reasonable



amount of time. The best-known algorithms for NP-hard problems have a

worst-case complexity that is exponential in the number of inputs.

A feasible solution with value close to the value of an optimal solution is

called an approximate solution. An approximate algorithm for P is an

algorithm that generates approximate solution for P

Let A be an algorithm that generates a feasible solution to every instance I

of a problem P.

Let F*(I) be the value of an optimal solution to I. Let F(I) be the value of the

feasible solution generated by A.

Definition: A is an absolute approximation algorithm for problem P if and

only if for every instance I of P, | F*(I) – F(I) | K for some constant k.

Definition: An -approximate algorithm is an approximate algorithm for

which f(n) for some constant .

Definition: A ( ) is an approximation scheme if and only if for every given

>O and problem instance I, A( ) generates a feasible solution such that

|F*(I) – F(I)|/F*(I) . Again, we assume F* (I) > O

Definition: An approximation scheme is a polynomial time approximation

scheme if and only if for every fixed >O it has a computing time that is

polynomial in the problem size.

8.4.1 Absolute Approximation

Below, we study some of the absolute approximations along with their

algorithms

Planar Graph Coloring

There are very few NP-hard optimization problems for which polynomial

time absolute approximation algorithms are known. One problem is that of

determining the minimum number of colors needed to color a planar graph

G= (V,E). It is known that every planar graph is four colorable. It is zero

colorable iff V=ø It is one colorable iff E=ø It is two colorable if it is bipartite.

Determining whether a planar graph is three colorable is NP – hard.

However, all planar graphs are four colorable. An absolute approximation

algorithm with |F*(I)-F(I)| 1 is easy to obtain. Algorithm 7.1 is such an

algorithm. It finds an exact answer when the graph can be colored using at

^

^

^

^



most two colors. Since we can determine whether a graph is bipartite in

time O (|V|+|E|), the complexity of the algorithm is to 0(|V|+|E|).

Below, we discuss an algorithm for approximate coloring

Algorithm 8.1

1. Algorithm A color (V,E,)

2. // Determine an approximation to the minimum number of colors.

3. {

4. if V=Ø then return 0;

5. else if E=Ø then return 1;

6. else if G is bipartite then return 2;

7. else return 4;

8. }

Maximum Programs Stored Problem

Assume that we have n programs and two storage devices (say disks or

tapes). We assume the devices are disks and our discussion applied to any

kind of storage device. Let li be the amount of storage needed to store the

ith program. Let L be the storage capacity of each disk. Determine the

maximum number of these n program over the disks) is NP-hard.

Theorem: Partition varies as the maximum programs stored.

Proof: Let {a1, a2,……an}, define an instance of the partition problem. We

can assume ai = 2T. Define an instance of the maximum programs stored

problem as follows: L=T and li=ai, 1 i . Clearly {a1,…..an} has a partition

if and only if n programs can be stored on the two disks.

By considering programs in order of non-decreasing storage requirement li,

we can obtain a polynomial time absolute approximation algorithm.

Function Pstore (Algorithm 8.2) assumes l1 l 2 …. l n and assigns

programs to disk 1 so long as enough space remains on this tape. Then it

begins assigning programs to disk 2. In addition to the time needed to

initially sort the programs into non-decreasing order of ll, O (n) time is

needed to obtain the storage assignment.



Below we discuss approximation Algorithm to store programs

Algorithm 8.2

1) Algorithm Pstore(l, n, L)

2) //Assume that l [i] l [i+1],1 i n.

3) {

4) i:=1;

5) for j:=1 to 2 do

6) {

7) sum:=0;//Amount of disk j already assigned

8) while (sum + l [i]) L do

9) {

10) write(“Store program “,i,”on disk”,j);

11) sum :=sum+l[i];i :=i+1 ;

12) if i>n then return;

13) }

14) }

15) }

Example: Let L=10, n=4, and (l 1, l 2, l 3, l 4)=(2,3,4,5). Function Pstore will

store programs 1 and 2 on disk 1 and only program 3 on disk 2. An optimal

storage scheme stores all four programs. One way to do this is to store

programs 1 and 4 on disk 1 and the other two on disk 2.

8.4.2 Approximation

Below, we study some Approximation

Scheduling Independent Tasks

Obtaining minimum finish time schedules on m, m 2, identical processors

is NP-hard. There exists a very simple scheduling rule that generates

schedules with a finish time very close to that of an optimal schedule. An

instance I of the scheduling problem is defined by a set of n task times ti, 1

i n, and m, the number of processors. The scheduling rule we are about to

describe is known as the LPT. An LPT schedule is a schedule that results

from this rule.

Definition: An LPT schedule is one that is the result of an algorithm that,

whenever a processor becomes free, assigns to that processor a task



whose time is the largest of those tasks not yet assigned. Ties are broken

in an arbitrary manner.

Example: Let m=3, n=6, and (t1, t2,t3,t4,t5,t6)=(8,7,6,5,4,3).In an LPT

schedule tasks 1, 2 and 3 are assigned to processor 1, 2 and 3 respectively.

Tasks 4, 5 and 6 are respectively assigned to process 3, 2 and 1. The finish

time is 11. Since ti/3 =11,the schedule is also optimal.

6 7 8 11

P1

P2

P3

1 6

2 5

3 4

Fig. 8.3: LPT schedule for the example

Bin Packing

In this problem, we are given n object that have to be placed in bin s of

equal capacity L. Object i requires li units of bin capacity. The objective is to

determine the minimum number of bins needed to accommodate all n

objects. No object may be placed partly in one bin and partly in another.

Example: Let L=10,n=6,and (l 1, l 2, l 3, l 4, l 5, l 6)=(5, 6, 3, 7, 5, 4). Fig 8.3

shows a packing of the six objects using three bins. Numbers in bins are

object indices. Obviously, at least three bins are needed.

The bin packing problem can be regarded as a variation of the scheduling

problem. The bins represent processors and L is the time by which all tasks

must be completed. The variable li, is the processing requirement of task i.

The problem is to determine the minimum number of processor needed to

accomplish this. An alternative interpretation is to regard the bins as tapes.

The variable L is the length of a tape, and li, the tape length needed to store

program i. The problem is to determine the minimum number of tapes

needed to store all n programs. Clearly, many interpretations exist for this

problem.

Theorem: The bin packing problem is NP-hard.

Proof: To see this, consider the partition problem Let {a1, a2…..an} be an

instance of the partition problem. Define an instance of the bin packing



problem as li = ai, 1 i n, and L= ai/2. Clearly, the minimum number of

bins needed is two if and only if there is a partition for {a1, a2……..an}.

(a) (b) (c)

Fig. 8.4: Optimal Packing for the example

8.4.3 Polynomial Time Approximation Schemes

The LPT rule leads to a (1/3-1/(3m))-approximate algorithm for the problem

obtaining an m processor schedule for n task . A polynomial time

approximation scheme is also known for this problem. This scheme relied

on the following scheduling rule Let k be some specified and fixed integer.

Obtain an optimal schedule for the K longest tasks. Schedule the remaining

n-k tasks using the LPT rule

Example: Let m = 2, n = 6, (t1, t2, t3, t4, t5, t6) = (8, 6, 5, 4, 4, 1), k = 4. The

four longest tasks have task times 8, 6, 5 and 4 respectively. An optional

schedule for these has finish time 12 (Fig. 8.4(a)). When the remaining two

task are scheduled using the LPT rule, the schedule of Fig. (8.4 (b)) results.

This has finish time 15. Fig. (8.4 (c)) shows as optimal schedule. This has

finish time 14.

Theorem: Let I be an m-processor instance of the scheduling problem. Let

F*(I) be the finish time of an optimal schedule for I and let F (I) be the length

of the schedule generated by the above scheduling rule. Then,

Proof: Let r be the finish time of an optimal schedule for the k longest tasks.

If F (I)=r, the F*(I)= F (I) and the theorem is proved. So, assume F (I)>r. Let

ti, 1 i n, be the task times of the n tasks of I.

Without loss of generality, we can assume ti ti+1, 1 i<n, and n>k. Also, we

can assume n>m. Let j, j>k, be such that task j has finish time F(I). Then, no

5 1

6 2

6 2

]m/k[1

m/11

)I(*F

|)I(F)I(*F|



processor is idle in the interval [0, F (I)–tj]. Since tk+1 tj, it follows that no

processor is idle in the interval [0, F )I)–tk+1].

Hence

1k1k1i i t)t)I(F(mt

and so, F*(I) 1kn

1 i tm

1m)I(Ft

m

1

or |F*(I)– F (I)| 1ktm

1m

Since ti tk+1, 1 i k+1 and at least one processor must execute at least

1+[k/m] of these k+1 tasks, it follows that

1kmk t1)I(*F

Combining these two inequalities, we obtain

mk

m1

mk 1

1

1

m/)1m(

)I(*F

|)I(F)I(*F|

Using the result of theorem, we can construct a polynomial time –

approximation scheme for the scheduling problem. This scheme has as

an input variable. For any input , it computes an integer k such that (1-1/m)/(1+[k/m]). This defines the k to be used in the scheduling rule described

above. Solving for k, we obtain that any integer k, k>(m – 1)/ – m,

guarantees -approximate schedules. The time required to obtain such

schedules, however, depends mainly on the time needed to obtain an

optimal schedule for k tasks on m machines. Using a branch and bound

algorithm this time is O (mk). The time needed to arrange the tasks so that ti

t i+1 and also to obtain the LPT schedule for the remaining n – k tasks is

O(n log n). Hence the total time needed by the -approximate scheme is

O(n log n+mk) = O(n log n)+m[(m-1)/ – m]).Since this time is not polynomial in

1/ (it is exponential in1/ ), this approximation scheme is not a fully

polynomial time approximation scheme. It is a polynomial time

approximation scheme (for any fixed m) as the computing time is polynomial

in the number of task n.




4. A feasible solution with value close to the value of an optimal solution is

called an ––––––––––.

5. An -approximate algorithm is an approximate algorithm for which ––––.

6. –––––––– are broken in an arbitrary manner.

7. The bin packing problem is –––––––––.

8.5 Summary

In this unit, we studied the following

f(n) = (g(n)) is the lower bound on time for any algorithm.

Cooks theorem states that satisfiability is in P if and only if P=NP.

A k-clique in a graph G is a complete sub-graph of G with k-vertices.

A vertex cover of G, where G=(V, E) is a undirected graph, is a subset S

V such that each edge of G is incident upon some vertex in X.

The bin packing problem is NP-hard.


1. What do you mean by Order Searching

2. Explain Cook’s Theorem.

8.7 Answers


1. compare x and A [i]

2. small degree

3. K-vertices

4. approximate solution

5. f(n) for some constant

6. Ties

7. NP-hard



Terminal Questions

1. In the ordered searching problem, we consider only those comparison

based algorithms in which every comparison between two elements of S

is of the type “compare x and A [i]”. The progress of any searching

algorithm that satisfies this restriction can be described by a path in a

binary tree. Each internal node in this tree represents a comparison:

between x and A[i]. (Refer Section 8.2).

2. Cook’s theorem states that satisfiability is in P if and only if P=NP. We

now prove this important theorem. We know that satisfiability in NP.

Hence, if P = NP, then satisfiability is in P. It remains to be shown that if

satisfiability is in P, then P = NP. To do this, we show how to obtain from

any polynomial time non- deterministic decision algorithm A and input I a

formula Q (A, I) such that Q is satisfiable if A has a successful

termination with input I. (Refer Section 8.3).



Unit 9 Graphs Structure

9.1 Introduction

Objectives

9.2 Preliminary Definitions and Notations

9.3 Trees

9.4 Summary


9.6 Answers

9.1 Introduction

In the previous units we discussed the different techniques of Algorithms. In

order to use these techniques in a proper way, you should be knowing the

concepts of graph. Because of its inherent simplicity, graph theory has a

very wide range of applications in engineering, in physical, social and

biological sciences, in linguistics and in numerous other areas. A graph can

be used to represent almost any physical situation involving discrete objects

and a relationship among them.

Objectives


use the definition of graph with related terminologies such as vertex,

edge etc.

apply the concept of directed graph

use the terminologies such as degree, complete graph, bipartite graph

and connectivity

use the definition of trees and properties of trees.

9.2 Preliminary Definitions and Notations

Definition: A linear graph (or simply a graph) G = (V,E) consists of a set of

objects V = {v1,v2…..} called vertices, and another set E = {e1,e2…}, whose

elements are called edges, such that each edge ek is identified with an

unordered pair (vi, vj) of vertices. The vertices vi, vj associated with edge ek

are called the end vertices of ek. The most common representation of a

graph is by means of a diagram in which the vertices are represented as

points and each edge as a line segment joining its end vertices. Often, this



diagram itself is referred to as the graph. The object shown in Fig. 9.1 for

instance, is a graph.

Fig. 9.1: Graph with five vertices and seven edges

An edge having the same vertex as both its end vertices called a self loop.

Edge e1 in Fig. 9.1 is a self loop. Also, note that the definition allows more

than one edge associated with a given pair of vertices, for example, edge e4

and e5 in Fig. 9.1. Such edges are referred to as parallel edges.

A graph that has neither self-loops nor parallel edges is called a simple

graph. Some authors use the term general graph to emphasize that parallel

edges and self-loops are allowed.

A graph is also called a linear complex, a 1 – complex, or a one-dimensional

complex. A vertex is also referred to as a node, a function, a point, o-cell or

an O-simplex. Other terms used for an edge are a branch, a line, an

element, a 1-cell, an arc, and a – simplex. In this unit, we shall generally use

the terms graph, vertex, and edge.

Definition: A directed graph G consists of a set of vertices

V = {v1,v2,……}, a set of edges E = {e1, e2…..}, and a mapping that maps

every edge onto some ordered pair of vertices (vi,vj). As in the case of

undirected graph, a vertex is represented by a point and an edge by a line

segment between vi and vj with an arrow directed from vi to vj.

Definition: A walk is defined as a finite alternating sequence of vertices and

edges, beginning and ending with vertices, such that edge is incident with

the vertices preceding and following it. No edge appears more than once in



a walk. A vertex, however, may appear more than once. A walk is also

referred to as an edge Train or a chain. The set of vertices and edges

constituting a given walk in a graph G is clearly a sub graph of G.

Vertices with which a walk begins and ends are called its terminal vertices,

Vertices v1 and v5 are the terminal vertices of the walk shown in Fig. 9.2. It is

possible for a walk to begin and end at the same vertex. Such a walk is

called a closed walk. A walk that is not closed (i.e. Terminal vertices are

distinct) is called an open walk (Fig. 9.2).

Fig. 9.2: An open Walk

An open walk in which no vertex appears more than once is called a path

(or a simple path or an elementary path). In the graph, v1a v2b v3d v4 is a

path where as v1a v2b v3d v4e v2f v5 is not a path. In other words, a path

does not intersect itself.

The number of edges in a path is called the length of a path. It immediately

follows, then, that an edge which is not a self-loop is a path of length one. It

should also be noted that a self-loop can be included in a walk but not a

path.

Definition: A closed walk in which no vertex (except the initial and the final

vertex) appear more than once is called a circuit. That is, a circuit is a

closed, nonintersecting walk. In Fig. 9.2., v2b v3d v4e v2 is, for example, a

circuit. Three different circuits are shown in Fig. 9.3 A. Clearly, every vertex



in a circuit is of degree two; again, if the circuit is a sub graph, one must

count degrees contributed by the edges in the circuit only.

Fig. 9.3 A: Three different circuits

A circuit is also called a cycle, elementary cycle, circular path and polygon.

In electrical engineering a circuit is sometime refered to as a loop.

The definitions in this section are summarized in Fig. 9.3 B the arrows are in

the direction of increasing restriction.

Fig. 9.3 B: Walks, Paths and Circuits as Subgraphs

Definition: The number of edges incident on a vertex vi with self-loops

counted twice, is called the degree, d(vi), of vertex vi. In Fig. 9.4, for

example,

d(v1) = d(v3) = d(v4) = 3, d(v2) = 4, and d(v5) = 1. The degree of a vertex is

sometimes also referred to as its valency.

Sub graph of G

Walk in G

Path in G Circuit in G

Any collection of edges in G

A non-edge-retracing sequence of edge of G

A non-intersecting closed walk in G

A non-intersecting open walk in G



Fig. 9.4: A Graph with five vertices and seven edges.

Definition: A simple graph in which there exists an edge between every pair

of vertices is called a complete graph. Complete graphs of two, three, four

and five vertices are shown in Fig. 9.5. A complete graph is sometimes also

referred to as a universal graph or a clique.

Fig. 9.5: Complete Graphs of two, three, four and five vertices.

Since every vertex is joined with every other vertex through one edge, the

degrees of every vertex is n –1 in a complete graph G of n vertices. Also the

total number of edge in G is 2

)1n(n.

Components of a graph: Let G = (V, E) be a disconnected graph.

We define a relation ~ on the set of vertices as follows:

v ~ u there is a walk from v to u.

Then this relation ~ is an equivalence relation.

Let {Vi}i be the collection of all equivalence classes.



Now, V = i

iV . Write Ei = {e E / an end point of e is in Vi} for

each i. Then (Vi, Ei) is a connected subgraph of G for every i . This

connected subgraph (Vi, Ei) of G is called a connected component (or

component) of G for every i .

The collection {(Vi, Ei)}i of subgraphs of G is the collection of all connected

components of G.

If G is a connected graph, then G contains only one connected component

and it is equal to G. Now, suppose that G is a disconnected graph. Consider

a vertex v in G. If each vertex of G is joined by some path to v, then the

graph is connected, a contradiction. So, there exists at least one vertex

which is not joined by any path to v. The vertex v and all the vertices of G

that are joined by some paths to v together with all the edges incident on

them form a component (G1, say). To find another component, take a vertex

u (from G) which is not in G1. The vertex u and all the vertices of G that are

joined by some paths to u together with all the edges incident on them form

a component (G2, say). Continue this procedure to find the components.

Since the graph is a finite graph, the procedure will stop at a finite stage. In

this way, we can find all the connected components of G. It is clear that, a

component itself is a graph.

Observations:

i. If G is a connected graph, then G is only the connected component of

G. So if G is a connected graph, then G contains only one connected

component.

ii. A disconnected graph G consists of two or more connected

components.

iii. Connected component of a graph G is a maximal connected subgraph

of G.

iv. Consider the graph given.

Fig 9.6: A graph



This graph is a disconnected graph with two components.

Definition: A graph G is called bipartite if its vertex set V can be

decomposed into two disjoint subsets V1 and V2 such that every edge in G

joins a vertex in V1 with a vertex in V2. Thus, every tree is a bipartite graph.

A bipartite graph can have no self-loop. A set of parallel edges between a

pair of vertices can all be replaced with one edge without affecting

bipartiteness of a graph.

Definition: A graph G is said to be connected if there is at least one path

between every pair of vertex in G. Otherwise, G is disconnected. For

instance the graph in Fig. 9.4 is connected but in Fig. 9.7 is disconnected A

null graph of more than one vertex is disconnected.

Fig. 9.7: A Disconnected Graph with two Components

It is easy to see that a disconnected graph consists of two or more

connected graphs. Each of these connected sub graphs is called a

component. The graphic in the above figure consists of two components.

Another way of looking at the components is as follows: Consider a vertex vi

in a disconnected graph G. By definition, not all vertices of G are joined by

paths to vi, Vertex vi and all the vertices of G that have paths to vi, together

with all the edges incident on them, form a component, obviously, a

component itself is a graph.

Definition: An edge that has not acquired any takes is a bridge. (A bridge is

an edge whose removal disconnects the graph).



Definition

A separable graph consists of two or more non-separable sub graphs. Each

of the largest non-separable sub graphs is called a block. The graphs in

Fig. 9.8 has five blocks (and three cut-vertices a, b, and c) ; each block is

shown enclosed by a broken line. Note that a non-separable connected

graph consists of just one block.

Fig. 9.8: Separate graph with three cut vertices and five blocks


1. A –––––– is also called a linear complex.

2. A –––––––– is defined as a finite alternating sequence of vertices and

edges.

3. A closed walk in which no vertex (except the initial and the final vertex)

appear more than once is called a ––––––––– .

4. A simple graph in which there exists an edge between every pair of

vertices is called a –––––––––––– .

5. A graph G is called ––––––––––– if its vertex set V can be decomposed

into two disjoint subsets.

6. An edge that has not acquired any takes is a –––––––– .

9.3 Trees

A tree is a connected graph without any circuits. The graph in

Fig. 9.9 A, for instance, is a tree. Trees with one, two, three and four

vertices are shown in Fig. 9.9 B. A graph must have at least one vertex, and

therefore so must a tree. Some authors allow the null tree, a tree without

any vertices.

It follows immediately from the definition that a tree has to be a simple

graph, that is, having neither a self-loop nor parallel edges (because they

both form circuits).



Trees appear in numerous instances. The genealogy of a family is often

represented by means of tree (in fact the term tree comes from family tree)

A river with its tributaries and subtributaries can be represented by a tree.

The sorting of mail according to zip code and the punched cards is done

according to a tree (called decision tree or sorting tree).

Fig. 9.9 A: Tree

Fig. 9.9 B: Tree with one, two, three, and four vertical



Fig. 9.9 C: Decision Tree

Fig. 9.9 C might represent the flow of mail, all the mail arrives at some local

office, vertex N. The most significant digit in the zip code is read at N, and

the mail is divided into 10 piles N1, N2, … N9 and N0 on the most significant

digit. Each pile is further divided into 10 piles according to second most

significant digit, and so on, till the mail is sub divided into 105 possible piles,

each representing a unique five-digit zip code.

Sub-graphs: A graph g is said to be a sub-graph of graphs G if all the

vertices and all the edges of g are in G, and each edge of g has the same

end vertices in g as in G. For instance, the graph in Fig. 9.10 B is a

sub-graph of one in Fig. 9.10 A. The concept of sub-graphs is akin to the

concept of subset in set theory. A sub-graph can be thought of as being

contained in another graph. The symbol from set theory, g h is used in

stating “g is a sub-graph of G”.



(A) (B)

Fig. 9.10: Graph(A) and one of its Sub-graph(B)

The following observations can be made immediately :

1) Every graph is its own sub-graph

2) A sub-graph of a sub-graph of G is a sub-graph of G

3) A single vertex in a graph G is a sub-graph of G

4) A single edge in G, together with its end vertices, is also a sub-graph

of G.

Definition: Two sub-graphs g1 and g2 of graph G are said to be edge disjoint if

g1 and g2 do not have any edges in common, note that although edge disjoint

graphs do not have any edge in common, they may have vertices in common.

Sub-graphs that do not even have vertices in common are said to be vertex

disjoint.

Definition: A tree T is said to be a spanning tree of a connected graph G, if

T is a sub-graph of G and T contains all vertices of G. For instance, the

sub-graphs in heavy lines in Fig. 9.11 is a spanning tree of the graph shown



Fig. 9.11: Spanning Tree

Since the vertices of G are barely hanging together in a spanning tree, it is a

sort of skeleton of the original graph G. This is why a spanning tree is the

largest (with maximum number of edges) tree among all trees in G. It is also

quite appropriate to call spanning tree a maximal tree sub-graph or maximal

tree of G.

It is to be noted that spanning tree is defined only for a connected graph,

because a tree is always connected, and in a disconnected graph of n

vertices we cannot find a connected sub-graphs with n vertices. Each

component of a disconnected graph, however, does have a spanning tree.

Thus, a disconnected graph with K component has a spanning forest

consisting of K spanning trees (A collection of trees is called a forest).


7. A ––––––––is a connected graph without any circuits.

8. A graph g is said to be a ––––––––of graphs G if all the vertices and all

the edges of g are in G

9. A ––––––––––vertex in a graph G is a sub-graph of G

10. A tree T is said to be a ––––––––– tree of a connected graph G if T is a

sub-graph of G and T contains all vertices of G



9.3 Summary

In this unit, we studied

Some basic concepts of graph theory and some elementary results

based on these concepts.

That graphs can be used to represent almost any problem involving

discrete arrangements of objects, where concern is not with the internal

properties of these objects but with the relationship among them.

The concepts of trees with basic applications to its different properties.


1. In a ––––– an edge is not only incident on a vertex, but is also incident

out of a vertex and incident into a vertex.

2. –––––– are digraphs in which for every edge(a, b) there is also an edge

(a, b) there is also an edge (b, a).

3. A digraph is said to be –––––– if for every vertex vi the in-degree equals

the out-degree.

4. A closed walk in which no vertex appear more than once is called a

___________.

5. A graph G is said to be –––––– if there is at least one path between

every pair of vertices in G.

6. A connected graph without any circuit in ––––––––––––.

7. Give the definition of Walk.

9.5 Answers


1. graph

2. walk

3. circuit

4. complete graph

5. bipartite

6. bridge.

7. tree

8. sub-graph

9. single

10. spanning



Terminal Questions

1. Graph

2. Symmetric digraphs

3. Balanced

4. Circuit

5. Complete

6. Spanning tree

7. A walk is defined as a finite alternating sequence of vertices and edges,

beginning and ending with vertices, such that edge is incident with the

vertices preceding and following it. No edge appears more than once in

a walk. A vertex, however, may appear more than once. A walk is also

referred to as an edge Train or a chain. The set of vertices and edges

constituting a given walk in a graph G is clearly a sub graph of G


Sikkim Manipal University Page No: 177

Unit 10 Trees and Algorithms Structure

10.1 Introduction

Objectives

10.2 Characterization of Trees

10.3 Rooted Trees and Applications

10.4 Spanning Trees

10.5 Algorithms for Spanning Trees

Kruskal Algorithm

Prim‟s Algorithm

10.6 Summary


10.8 Answers

10.1 Introduction

After studying the concept of graphs in the previous unit, now, you will study

the concept of trees. Trees are extensively used as models in area like

computer science, chemistry and in search procedures. These are also

useful in design of wide range of algorithms. In this unit, we will provide

some characterizations of trees with suitable illustrations. We will also

introduce a special type of trees namely rooted trees and binary trees. We

will give few applications of these trees. The spanning trees will be

introduced and the algorithms for spanning trees will also be obtained.

Objectives


describe the connected graphs without circuits (called trees) with their

properties

characterize a tree from a connected graph

apply the concept of trees in real life situations

analyze and write algorithms for spanning trees.

10.2 Characterization of Trees

The concept of a „tree‟ plays a vital role in the theory of graphs. First, we

introduce the concept of „tree‟, study its properties and some of its

applications. Later, we introduce the concept of „spanning tree‟, and study

the relationships among circuits and trees.



Definition: A connected graph without circuits is called a tree. A collection

of trees is called a forest.

Example:

Trees with one, two, three and four vertices are given in the Fig. 10.1

Fig. 10.1

Observations

i) A tree contains at least one vertex.

ii) A tree without any edge is referred to as a null tree.

iii) The trees considered are finite and a tree is always a simple graph.

Some natural examples

i) The list of the ancestors of a family may be represented by a tree. This

tree referred to as a family tree.

ii) A river with its tributaries and sub tributaries may be represented by a

tree. This tree is referred to as a river tree.

iii) The sorting of mail according to zip code are done according to a tree.

This tree is called decision tree (or) sorting tree.

Theorem: In a tree T, there is one and only one path between every pair of

vertices.

Proof: Suppose T is a tree.

By definition, T is a connected graph and contains no circuits.

Since T is connected, there exists at least one path between every pair of

vertices in T. Suppose that between two vertices a and b of T, there are two

distinct paths. Now, the union of these two paths will contain a circuit in T, a

contradiction (since T contains no circuits).

This shows that there exists one and only one path between a given pair of

vertices in T.

Theorem: If there is one and only one path between every pair of vertices in

G, then G is a tree.



Proof: Assume that there is one and only one path between every pair of

vertices in a graph G. Clearly G is connected.

If possible, suppose that G contains a circuit. Then, there is at least one pair

of vertices a, b such that there are two distinct paths between a and b,

which is a contradiction to our assumption. So G contains no circuits. Thus

G is a tree.

Theorem: A tree G with „n‟ vertices has (n - 1) edges.

Proof: (Use induction on n):

Step (i): If n = 1, then G contains only one vertex and no edge.

So the number of edges in G is n -1 = 1 - 1 = 0.

Step (ii): Induction hypothesis: The statement is true for all trees with less

than „n‟ vertices.

Step (iii): Let us consider a tree with „n‟ vertices.

Let „ek‟ be any edge in T whose end vertices are vi and vj.

Since T is a tree, there is no other path between vi and vj.

So by removing ek from T, we get a disconnected graph.

Also, T- ek consists of exactly two components(say T1 and T2).

Since T is a tree, there were no circuits in T and so there were no circuits in

T1 and T2. Therefore, T1 and T2 are also trees.

It is clear that |V(T1)| + |V(T2)| = |V(T)| where V(T) denotes the set of

vertices in T.

Also, |V(T1)| and |V(T2)| are less than n.

Therefore, by the induction hypothesis, we have

|E(T1)| = |V(T1)| - 1 and |E(T2)| = |V(T2)| - 1.

Now |E(T)| - 1 = |E(T1)| + |E(T2)| = |V(T1)| - 1 + |V(T2)| - 1

Thus |E(T)| = |V(T1)| + |V(T2)| - 1 = |V(T)| - 1 = n-1.

Theorem: Any connected graph with „n‟ vertices and n -1 edges is a tree.

Proof: Let „G‟ be a connected graph with n vertices and n - 1 edges.

It suffices to show that G contains no circuits.

If possible, suppose that G contains a circuit. Let „e‟ be an edge in that

circuit.



Since ‘e‟ in a circuit, we have that G - e is still connected.

Now G - e is connected with „n‟ vertices, and so |G - e| n -1 edges, a

contradiction (to the fact that G - e contain only (n-2) edges, and so G

contains no circuits. Therefore, G is a tree.

Definition: A connected graph is said to be minimally connected if the

removal of any one edge from the graph provides a disconnected graph.

Example:

i) Graph given in Fig. 10.2 A is not minimally connected.

ii) Graph given in Fig10.2 B is minimally connected.

iii) Every circuit is not minimally connected.

iv) Every tree is minimally connected.

Fig. 10.2 A Fig. 10.2 B

Problem: A graph G is a tree if and only if it is minimally connected.

Solution: Part (i): Assume that G is a tree. If G is not minimally connected,

then there exists an edge „e‟ such that G - e is connected. That is, e is in

some circuit, which implies G is not a tree, a contradiction. Hence G

minimally connected.

Part (ii): Suppose that G is minimally connected.

If G contains a circuit, then by removing one of the edges in the circuit, we

get a connected graph, a contradiction (to the supposition). This shows that

G contains no circuits. Thus G is a tree.

Observation: To join „n‟ given distinct points, the minimum number of line

segments needed is n -1.

Problem: If a graph G contains n vertices, n -1 edges and no circuits, then

G is a connected graph.

Proof: Let G be a graph with „n‟ vertices, n – 1 edges and contains no circuits.



Suppose that (in a contrary way) G is disconnected.

G consists of two or more circuitless components (say, g1, g2, , gk) (here k 2).

Choose a vertex vi in gi, for 1 i k.

Add new edges e1, e2, …, ek-1 where ei = 1iivv to get a new graph G*.

It is clear that G* contains no circuits and connected, and so G* is a tree.

Now G* contains n vertices and

(n - 1) + (k - 1) = (n + k - 2) n edges, a contradiction (since a tree

contains (n - 1) edges). This shows that G is connected.

Characterization for Tree

For a given graph G, the following conditions are equivalent:

i) G is connected and is circuit less,

ii) G is connected and has n -1 edges,

iii) G is circuitless and has n -1 edges,

iv) There is exactly one path between every pair of vertices in G,

v) G is a minimally connected graph,

vi) G is a tree.

Problem: If G is a tree (with minimum two vertices), then there exists at

least two pendant vertices.

Proof: Let G be a tree with |V| 2.

Let v0e1v1e2v2e3 . . . vn-1envn be a longest path in G (since G is finite graph,

it is possible to find a longest path).

Now we show that d(v0) = 1 = d(vn).

If d(v0) > 1, then there exists at least one edge e with end point v0 such

that e e1. If e {e1, e2, … en}, then e = ei for some i 1.

So either vi-1 = v0 or vi = v0 v0 repeated in the path, a contradiction.

Therefore, e {e1, e2, e3 , … en} and so e, e1, e2, e3 , … en is a path of length

n + 1, a contradiction.

Thus, d(v0) = 1. In a similar way, we can show that d(vn) = 1. Hence v0, vn

are two pendant vertices.



Definition: Let G be a connected graph. The distance between two vertices

v and u is denoted by d(v , u) and is defined as the length of the shortest

path [that is, the number of edges in the shortest path] between v and u. In

a connected graph, we can find the distance between any two given

vertices.

Example:

i) Consider the connected graph given in Fig. 10.3A. Here some of the

paths between v1 and v2 are (a, e), (a, b, f), (b, c, e), (b, f), (b, g, h), (b,

g, i, j), (b, g, i, k). Here there are two shortest paths (a, e) and (b, f)

each of length 2. Hence, d(v1, v2 ) = 2.

Fig. 10.3 A

ii) Consider the tree given in the Fig. 10.3 B. Here d(a, b) = 1, d(a, c) =

2, d(a, d) = 2, d(b, d) = 1.

Fig. 10.3 B


1. Any connected graph with „n‟ vertices and n -1 edges is a –––––––– .

2. A graph G is a tree if and only if it is –––––––––––– .

10.3 Rooted Trees and Applications

Rooted trees are extensively used in the computer search methods, binary

identification problems, and variable length binary codes.

j

b

d

c

v2

v1 i

h

k

g

e

a

f

a

b c d



Definition: A tree in which one vertex (called the root) is distinguished from

all the other vertices, is called a rooted tree. In a rooted tree, the root is

generally marked in a small triangle (or small circle).

Example:

Distinct rooted trees with four vertices, were given in Fig. 10.4

Fig. 10.4

Generally, the term „tree‟ means trees without any root. However, they are

sometimes called free trees (or) non-rooted trees. A variety of rooted trees

(called the Binary rooted trees) is of particular interest.

Definition: A tree in which there is exactly one vertex of degree 2, and all

other remaining vertices are of degree one or three, is called a binary tree.

Fig-.10.5

v3

v2

v4

v6

v8

v5

v11

v7

v1

v10

v13

v9

v12



i) The Fig. 10.5 represents a binary tree (since the only vertex „v1‟ is of

degree 2, and all other vertices are of degree either 1 or 3).

ii) The vertex of degree 2 (that is, v1) is distinct from all other vertices, this

vertex v1 is the root.

iii) In a binary tree, the vertex with degree 2 serves as a root. So, every

binary tree is a rooted tree.

Properties of Binary trees

Property (i): The number of vertices n, in a binary tree is always odd.

Property (ii): The number of pendent vertices is 2

1n.

Property (iii): Number of vertices of degree 3 is

= n - p - 1 = n - (2

1n) - 1 =

2

3n.

Example:

In the graph given in Fig. 10.5, we have that n = 13, p =2

1n =

2

113 =

2

14=7. Therefore, number of vertices of degree 3 is

2

3n=

2

313 = 5.

Definition:

A non-pendent vertex in a tree is called an internal vertex.

Observation:

i) The number of internal vertices in a Binary tree is

2

1n = (p - 1) where p = the number of pendent vertices.

ii) In the binary tree given in Fig. 10.5, the internal vertices are v1, v3, v4,

v5, v6, v9. These are 6 (=7 - 1 = p - 1) in number.

Definition: Let v be a vertex in a binary tree. Then v is said to be at level I if

v is at a distance of I from the root.

Example:

A 13-vertex, 4-level binary tree is given in Fig. 10.6



Fig. 10.6

Here, the number of vertices at levels 0, 1, 2, 3, 4 are 1, 2, 2, 4 and

4 respectively.

Definition: The sum of path lengths from the root to all pendent vertices is

called the path length (or) external path length of a tree.

Example:

i) The path length of the binary tree given in Fig. 10.6 is:

1 + 3 + 3 + 4 + 4 + 4+ 4 = 23.

ii) In the Figures 10.7 A and B, there are two 11-vertex binary trees.

Fig. 10.7 A

The path length of graph (fig. 10.7 A): 2 + 2 + 3 + 3 + 3 + 3 = 16.

level -1

level - 0

level -2

level -3

level 1

level 4

level 0

level 2

level 3



Fig. 10.7 B

The path length of graph (fig. 10.7 B): 1 + 2 + 3 + 4 + 5 + 5 = 20.

Search procedures:

Each vertex of a binary tree represents a test with two possible outcomes.

We start at the root. The outcome of the test at the root sends us to one of

the two vertices at the next level, where further tests are made and so on.

Reaching a specified pendent vertex (that vertex which represents the goal

of the search), terminates the search.

For such search procedures, it is often important to construct a binary tree in

which, for a given number of vertices n, the vertex frothiest from the root in

as close to the root as possible.

i) There can be only one vertex (the root) at level 0. Number of vertices

at level one is at most 2. Number of vertices at level two is at most 22

and so on. So the maximum number of vertices possible in a k -level

binary tree is 20 + 21 + 22 + . . . + 2k.

So n 20 + 21 + 22 + . . . + 2k

ii) The maximum number among the levels of the vertices in a binary tree

is called height of the tree. So height = max { level of a vertex v /

v V}.

This height is denoted by l max .

iii) To construct a binary tree for a given n such that the farthest vertex

is as far as possible from the root, we must have exactly two vertices at

each level, except at the 0 level. So max l max = 2

1n.

level 0

level 5

level 1

level 2

level 4

level 3



An Application: (Coke Machine Problem): The machine is said to have a

sequence of tests (for example, it should be capable of identifying the coin

that is put into the machine). We suppose that five rupees coin, two rupees

coin, one rupee coin and fifty paise coin can go through the slot. So the

machine can identify only these four coins. Every coin put in, is to be tested

by the machine. Each test got the effect of partitioning the coins into two

complementary sets. [Suppose a coin is put into the machine. It should test

whether the coin is “five rupee coin”. If it is not a five rupees coin, then it

should test whether it is a two rupees coin and so on]. We suppose the time

taken for each test is –

Test Pattern-1: One type of testing pattern was shown in Graph-(i), given

in Fig. 10.8 A.

Suppose the statistical data tells that

w1 = probability of putting a Rs 5 coin = 0.5



w4 = probability of putting a Rs 0.5 coin = 0.1

Now

)v(lw ii = )v(lw 11 + )v(lw 22 + )v(lw 33 + )v(lw 44

= (0.5) (1) + (0.2) (2) + (0.2) (3) + (0.1) (4) = 1.9

Fig. 10.8 A

not Rs 0.5 coin

Rs 0.5 coin

Rs 1 coin

Rs 5 coin

v3 0.2

v1

v2

v4 0.1

0.2

0.5

not Rs 1 coin

not Rs 2 coin

not Rs 5 coin

Rs 2 coin



So expected time to be taken by the machine for testing one coin is 1.9t.

Thus, if the machine follows (for its testing pattern) the binary tree given in

Graph-(a), then the expected time for testing one coin is equal to 1.9t.

Test Pattern-2: Another type of testing pattern was given in the

Fig. 10.8 B.

)v(lw ii = )v(lw 11 + )v(lw 22 + )v(lw 33 + )v(lw 44

= (0.5) (2) + (0.2) (2) + (0.2) (2) + (0.1) (2) = 2.0

Fig. 10.8 B

So, here, the expected time to be taken by the machine for testing one coin

is 2t.

Thus, if the machine follows (for its testing pattern) the binary tree given in

graph 10.8 B then the expected time for testing one coin is 2t.

Definition: A graph G in which every vertex is assigned a unique name or

label (that is, no two vertices have the same label) is called a labeled graph.

Otherwise is said to be an unlabeled graph.

Example:

If n = 4, then there are 16 trees, shown below.

Take the vertex set V = {A, B, C, D}.

v1 v2 v3 v4

0.1 0.2 0.2 0.5

Rs 5 or Rs 2

Rs 5 Rs 2

Rs 1 Rs 0.50

Rs 1 or Rs 0.50



The following are the 16 trees of four labeled vertices

B

C

A

D

A

D C

B A

C D

B

C

B A

D

A B

D C D C

B A

A B

C D

A B

C D

A

C

B

D

B

D

A

C

A B

C D

A B

C D

A B

C D

A B

C D

A B

C D

A B

C D



Observations:

i) When we count the number of different graphs, the distinction between

labeled and unlabeled graphs is important.

ii) Consider the graphs in above example the 5th, 6th, 7th and 8th, are

counted as four different trees (even through they are isomorphic),

because they are labeled. If there is no distinction between A, B, C,

D, then these four trees counted as one.

iii) A careful inspection of the 16 graphs will reveal that the number of

unlabeled trees with four vertices (no distinction made between A, B,

C, D), is two. These two graphs were.

Cayley’s Theorem:

The number of labeled trees with n vertices (n 2) is n(n-2) .

Example:

Part-(i): Suppose the given tree is T1 (Fig. 10.9 A). a1 = the pendent vertex

with smallest label. So, a1 is the vertex 2. Now b1 = 1. After removing a1

and the edge (a1, b1), in the remaining graph, a2 = the pendent vertex with

smallest index = 4.

Fig. 10.9 A

a1

1

2

6

5

3

7

9

4

8



Now b2 = 1. a3 = 1, b3 = 3, a4 = 3, b4 = 5, a5 = 6, b5 = 5, a6 = 7, b6 = 5, a7 = 5,

b7 = 9.

Therefore, we have the sequence (b1, b2, b3, b4, b5, b6, b7)=(1, 1, 3, 5, 5, 5, 9).

Part (ii): (converse of part (i)): we have to construct a tree with n = 9 vertices.

Consider 1, 2, 3, 4, 5, 6, 7, 8, 9 . . . . . . . . . (i)

Given (n - 2)-tuple is (1, 1, 3, 5, 5, 5, 9) . . . . . . . (ii)

Fig. 10.9 B

Observe the sequence in (ii). First join 1 and 3 , 3 and 5, 5 and 9.

Then, we get the graph given in Fig. 10.9 B. Now the least number in (i)

which is not in (ii), is 2. So we join 2 and 1. Next least in (i) which is not in (ii)

is 4. So we join 4 and 1. Cancel 3 in (i) and 3 in (ii). Also cancel 5 in (i) and 5

in (ii) (in (ii), canceling of only one 5 is allowed). The next least which is not

in (ii) is 6. So we join 5 and 6. The next least which is not in (ii) is 7. So we

join 7 and 5. The next least which is not in (ii) is 8. The remaining number

available in (ii) is 9. So, we join 8 and 9.

Fig. 10.9 C

1 3

5

9

8

1

2

6

5

3

7

9

4



Now, we will study the tree as a subgraph of another graph. A given graph

may have numerous subgraphs. If e is the number of edges in G, then

there are 2e distinct subgraphs possible. Obviously, some of these

subgraphs will be trees. Out of these trees, we are particularly interested in

certain type of trees, called spanning trees.


3. A tree in which one vertex (called the root) is distinguished from all the

other vertices, is called a ––––––––––––– .

4. The number of pendent vertices is –––––––––.

10.4 Spanning Trees

Definition: A tree T is said to be a spanning tree of a connected graph G

if T is a subgraph of G and T contains all the vertices of G.

Example:

Consider the graph G given in Fig. 10.10 A. Graph T (given in Fig. 10.10B)

is a spanning tree of G.

Fig. 10.10 A: Graph-G Fig. 10.10 B: Graph-T

v3

v1

v4 v2

v5

v6

v7

b2

b3

b1

b6

b4

c5

c6

c8

c7

c1

b5

c2

c3

c4

v7

v1

v2

v3

v4

v5

v6



Observations:

i) Spanning trees are the largest (with the maximum number of edges)

trees among all trees in G. Spanning tree is also called a maximal tree

subgraph or maximal tree of G.

ii) Spanning is defined only for a connected graph (since a tree is always

connected).

iii) Each component of a disconnected graph, does have a spanning tree.

Thus, a disconnected graph with k components contains a spanning

forest consisting of k spanning trees.

Theorem: Every connected graph has at least one spanning tree.

Definition:

i) An edge in a spanning tree T is called a branch of T.

ii) An edge of G that is not in a given spanning tree T is called a

chord. In electrical engineering, chord some times referred to as tie or a

link.

iii) Let T be any spanning tree of a connected graph G, and T1 is the

complement of T in G. Then, each edge in T is called a branch (with

respect to T), and the set of all edges in T is called the branch set.

Each edge in T1 is called a chord (with respect to T), and the set of

edges in T1 is called the chord set (or) Tie set. T1 is called as the

cotree. We may write T instead of T1.

Observations:

i) Branches and chords are defined only with respect to a given spanning

tree.

ii) An edge that is a branch with respect to one spanning tree T1 (of G)

may be a chord with respect to another spanning tree T2.

Theorem: With respect to any of its spanning trees, a connected graph of

„n‟ vertices and „e‟ edges has „n-1‟ tree branches and e - n + 1 chords.

Proof: Let G be any connected graph on „n‟ vertices and e edges. Let T be

any spanning tree in G. Since every spanning tree of G contains all vertices

of G, we have that |V(T)| = n and so |E(T)| = |V(T)| - 1 = n - 1. Since

every edge of a spanning tree T is called a branch of T, we have that G

contains n - 1 branches. Since the number of edges in G is e , we have

that the number of chords of T is e - (n - 1) = e - n + 1.



Example:

There is a form consisting of six walled plots of land as shown in the fig and

these plots are full of water. We find the minimum number of walls are to be

broken so that all the water can be drained out.

Consider the wall joints as vertices, and walls as edges. Then, we can

consider it as a graph. In this graph, the number of vertices is n = 10, and

the number of edges is e = 15.

Fig. 10.11

If there exists a circuit, then the water inside the circuit cannot be drained

out. So, we have to remove minimum number of edges so that the graph do

not contain circuits. To have this, we should have a spanning tree with

(n - 1) edges. Hence, we have to break e - (n - 1) = e - n + 1 = 15 - 10

+ 1 = 6 edges (walls) so that all the water can be drained out.

If we add an edge between any two vertices of a tree, then a circuit is

created. This is because, there already exists one path between any two

vertices of a tree, adding an edge in between, creates an additional path,

and hence a circuit. The concept of a fundamental circuit has an enormous

significance in electrical network analysis.

Definition:

Let T be any spanning tree of a connected graph G. Adding any one chord

to T will create exactly one circuit. Such a circuit formed by adding a chord

to a spanning tree, is called a fundamental circuit.

Example:

Consider the graph G (given in Fig. 10.12A), and its spanning tree T (given

in Fig. 10.12 B) of G. Now, if we add the chord c1 to T, we get a circuit

„b1 b2 b3 b5 c1‟ which is called as fundamental circuit (given in Fig. 10.12C).



Fig. 10.12 A Fig. 10.12 B Fig. 10.13 C

Graph-G Spanning tree-T Fundamental circuit-F

Note: A connected graph G is a tree if and only if adding an edge

between any two vertices in G creates exactly one circuit.

Note: For a tree G is a tree, by adding a new edge between any two

vertices in G creates exactly one fundamental circuit.


5. Every connected graph has at least one –––––––––––––––.

10.5 Algorithms for Spanning Trees

Definition:

i) A graph G is said to be a weighted graph if all the edges e of G were

assigned by a corresponding real number (e) (read as the weight of e).

ii) Let T be any spanning tree of a connected graph G. then the weight of

T, that is, (T) is defined as the sum of weights of all branches in T.

iii) A spanning tree with the smallest weight in a weighted graph is called a

shortest spanning tree (or) minimal spanning tree (or) shortest-distance

spanning tree.

Observation:

i) Different spanning trees of G may have different weights. Among all the

spanning trees of G, one with the smallest weight is of practical

significance.

c6

c5

c7

c8

c4

c3

Fig. 10.10 B Graph-T

c1

b2

b1

b4

b5 b3 b6

c2 b1

b2

b3

b4

b6 b5

c1 b1

b2

b3

b4

b6 b5



ii) Let G be a graph on n vertices in which every edge has a unit weight .

Then all the spanning trees have the same weight of (n - 1) units.

Applications:

i) Suppose that we are to connect n cities v1, v2, …, vn through a network

of roads.

ii) The cost Cij of building a direct road between vi and vj is given for

pairs of cities vi and vj where roads can be built. (Note that there may

be pairs of cities between which no direct road can be built).

iii) Now the problem is: To find the least expensive network that connects

all „n‟ cities together. That is, to find a shortest spanning tree in a

connected weighted graph of „n‟ vertices.

iv) Here the connected network must be a tree. Otherwise, we can always

remove some edges and get a connected graph with smaller weight.

Thus „the problem of connecting „n‟ cities with a least expensive

network is the problem of finding a shortest spanning tree in a

connected weighted graph of „n‟ vertices.

10.5.1 Kruskal Algorithm:

This algorithm is used in finding shortest spanning tree

Step-(i): List all the edges of G in order of non-decreasing weight. Now we

select an edge e1 of G such that (e1) is as small as possible and „e1‟ is not

a loop.

Step-(ii): Select next smallest edge from the set of all remaining edges of

G such that the selected edge do not form a circuit with the edges that

have already been chosen.

Step-(iii): We continue this process of taking smallest edges among those

not already chosen, provided no circuit is formed with those, that have been

chosen already.

[If edges e1, e2, …, ei have been chosen, then chose ei+1 from E -

{e1, e2, …, ei} in such way that graph with {e1, e2, …, ei+1} is acyclic and

(ei+1) is as small as possible].

Step-(iv): If a graph G has „n‟ vertices, then we will stop this process

after choosing n - 1 edges. These edges form a subgraph T, which is not

cyclic. (Thus T is a shortest spanning tree of G).



Example: Consider the connected weighted graph G. Listing:

)1(AB , )1(AD , )2(BC , )2(CD , )3(AE , )3(FE , )3(AF , )4(EB ,

)4(AC , )4(ED , )5(FD .

Step-(i): Consider a spanning subgraph of G without edges.

Select the edge with minimum weight. For this graph the edge AB is of

minimum weight.

Add this edge e1 = AB to the spanning null graph. Then, we get the

subgraph-G1.

Step(ii): Select the edge e2 such that G1 + e2 contains no circuits and e2

got minimum weight among such edges. For this graph e2 = AD. Now, add

e2 = AD to G1. Then we get the subgraph-G2.

Step-(iii): We continue this process. For this example, we add e3 = BC to

G2 to get the subgraph-G3.

Graph-G Subgraph-G1

Next, we add e4 = AE to G3 to get the subgraph-G4. Next, we add

e5 = EF to get the subgraph-G5. If we add any edge to G5, we get a circuit.

So, the process is to be stopped here.

C

2

5 1

D

4

A

B

E

F

1

2 3 4

3

3

4

A

B 1 F

E

D

C



Subgraph-G2 Subgraph-G3

Subgraph-G4 Graph G5 or T

The final graph G5 is a subgraph of G and contains no cycles. Thus G5 is

a shortest spanning tree of G. Write T = G5. Now (T) = sum of the weights

of the branches in T = 1 + 2 + 1 + 3 + 3 = 10

10.5.2 Prim's Algorithm:

This algorithm is used to find the shortest spanning tree

Step- (i): Choose any vertex v1 in G.

Step-(ii): Choose an edge e1 = v1v2 of G such that v1 v2 and „e1‟ has

the smallest weight among the edges of G incident with v1. [For

convenience, we can form a table indicating the weights].

Step-(iii): If the edges e1, e2, …, ei have been already chosen involving

end vertices v1, v2, …, vi+1 , choose an edge „ei+1‟, where ei+1 = vjvk with

vj { v1, v2, …,vi+1}, and vk {v1, …, vi+1} such that ei+1 has the smallest

weight among the edges of G with precisely one end in {v1, v2, …, vi+1}.

A

B

E

1 F

1 2

D C

3

3

F

A

B

E

1

1 2

D C

3

F

E

A

B

1

1 2

D C C

A

B

1

1

D

F

E



Note that after adding ei+1, the graph should be a cyclic.

Step-(iv): The process will stop after choosing the n-1 edges (otherwise,

repeat the step (iii)).

Illustration: Consider the connected weighted graph G. Let us start with C.

Step-(i): Take the spanning null subgraph G0 of G.

Let us choose v1 = A. The edge AD is incident on v1 and has the smallest

weight among the edges incident on v1 = A. Write G1 = (G0 + AD)

Step-(ii): In this step we select DC and write G2 = (G1 + DC).

Step-(iii): We select AB and write G3 = (G2 + AB).

Step-(iv): We select AF and write G4 = (G3 + AF).

Step-(v): We select FE and write G5 = (G4 + FE).

Step-(vi): Since G5 is a spanning tree, the process stops here.

Write T = G5.

5 1

D

4 2

A

C

B

E

F

1

2 3 4

3

3

4

Given graph G Graph-G1

D

A

F

E

B

C



Graph-G2 Graph-G3 Graph-G4 Graph-G5 = T

Hence, the shortest spanning tree of G is G5 = T.

Now, (T) = 1 + 1 + 2 + 3 + 3 = 10.


6. Draw trees with 5 vertices.

7. Find all spanning trees of the graph.

8. Find all spanning trees of the following grpah.

9. The complete graph on „n‟ vertices Kn: has 2nn different spanning

trees. Give all the spanning trees of K4.

A A

B

E

1

F

1

2

D

C

3

3

B

1

F

1

2

D

C

3

E

A

B

1

1

2

D

C

F

E

A

1

1

D

C

F

E

B

o o o

o

o

o o o

o o



10. Find a spanning tree for each of the following graphs

11. Which connected simple graphs have exactly one spanning tree ?

12. Use Prim‟s Algorithm to find a minimal spanning tree for the following

weighted graph.

10.6 Summary

This unit dealt with a special type of graphs namely trees, which are

extensively used in search procedures and design of computer algorithms.

You got an idea to find the different spanning trees from a given connected

graph.

Different characterizations and properties of trees were given. Also, some

applications of binary trees were discussed.

This unit highlights that spanning trees play a vital role in multicasting over

internet protocol net works.

(i) o

o

o

o o

o o

o o

(ii)

o

o

o

o

(iii)

o

o

o o

o

(iv)

o

o

o o o

o

o o

o

8

A

3

6

4 7

2

4

6

8 6

C

B E

D

o

o

o

o

o




1. Draw two different binary trees with five vertices having maximum

number of leaves.

2. Draw a graph with the given specification. In case, if not possible, then

explain why no such graphs exist.

i) full binary tree, five internal vertices

ii) full binary tree, five internal vertices, seven terminal vertices

iii) full binary tree, twelve vertices

iv) full binary tree, nine vertices.

3. If T1 and T2 are two spanning trees in a graph, then show that

i) The number of edges in T1 not in T2 is equal to the number of

edges in T2 not in T1. (ii) d(T1, T2) = d(T2, T1).

4. Let N(g) = the number of edges in a graph g. Then N(Ti Tj) = the

number of edges in Ti Tj , and d(Ti , Tj) = The distance between Ti

and Tj. Show that d(T1, T2) = 2

1N(T1 T2).

5. Consider the table of airline distances in miles between six of largest cities

in the world: London, Mexico, New York, Paris, Peking and Tokyo.

L MC NY Pa Pe T

L - 5558 3469 214 5074 5959

MC - 2090 5725 7753 7035

NY - 3636 6844 6757

Pa - 5120 6053

Pe - 1307

T -

Write the spanning tree for the above

10.8 Answers


1. Tree

2. Minimally connected

3. Rooted tree

4. 2

1n

5. Spanning tree

L



6.

7. The given graph has four vertices and so each spanning tree must

have 4 – 1 = 3 edges. They are

8.

9. K4 has 244 = 16 different spanning trees. Each having three edges.

10. 100.

o

o

o

o

o

o o

o

o

o

o

o

o

o

o

o o

o

o

o

o o o o o o

o

o o

o

o

o

o

o o o o o

o o

o o o o o o

o

o o

o

o

o o o o o

o o o o o

o o o

o

(i) o

o o

o

o

o

o

o

o

(ii) o

o o o

(iii) o

o

o

o

o

(iv)

o

o o

o

o

o o

o

o



11. Tree.

12. Choose edges AE, AC, DC, AB.

Terminal Questions

1.

2. i.

ii. Not possible, since any full binary tree with five internal vertices have

six terminal vertices, but not seven.

iii. No, a full binary tree has 2k + 1 vertices where „k‟ is the number of

internal vertices but 2k + 1 is odd.

iv.

3. i) Let T1, T2 be two spanning trees. Now |E(T1)| = n - 1 = |E(T2)|.

Suppose T1 contains m edges which are not in T2 . Now the number

of edges in T2 not in T1 is

(n - 1) - [( n - 1) - m] = m. follows from (i).

b

c

a

e

f g

j k

o

o

o o

o

o

o

d

h i o o

o o

o

o

o

o

o

o

o

o o

o

o o

o o o

o

o o

o



4. N(T1 T2) = number of edges in T1 T2 = (number of edges in T1 but

not in T2) + (number of edges in T1 but not in T2) = d(T1, T2) + d(T2, T1)

= 2 d(T1, T2) d(T1, T2) = 2

1N(T1 T2)

5.

Fig. A Fig.B: Shortest spanning tree

6757

5074

1307

T

Pe

Pa

214

2090

3469

5074

1307

M

C

NY

T

L

Pa

Pe 5725

6053

7753

3469

5120 3636

5558

2090

595

9

MC

NY

7035

6844

214



Unit 11 Traversability Structure

11.1 Introduction

Objectives

11.2 Eulerian Graphs

11.3 Hamiltonian Circuits and Paths

11.4 Travelling Salesman problem

11.5 Summary


11.7 Answers

11.1 Introduction

In the previous unit we, studied Trees and their algorithms. Now, in this unit,

we will discuss Euler graphs, Hamiltonian paths and Hamiltonian circuits in

a graph G with special properties. A given graph G can be characterized

and studied in terms of the presence or absence of these sub-graphs. Many

physical problems can be represented by graphs and solved by observing

the relevant properties of the corresponding graphs.

Objectives


identify a Euler graph

use the characteristics of Euler graphs

recognize Hamiltonian paths and circuits in Graphs

use Travelling Sales man problem in Hamiltonian circuits .

11.2 Eulerian Graphs

Graph theory was born in 1736 with Euler’s famous paper. In the same

paper, Euler posed (and then solved) a more general problem.

We state the following results before giving the formal definitions of Euler’s

graphs.

A graph G is disconnected its vertex set V can be partitioned into two

non-empty disjoint subsets V1 and V2 such that there exists no edge in

G, whose one end-vertex is in V1 and the other end vertex is in V2.



If a graph (either connected or disconnected) has exactly two vertices of

odd degree, then, there exists a path joining these two vertices.

A simple graph with ‘n’ vertices and ‘k’ components can have at most (n

- k) (n -k + 1)/2 edges. [or the maximum number of edges in a simple

graph with ‘n’ vertices and ‘k’ components is (n - k)(n - k + 1)/2].

Definition: Let G be a graph. A closed walk running through every edge of

the graph G exactly once is called an Euler line. A graph ‘G’ that contains

Euler line is called an Euler graph.

Example: The graphs given in the Fig. 11.1A A and fig 11.1 B are Euler

graphs.

Fig. 11.1 A Fig. 11.1 B

i. Consider Fig-A, 123456789(10)(11)(12)13579(11)1 is an Euler line.

ii. Consider Fig-B, 23456789(10)12(11)357(11)8(10)2 is an Euler line.

A Euler graph may contain isolated vertices. If G is Euler graph and it

contains no isolated vertices, then it is connected. Hereafter, we consider

only those Euler graphs that do not contain isolated vertices. So, the Euler

graphs that we consider are connected.

Theorem: A given connected graph G is a Euler graph all the vertices of

G are of even degree.

Proof: Suppose G is a Euler graph. Then G contains a Euler line. So, there

exists a closed walk running through all the edges of G exactly once. Let v

V be a vertex of G. Now, in tracing the walk, it goes through two incident

edges on v with one entered v and the other exited.

8

1

2

3

5

7 9 10

11

12

4

6

10

4

8 6 9

3

5

1 2

11

7



Fig. 11.2

This is true not only for all the intermediate vertices of the walk, but also true

for the terminal vertex because we had the exit and entered at the same

vertex at the beginning and ending of the walk. Therefore, if v occurs k times

in the Euler line, then d(v) = 2k. Thus, if G is a Euler graph, then the

degree of each vertex is even.

Converse: Suppose all the vertices of G are of even degree. Now, to show

that G is a Euler graph, we have to construct a closed walk starting at an

arbitrary vertex v and running through all the edges of G, exactly once. To

find a closed walk, let us start from the vertex v. Since every vertex is of

even degree, we can exist from every vertex we entered, the tracing cannot

stop at any vertex but at ‘v’. Since ‘v’ is also of even degree, we shall

eventually reach ‘v’ when the tracing comes to an end. If this closed walk

(h, say) includes all the edges of G, then G is an Euler graph. Suppose the

closed walk h does not include all the edges. Then, the remaining edges

form a subgraph h1 of G. Since both G and h have all their vertices of even

degree, the degrees of the vertices of h1 are also even. Moreover, h1 must

touch h at least one vertex ‘a’ because G is connected. Starting from ‘a’ we

can again construct a new walk in graph h1. Since all the vertices of h1 are

of even degree, and this walk in h1 must terminates at the vertex ‘a’. This

walk in h1 combined with h forms a new walk which starts and ends at vertex

v and has more edges than those are in ‘h’.

We repeat this process until we obtain a closed walk that travels through all

the edges of G. In this way, one can get an Euler line. Thus, G is a Euler

Graph.

Konigsberg Bridges Problem

In the graph of the Konigsberg bridges problem, there exist vertices of odd

degree. So all the vertices are not of even degree. Hence by the above

Theorem, we conclude that "the graph representing the Konigsberg bridges

v



problem" is not a Euler graph. So we conclude that it is not possible to walk

over each of the seven bridges exactly once and return to the starting point.

Fig. 11.3: Konigsberg bridge

Theorem: A connected graph G is a Euler graph if and only if it can be

decomposed into circuits.

Proof: Suppose graph G can be decomposed into circuits; that is, G is a

union of edge-disjoint circuits. Since the degree of every vertex in a circuit

is two the degree of every vertex in G is even. Hence G is an Euler graph.

Conversely, let G be an Euler graph. Consider a vertex v1. There are atleast

two edges incident at v1. Let one of these edges be between v1 and v2.

Since vertex v2 is also of even degree, it must have at least another edge,

say, between v2 and v3. Proceeding in this fashion, we eventually arrive at a

vertex that has previously been traversed, thus forming a circuit . Let us

remove from G. All vertices in the remaining graph (not necessarily

connected) must also be of even degree. From the remaining graph remove

another circuit in exactly the same way as we removed from G. Continue

this process until no edges are left. Hence the theorem.

Arbitrarily Traceable Graphs:

Consider the graph in Fig. 11.4, which is a Euler graph. Suppose that we

start from vertex ‘a’ and trace the path a b c.

A

B

D C



Fig. 11.4: Arbitrarily traceable graph from c

Now at c we have the choice of going to a, d or e. If we took the first choice,

we would only trace the circuit a b c a, which is not an Euler line. Thus,

starting from a, we cannot trace the entire Euler line simply by moving along

any edge that has not already been traversed.

Such a graph is called an arbitrarily traceable graph from vertex v. For

instance, the Euler graph in Fig. 11.4 is an arbitrarily traceable graph from

vertex C, but not from any other vertex.

A Euler graph G is arbitrarily traceable from vertex v in G if and only if

every circuit in G contains v.


1. A simple graph with ‘n’ vertices and ‘k’ components can have at most

–––––––––– edges.

2. A connected graph G is an Euler graph if and only if it can be

decomposed into –––––––––––––.

11.3 Hamiltonian Circuit and Path

A Euler line of a connected graph was characterized by the property of

being a closed walk that traverses every edge of the graph (exactly once).

Hamiltonian Circuit: A Hamiltonian circuit in a connected graph is defined

as a closed walk that traverses every vertex of G exactly once, except of

course, the starting vertex, at which the walk also terminates.

A circuit in a connected graph G is said to be Hamiltonian as it includes

every vertex of G. Hence a Hamiltonian circuit in a graph of n vertices

consists of exactly n edges. Obviously, not every connected graph has a

Hamiltonian circuit.

a b

c

d e



The resemblance between the problem of as Euler line and that of a

Hamiltonian circuit is deceptive. The latter is infinitely more complex.

Although one can find Hamiltonian circuits in many specific graphs.

Hamiltonian Path: If we remove any one edge from a Hamiltonian circuit,

we are left with a path. This path is called a Hamiltonian path. Clearly, a

Hamiltonian path in a graph G traverses every vertex of G. Since a

Hamiltonian path is a sub-graph of a Hamiltonian circuit (which in turn is a

subgraph of another graph) every graph that has a Hamiltonian circuit also

has a Hamiltonian path. There are many graph with Hamiltonian paths that

have no Hamiltonian circuits. The length of a Hamiltonian path (if it exists) in

a connected graph of n vertices is n–1.

In considering the existence of a Hamiltonian circuit (or path), we need only

consider simple graphs. This is because a Hamiltonian circuit (or path)

traverses every vertex exactly once. Hence it cannot include a self-loop or a

set of parallel edges. Thus a general graph may be made simple by removing

parallel edges and self-loops before looking for a Hamiltonian circuit in it.

A given graph may contain more than one Hamiltonian circuit. Of interest

are all the edge-disjoint Hamiltonian circuits in a graph. The determination

of the exact number of edge-disjoint Hamiltonian circuits (or paths) in a

graph in general is also an unsolved problem. However, the number of

edge-disjoint Hamiltonian circuits in a complete graph with odd number of

vertices is an unsolved problem.


3. A circuit in a connected graph G is said to be ––––––––– it includes

every vertex of G.

4. A Hamiltonian path in a graph G –––––––– every vertex of G.


A problem closely related to the question of Hamiltonian circuits is the

travelling-salesman problem, stated as follows: A salesman is required to

visit a number of cities during a trip. Given the distances between the cities,

in what order should he travel so as to visit every city precisely once and

return home, with the minimum mileage travelled?



Representing the cities by vertices and the road between them by edges,

we get a graph. In this graph, with every edge ei there is associated a real

number (the distances in miles, say), w(ei). Such a graph is called weighted

graph; w(ei) being the weight of edge eI .

In our problem, if each of the cities has a road to every other city, we have a

complete weighted graph. This graph has numerous Hamiltonian circuits, and

we are to pick the one that has the smallest sum of distances (or weights).

The total number of different Hamiltonian circuits in a complete graph of n

vertices can be shown to be 2

)!1n(. This follows from the fact that starting

from any vertex we have n–1 edges to choose from the first vertex, n–2 from

the second, n–3 from the third and so on. These being independent choices,

we get (n–1)! possible number of choices. This number is, however, divided

by 2, because each Hamiltonian circuit has been counted twice.

Theoretically, the problem of the travelling salesman can always be solved

by enumerating all (n–1)!/2 Hamiltonian circuits, calculating the distance

traveled in each, and then picking the shortest one. However, for a large

value of n, the labor involved is too great even for a digital computer.


5. The problem of the travelling salesman can always be solved by

enumerating all –––––––– Hamiltonian circuits

6. State True or False.

a) A given connected graph G is an Euler graph if and only if all

vertices of G are of odd degree.

b) A connected graph G is an Euler graph if and only if it can be


c) If we remove any one vertex from Hamiltonian circuit, we are left with

a path. This path is called Hamiltonian path.

11.5 Summary

If some closed walk in a graph contains all the edges of the graph, then

the walk is called an Euler line and the graph an Euler graph.

A connected graph G is a Euler graph if and only if it can be




A Hamiltonian circuit in a connected graphs is defined in a closed walk

that traverses every vertex of G exactly once.


1. Prove that a connected graph G is a Euler graph if and only if it can be

decomposed into circuits

2. Write short notes on Travelling Salesman Problem

3. Prove that a given connected graph G is a Euler graph if and only if all

the vertices of G are of even degree

11.7 Answers


1. (n - k) (n -k + 1)/2

2. Circuits

3. Hamiltonian

4. Traverses

5. (n–1)!/2

6. a) False

b) True

c) True

Terminal Questions

1. Suppose graph G can be decomposed into circuits; that is, G is a union

of edge-disjoint circuits. Since the degree of every vertex in a circuit is

two, the degree of every vertex in G is even. Hence, G is an Euler

graph.

Conversely, let G be a Euler graph. Consider a vertex v1. There are

atleast two edges incident at v1. Let one of these edges be between v1

and v2. Since vertex v2 is also of even degree, it must have at least

another edge, say between v2 and v3. Proceeding in this fashion, we

eventually arrive at a vertex that has previously been traversed, thus

forming a circuit . Let us remove from G. All vertices in the remaining

graph (not necessarily connected) must also be of even degree. From

the remaining graph remove another circuit in exactly the same way as

we removed from G. Continue this process until no edges are left.

Hence the theorem.



2. Representing the cities by vertices and the road between them by

edges, we get a graph. In this graph, with every edge ei there is

associated a real number (the distances in miles, say), w(ei). Such a

graph is called weighted graph; w(ei) being the weight of edge eI .

In our problem, if each of the cities has a road to every other city, we have a

complete weighted graph. This graph has numerous Hamiltonian circuits,

and we are to pick the one that has the smallest sum of distances (or

weights).

The total number of different Hamiltonian circuits in a complete graph of

n vertices can be shown to be 2

)!1n(. This follows from the fact that

starting from any vertex we have n–1 edges to choose from the first

vertex, n–2 from the second, n–3 from the third and so on. These being

independent choices, we get (n–1)! possible number of choices. This

number is, however, divided by 2, because each Hamiltonian circuit has

been counted twice.

3. Suppose, G is a Euler graph. Then, G contains an Euler line. So there

exists a closed walk running through all the edges of G exactly once. Let

v V be a vertex of G. Now in tracing the walk it goes through two

incident edges on v with one entered v and the other exited.

This is true not only for all the intermediate vertices of the walk, but also

true for the terminal vertex because we exited and entered at the same

vertex at the beginning and ending of the walk. Therefore if v occurs k

times in the Euler line, then d(v) = 2k. Thus if G is an Euler graph, then

the degree of each vertex is even.

Converse: Suppose all the vertices of G are of even degree. Now, to

show that G is a Euler graph, we have to construct a closed walk

starting at an arbitrary vertex v and running through all the edges of G,

exactly once. To find a closed walk, let us start from the vertex v. Since

every vertex is of even degree, we can exist from every vertex we

entered, the tracing cannot stop at any vertex but at ‘v’. Since ‘v’ is also

of even degree, we shall eventually reach ‘v’ when the tracing comes to

an end. If this closed walk (h, say) includes all the edges of G, then G

is a Euler graph. Suppose, the closed walk h does not include all the

edges. Then the remaining edges form a subgraph h1 of G. Since both



G and h have all their vertices of even degree, the degrees of the

vertices of h1 are also even. Moreover, h1 must touch h at least one

vertex ‘a’ because G is connected. Starting from ‘a’ we can again

construct a new walk in graph h1. Since all the vertices of h1 are of even

degree, and this walk in h1 must terminates at the vertex ‘a’. This walk in

h1 combined with h forms a new walk which starts and ends at vertex v

and has more edges than those are in ‘h’.

We repeat this process until we obtain a closed walk that travels through

all the edges of G. In this way, one can get a Euler line. Thus, G is a

Euler Graph.



Unit 12 Representations of Graphs Structure

12.1 Introduction

Objectives

12.2 Adjacency and Incidence Matrix

12.3 Circuit Matrix

12.4 Path Matrix

12.5 Summary


12.7 Answers

12.1 Introduction

Even though, a pictorial representation of a graph is very convenient for a

visual study, other representations are better for computer processing and

constructing graphical algorithms. Many results of matrix algebra can be

applied to study the structural properties of graphs. In this unit, we will study

the adjacency, incidence, circuit, and path matrices with illustrations. We

will observe properties of each of these matrices. We will also discuss

correspondence between some graph theoretic operations (properties) and

matrix properties.

Objectives


write the matrix representations of a given graph

give different matrix representations of graphs

explain the algebraic properties of matrices of graphs

apply the properties to switching networks

12.2 Adjacency and Incidence Matrix

Definition: Let G be a graph with n vertices v1, v2, …, vn and no self-loops

and no parallel edges. We define an n n adjacency matrix A = [ ija ]n n, as

follows:

aij = 1 if vi is adjacent to vj;

= 0 other wise.



Fig. 12.1

6x6001000

001011

110010

000010

011101

010010

v

v

v

v

v

v

vvvvvv

6

5

4

3

2

1

654321

Matrix

Observations:

i) The entries along the principal diagonal of the adjacency matrix are all

0’s (since it has no self loops).

ii) The degree of a vertex equals the number of 1’s in the corresponding

row (or a column).

iii) Given any square, symmetric, binary matrix X of order n, one can

always construct a graph G of n vertices (and no parallel edges) such

that X is the adjacency matrix of G.

Definition: Let G be a graph with n vertices, e edges and no self-loops. We

define an n e matrix I = [ ija ]n e, whose n rows correspond to the n

vertices, and the e columns correspond to the e edges, as follows:

aij = 1 if jth edge ej is incident on ith vertex vi;

= 0 other wise.

v1

v2

v3

v4

v5

v6

d

f g

a

c

e



This matrix I is said to be the vertex-edge incidence matrix (or) simply

incidence matrix. This matrix I (for the graph G) may be denoted as I(G).

Example

Consider the graph G given in Fig. 12.2

Fig. 12.2

The incidence matrix I(G) is an n e = 6 8 matrix as given below:

I(G) =

8x600000011

01001100

00010111

10000000

11110000

00101000

v

v

v

v

v

v

hgfedcba

6

5

4

3

2

1

Matrix

A matrix whose entries are only either 0 or 1, is called a binary matrix (or)

(0, 1)-matrix. We can treat 0 and 1 as two elements from GF(2), the Galois

field modulo 2.

Observations:

The following are some observations on the incidence matrix I of a graph G.

i) Each column of I contains exactly two 1’s, since every edge is incident

on exactly two vertices

ii) The number of 1’s in each row is equal to the degree of the

corresponding vertex.

iii) A row contains only 0’s, denote an isolated vertex.

iv) If two edges a, b are parallel edges (in the graph G), then the

corresponding columns (in the incidence matrix I) are identical.

v1

v2

v3

v4

v5

v6

d

f g

a

b

c

e



v) If the given graph G is disconnected and consists of two components

g1 and g2, then (we can reorder the list of edges and the list of vertices,

if necessary) then the incidence matrix I(G) of G can be written in a

block-diagonal form as given below:

I(G) = 1

2

I(g ) 0

0 I(g ) where, I(g1) and I(g2) are the incidence matrices

of the components g1 and g2 respectively. (This observation follows

from the fact that no edge in g1 is incident on a vertex of g2, and vice

versa).

[For this, suppose v1, v2, …, vm are vertices and e1, e2, …, en are edges of

G; the subgraphs g1, g2 are two components of G; the vertices v1, v2, …,

vs are the vertices in g1; the vertices vs+1, ..., vm are vertices in g2; the

edges e1, e2, …, ek are in g1; the edges ek+1, …, en are in g2. If we take

vertices and edges in this order, then we get that I(G) = 1

2

I(g ) 0

0 I(g )]

Theorem: Two graphs G1 and G2 are isomorphic if and only if their

incidence matrices I(G1) and I(G2) differ only by permutations of rows and

columns.

Note: If I(G) is an incidence matrix of a connected graph G with n vertices,

then the rank of I(G) is n -1. [Note that the entries 0, 1 are elements from

ℤ2 or the elements of GF(2)].

Note:

i) If we extend the content of the above note for a disconnected graph G

with k components, then the rank of I(G) is n - k. This number n - k is

called as the rank of the graph G with k components.

ii) If we remove any one row from the incidence matrix of a connected

graph, then the remaining (n - 1) e submatrix is of rank n - 1 .

(That is, the remaining n - 1 vectors are linearly independent).

iii) Such (n - 1) e submatrix of I is called a Reduced incidence matrix

and it is denoted by Af. The vertex corresponding to the deleted row

(to get Af) is called the reference vertex.

iv) Any vertex of a connected graph can be made the reference vertex.

Note: The reduced incidence matrix of a tree is non-singular.



Example: In a disconnected graph, there exists vertices which cannot be

made as the reference vertices. To observe this, consider the graph G given

in Fig. 12.3. This graph is a disconnected graph with two components

(that is, k = 2), n = 5 vertices and n - 1 = 4 edges.

Fig. 12.3

Now the incidence matrix is given by

I(G) =

1001

1100

0110

0011

0000

v

v

v

v

v

dcba

5

4

3

2

1

If =

1001

1100

0110

0011

[after removing the 1st row].

If we add the rows, we get 0 (mod 2). So this matrix is a singular matrix (that

is, det (If) = 0). Note that this is not a reduced matrix.

Definition: Let g be a subgraph of a graph G. Suppose I(g) and I(G) are

the incidence matrices of g and G respectively. Then I(g) is called a

submatrix of I(G). (So permutation of rows (or) columns are permitted).

Note: There is a one-to-one correspondence between each n k submatrix

of I(G) and a subgraph of G with k edges, where k being any positive

integer less than e, and n being the number of vertices in G.

Note: Let I(G) be an incidence matrix of a connected graph G with n

vertices.

v1

v2

v3 v4

v5

c

b d

a



An (n - 1) (n - 1) submatrix of I(G) is non singular if and only if the n - 1

edges corresponding to the n - 1 columns of this matrix constitute a

spanning tree in G.

Definition: Each largest non-separable subgraph is called a block.

A separable graph consists of two or more non-separable subgraphs.

Example: Consider the graph given in Fig. 12. 4, which contains two blocks.

Fig. 12.4


1. The entries along the ––––––––– of the adjacency matrix are all 0’s

2. The number of 1’s in each row is equal to the degree of the

corresponding ––––––––––––– .

3. The reduced incidence matrix of a tree is ––––––––––––– .

4. Each largest non-separable subgraph is called a –––––––––––––.

12.3 Circuit Matrix

Definition: Let the number of edges in G be e and the number of circuits in

G be q. Then, a circuit matrix

B = ij q eb , (0, 1)-matrix defined as follows:

bij = 1 if the ith circuit includes the jth edge;

Block-1 Block-2



= 0 otherwise.

The circuit B of G may be denoted by B(G).

Example: Consider the graph given in Fig. 12.5

Fig. 12.5

For this graph, n = number of vertices = 6

e = number of edges = 8

Here, the graph has 4 different circuits:

{a, b}, {c, d, f, e}, {d, f, g}, and {c, e, g}.

Therefore, the circuit matrix B(G) is a 4 8, (0, 1)-matrix as given below.

B(G) =

8x400111100

01101000

01010100

00000011

4

3

2

1

hgfedcba

Matrix

Observations:

i) If a column of B(G) contains all zeros, then the related edge does not

belong to any circuit. An edge that does not lie in any circuit is said to

be a non-circuit edge.

ii) Each row of B(G) corresponds to a circuit. So each row may be called

as a circuit vector.

iii) If a row contains only one "1", then the related edge is a self-loop.

iv) The number of 1’s in the ith-row is equal to the number of the edges in

the ith circuit.

h

v1

v2

v3

v4

v5

v6

d

f g

a

b

c

e



v) If a graph G is separable (or disconnected) and consists of two blocks

(or components) g1 and g2, then the circuit matrix B(G) can be written

in a block-diagonal form as B(G) = )(0

0)(

2

1

gB

gB where, B(g1)

and B(g2) are the circuit matrices of g1 and g2 respectively.

Theorem: Let I and B be the incidence matrix and circuit matrix of a (self-

loop free) graph whose columns are arranged with the same ordering of

edges. Then every row of B is orthogonal to every row of I. That is, IBT = BIT

= 0 (mod 2) where IT, BT are the transpose matrices of I and B respectively.


Fig. 12.6

The incidence matrix I(G) is given by

I (G) = I =

8x600000011

01001100

00010111

10000000

11110000

00101000

v

v

v

v

v

v

hgfedcba

6

5

4

3

2

1

h

v1

v2

v3

v4

v5

v6

d

f g

a

b

c

e



B(G) = B =

8x400111100

01101000

01010100

00000011

4c

3c

2c

1c

hgfedcba

The list of all circuits (of the graph) used in getting B(G) is given below: c-1

= {a, b}, c-2 = {c, e, g}, c-3 = {d, f, g}, c-4 = {c, d, f, e}

Now, BT (transpose of B) =

480000

0110

1100

1010

1100

1010

0001

0001

Therefore, IBT =

00000011

01001100

00010111

10000000

11110000

00101000

0000

0110

1100

1010

1100

1010

0001

0001

=

0000

0000

0000

0000

0000

0000

= 0 (mod

2)

Note:

Let G be a connected graph with n vertices, e edges and T a spanning tree.

Then, T contains (n - 1) edges. The remaining e - (n - 1) = e - n + 1 edges

are chords with respect to T. By joining a chord, we get a fundamental

circuit. So, there are e-n+1 fundamental circuits with respect to T.



Fundamental circuit matrix:

A submatrix (of a circuit matrix) in which all rows corresponds to a set of

fundamental circuits, is denoted by Bf.

Example:

Consider the graph G and its spanning tree T given in Figure 12.7A and

12.7B respectively. The fundamental circuit matrix with respect to the

spanning tree T is given below

Fig. 12.7 A Graph G Fig. 12.7 B Spanning tree T

Bf =

1100100

1000010

1011001

Matrix

Here, 1, 2, 3 are the circuits corresponds to the chords e2 , e3, e6 respectively.


5. An edge that does not lie in any circuit is said to be a –––––––––––––.

12.4 Path matrix

This matrix is used in communications and transport net works.

This is also a (0,1) matrix, which is defined for a specific pair of vertices in a

graph, (x, y), and it is denoted by P(x, y). The rows in P(x, y) correspond to

different paths between vertices x and y, and the columns correspond to the

edges in G.

e4

e1

e2

e3 e5

e6

e7

e1

e4

e5 e7

Chrods

1

2

3

e2 e3 e6 e1 e4 e5 e6

chords



Definition: P(x, y) = (Pij)p q, where, p = number of paths between x and y

and

q = total number of edges.

th th

ij

1, if j edge lies in i path, andP =

0, otherwise

Observations:

i) A column of all 0’s corresponds to an edge that does not lie in any path

between x and y.

ii) A column of all 1’s corresponds to an edge that lies in every path

between x and y.

iii) There is no row with all 0’s.

Theorem:

If the edges of a connected graph are arranged in the same order for the

columns of the incidence matrix I and the path matrix P(x, y), then the

product (mod 2)

I PT(x, y) = M

Where, the matrix M has 1’s in two rows x and y, and the rest of the n-2

rows are all 0’s.


6. The matrix used in communications and transport net works is –––––– .

12.5 Summary

In this unit, we discussed representation of a graph in several matrix

forms.

A matrix representation is a convenient and useful way of representing a

graph to a computer.

The algebra of matrices can be used to identify certain properties of

graphs.




1. Draw a graph for each of the following adjacency.

i) ii) iii)

0101

1011

0100

1100

0110

1001

1001

0110

010

101

010

2. Write adjacent matrix for each of the following graphs.

i) 3. Write adjacency matrix to represent the graphs shown below. i) 4. Draw the digraph corresponding to the following adjacency matrix.

0011

1011

0100

1100

v

2

v

4

v

3

v

5

v

1

o o

o o o

iii) v

2

v

5

v

3

v

4

v

1 o

o o

o

o

ii)

v

1 v

2

v

3

v

4

o

o o

o

o o

o

ii)

o

o o

o

iii)

o

o o

o



5. Verify whether or not the following graphs isomorphic.

6. Find the incidence matrices of the following graphs.

12.7 Answers


1. Principal Diagonal

2. Vertex

3. Non-Singular

4. Block

5. Non-Circuit Edge

6. Path Matrix

G: v5

v2

v3

v4

v1

o o o

o

o 1

1v 1

3v

1

4v

1

2v

1

5v

o

o

o

o o

G1:

e2

v1

e1

e5

e3

e4

v2 v3

v4

o

o

o

o

i) (ii)

e1

e2

e4

e7 e6 e3

e5

e8 d f

e a

b

c

o o

o

o o

o



Terminal Questions

1. 2.

i)

01000

10110

01000

01001

00110

ii)

1210

2012

1103

0230

iii)

00110

00111

11000

11001

01010

3.

i)

0101

1011

1101

1110

ii)

020

211

010

iii)

0001

1000

0100

0110

4.

5. Each of the graphs G and G1 have the adjacency matrix:

I(G) = I(G1) =

01110

10101

11010

10101

01010

Therefore, G and G1 are isomorphic.

o

o

o

o

i)

o

o

o

o ii)

o

o o

o

iii) o

o o



6.

i)

11010

01100

00111

10001

e1 e 2 e3 e4 e5

v1

v2

v3

v4

ii)

00000011

01001100

00010111

10000000

11110000

00101000

a

b

c

d

e

f

e1 e2 e3 e4 e5 e6 e7 e8



Unit 13 Directed Graphs Structure

13.1 Introduction

Objectives

13.2 Types of Directed Graphs

13.3 Binary Relation As a Digraph

13.4 Euler’s Digraphs

13.5 Matrix Representation of Digraphs

13.6 Summary


13.8 Answers

13.1 Introduction

The graphs so far studied are undirected graphs (no direction was assigned to

the edge in a graph). In this unit, we shall consider directed graphs (graphs in

which edges have directions). Many physical situations require directed graphs.

The applications of directed graphs are in the street map of city with one-way

streets, flow networks with valves in the pipes, and electrical networks.

Directed graphs in the form of signal flow graphs are used for system analysis

in control theory. Most of the concepts and terminology of undirected graphs

are also applicable to directed graphs. In this unit we will discuss some of the

properties of directed graphs which are not shared by undirected graphs.

Objectives


write the indegree and outdegree of a vertex

explain the tournaments and Euler’s digraphs

give the different matrix representations of digraphs

apply the properties to flows, network and traffic problem.

13.2 Types of Directed Graphs

The different types of directed graphs are discussed below:

Definition: A directed graph (or) a digraph D consists of a non-empty set V

(the elements of V are normally denoted by v1, v2, …) and a set E (the



elements of E are normally denoted by e1, e2, ….) and a mapping that

maps every element of E onto an ordered pair (vi, vj) of elements from V.

The elements of V are called as vertices or nodes or points. The elements

of E are called as edges or arcs or lines. If e E and vi, vj V such that

(e) = (vi, vj), then we write e = jivv . In this case, we say that e is an edge

between vi and vj. (We also say that e is an edge from vi to vj). (We also say

that e originates at vi and terminates at vj). (An edge from vi to vj is denoted

by a line segment with an arrow directed from vi to vj).

Note:

i) A directed graph is also refered to as an oriented graph.

ii) Let D be a directed graph and e = vu .

Then, we say that e is incident out of the vertex v, and incident into the

vertex u. In this case, we say that the vertex v is called the initial vertex and

the vertex u is called the terminal vertex of e.

Example: The graph given in Figure-13.1 is a digraph with 5 vertices and

ten edges.

Fig .13.1

Here, v5 is the initial vertex and v4 is the terminal vertex for the edge e7. The

edge e5 is a self-loop.

Definition:

i) The number of edges incident out of a vertex v is called the out degree

(or) out-valence of v. The out degree of a vertex v is denoted by d+(v).

ii) The number of edges incident into v is called the indegree (or) in-

e5

e9

v2

e10

e7

v1

e8

e6

v4

v3

v5

e1 e4

e3 e2



valance of v. The indegree of a vertex v is denoted by d-(v). Note that

the degree of v is equal to the sum of indegree and out degree of v, for

any vertex v in a graph. (In symbols, we can write as d(v) = d+(v) + d-(v)

for all vertices v).


i) Here, d+(v1) = 3, d+(v2) = 1, d+(v3) = 1, d+(v4) = 1, d+(v5) = 4. d - (v1) = 1,

d - (v2) = 2, d - (v3) = 4, d -(v4) = 3, d - (v5) = 0.

ii) d(v1) = 4 = 3 + 1 = d+(v1) + d-(v1) ; d(v2) = 3 = 1 + 2 = d+(v2) + d-(v2),

and so on.

Problem: Let D be a directed graph. Show that the sum of all in-degrees of

the vertices of D, is equal to the sum of all out-degrees of the vertices of D;

and each sum is equal to the number of edges in D.

[In other words,

n

1i

i )v(d =

n

1i

i )v(d = q where q is the number of

edges in D, and {v1, v2, …, vn} is the set of all vertices in D].

Solution: Let D be any digraph with n vertices and q edges. When the

indegrees of vertices are counted, each edge is counted exactly once.

[This is because every edge goes to exactly one vertex].

So

n

1i

i )v(d = q ... (i)

Similarly, when the out-degrees of vertices are counted, each edge is

counted exactly once. [This is, because each edge goes out of exactly one

vertex].

So

n

1i

i )v(d = q …. (ii).

From (i) and (ii), we get that

n

1i

i )v(d =

n

1i

i )v(d = q.

Example: Consider the digraph in Fig. 13.1 Here, the number of vertices

n = 5, and the number of edges q = 10.



Now,

n

1i

i )v(d = d+(v1) + d+(v2) + d+(v3) + d+(v4) + d+(v5) = 3 + 1 + 1 + 1

+ 4 = 10 = q

Also,

n

1i

i )v(d = d -(v1) + d -(v2) + d -(v3) + d -(v4) + d -(v5) = 1 + 2 + 4

+ 3 + 0 = 10 = q.

Therefore,

n

1i

i )v(d =

n

1i

i )v(d = q = the number of edges.

Definitions:

i) A vertex v is said to be an isolated vertex if the out degree of v and the

indegree of v are equal to zero. (In symbols, d+(v) = 0 = d-(v)).

ii) A vertex v in a digraph D is said to be a pendent vertex if it is of degree

1. (In other words, a vertex is said to be a pendent vertex if the degree

of v = d+(v) + d -(v) = 1).

iii) Two directed edges are said to be parallel edges if they are mapped

onto the same ordered pair of vertices.

iv) In other words, two directed edges e and f are said to be parallel edges

if both e and f originates from the same vertex, and also terminates at

the same vertex. (In the graph given in Fig. 13.1, the edges e8, e9, and

e10 are parallel edges. The edges e2 and e3 are not parallel).

Note:

i) Let D be directed graph. If we disregard the orientation (that is,

direction) of every edge in D, then we get an undirected graph. The

undirected graph obtained (in this way) from D is called the undirected

graph corresponding to D.

ii) Suppose, H is an undirected graph. To each edge of H we can assign a

direction. Then the digraph obtained is called an orientation of H. (or a

directed graph associated with H).

Definition: Let D1 and D2 be two digraphs. These two graphs are said to be

isomorphic if

i) the corresponding undirected graphs are isomorphic, and

ii) the directions of corresponding edges must be same.



Example:

i) Consider the directed graphs D1 and D2. The two digraphs D1 and D2

are not isomorphic (because the direction of the edge e4 in D1 is

different from the direction of the edge e4 in D2).

Fig. 13.2: Graph D1 Graph D2

ii) Note that the undirected graphs corresponding to D1 and D2 are

isomorphic.

Definition: A digraph that has neither self-loops nor parallel edges is called

a simple digraph.

In other words, a directed graph D is said to be a simple digraph if the

corresponding undirected graph is a simple graph. The two digraphs given

in Fig. 13.2 are simple directed graphs.

Definitions:

i) Digraphs that have at most one directed edge between any pair of

vertices, but are allowed to have self-loops are called the asymmetric

(or) anti-symmetric digraphs.

ii) A digraph D is said to be a symmetric digraph if for every edge (a, b) in

D there is also an edge (b, a) in D.

iii) A digraph that is both simple and symmetric is called a simple

symmetric digraph.

iv) A digraph that is both simple and asymmetric is called simple

asymmetric digraph.

v) A simple digraph is said to be a complete symmetric digraph if it

satisfies the following condition: "Given any two vertices v1 and v2,

there corresponds exactly one edge directed from v1 to v2".

vi) A complete asymmetric digraph (or) tournament (or) a complete

tournament. is an asymmetric digraph in which there is exactly one

edge between every pair of vertices.

e1 e4

e5

e2

e3

e6

e1 e4

e5

e2

e3

e6



Example: In the following,

i) The graph G1 is not asymmetric since there exist two directed edges

between v1 and v2.

ii) The graph G2 is asymmetric.

iii) Any null graph is asymmetric.

Fig. 13.3 G1 G2 G3

iv) The digraph G3 is a complete symmetric digraph.

Observation:

i) A tournament (means, complete asymmetric digraph) of n vertices

contains 2

1nn edges.

The directed graph given in fig. 13.2 is a tournament. In this graph,

the number of vertices is n = 4.

The number edges is 6 = 2

144 =

2

1nn.

ii) A complete symmetric graph of n vertices contains n(n - 1) edges. The

digraph given in fig. 13.3, G3 is a complete symmetric graph on

n = 4 vertices.

In this graph, the number of edges is 12 = 4(4 -1) = n (n - 1).

Definition: A digraph D is said to be balanced if the in-degree of v is equal to

the out-degree of v for every vertex v in D. In other words, a digraph D is said to

be balanced if d+(v) = d-(v) for all vertices v in the digraph D. A balanced

digraph is also known as pseudo symmetric digraph (or) an isograph.

v2 v3

v1



A balanced digraph is said to be a regular digraph if it satisfies the following

two conditions:

i) The in-degrees of all the vertices of D are equal; and

ii) The out-degrees of all the vertices of D are equal.


1. A directed graph is also refered to as an –––––––––––– .

2. The number of edges incident out of a vertex v is called the––––––––––.

3. A vertex v is said to be an isolated vertex if the out degree of v and the

indegree of v are equal to––––––––––––––.

13.3 Binary Relation As a Digraph

Let X be a set. Represent the elements of X by the symbols x1, x2, …

Suppose that R is a relation on X. We represent the elements of X as

vertices and draw a directed edge from xi to xj if (xi, xj) R. Then, we get a

directed graph which represents the given relation R on X.

Example: Consider the set X = {3, 4, 5, 7, 8} and the relation (R, >) on X.

Then, R = {(8, 3), (8, 4), (8, 5), (8, 7), (7, 3), (7, 4), (7, 5), (5, 4), (4, 3)}

= {(x, y) / x > y, x, y X}.

The graph D given below represents this relation R on X.

Fig 13.4

It is clear that, every binary relation on a finite set can be represented by a

directed graph without parallel edges. Conversely, if a directed graph D

without parallel edges was given, then there corresponds a binary relation

on the set of vertices.

3

4

8

5 7



Definition:

i) Let X be a set, and R a relation on X (that is, R X X). If (x, y) R,

then we also write xRy. The relation R is said to be reflexive relation if

xRx for all x X.

(Note that the digraphs of a reflexive relation have a self-loop at every

vertex).

We call a directed graph representing a reflexive binary relation on its

vertex set as reflexive digraph”.

A digraph in which no vertex has a self-loop is called an irreflexive

digraph.

ii) Let X be a set, and R a relation on X. We say that R is symmetric if a,

b X, aRb bRa. Note that a directed graph representing a

symmetric relation is a symmetric digraph.

Example: The graph given in G1 is the digraph of a relation which is

symmetric but not reflexive. This relation is on the set {x1, x2, x3, x4}. Some

authors represent this relation by the undirected graph given in G2. Note that

in the graph G2, only one undirected edge is taken between a pair of

vertices (that are related under the relation R).

Fig. 13.5: Graph G1 Graph G2

Definition: Let X be a set, and R a relation on X. The relation R is said to be

transitive if a, b, c R, aRb, bRc aRc. A digraph representing a transitive

relation is called a transitive digraph. [Observe that the graph given in 13.4

is a transitive digraph].

Let X be a set, and R a relation on X.

i) The relation R is said to be an equivalence relation if it is reflexive,

symmetric and transitive.

x1 x2

x3 x4

x1 x2

x4 x3



ii) A digraph representing an equivalence relation is called an equivalence

digraph.

Example: Consider the binary relation R (= “Congruent modulo 3”) defined

on the set X = {10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20}.

We can observe that this relation R on X is an equivalence relation. The

related equivalence graph was given. We are using undirected graph here.

Observe that the set of vertices is divided into three disjoint equivalence

classes. Each of these sets form separate components. Each component is

an undirected subgraph (due to symmetry, we are using undirected graph

with a self-loop at each vertex). Also, note that (since any two elements in

an equivalence class are related) any two vertices inside the component

were joined by an edge.

Definition: Let X be a set, and R is a binary relation on X. This relation R on X

may be represented by a matrix. This matrix is called as relation matrix. It is a

(0,1)-n n-matrix, where n is the number of elements in the set X = {x1, x2, …,

xn}.

The relation matrix ( ija ) is defined as follows:

ija = 1 if xiRxj,

= 0 otherwise.

11 14

17 20

)3(mod2

10 13

16 19

)3(mod1

15

12

18

)3(mod0



Example: The relation matrix of the relation R (" usual grater than") on the

set

X = {3, 4, 5, 7, 8}, is the Matrix

01111

00011

01011

00001

00000

8

5

7

4

3

85743

Definitions:

i) A path v0e1v1e2v2 … envn is said to be a directed path if ek is oriented

from vk-1 to vk for each 1 k n. A path which is not a directed path

is called a semi-path. In a directed graph, the word "path" refers to

either a directed path or a semi-path.

ii) Let D be a directed graph. A directed walk (in D) from a vertex v to u is

an alternating sequence of vertices and edges beginning with v and

ending with u such that each one of the edges is oriented (or directed)

from the vertex preceding it to the vertex following it. [In other words, a

walk v0e1v1e2v2 … envn in the undirected graph D is said to be a

directed walk if ek is oriented from vk-1 to vk for all 1 k n].

Example: Consider the graph given in the Fig-13.1

i) The path v5 e8 v3 e6 v4 e3 v1 is a directed path from v5 to v1.

ii) If we disregard the orientation in D, then v5 e7 v4 e6 v3 e1v1 is a path.

Observe that it is not a directed path. So it is a "semi-path".

Since a directed walk is a walk, no edge appears more than once (but a

vertex may appear more than once).

Definitions: Let D be a digraph.

i) A walk which is not a directed walk is called as semi-walk. We use the

term "walk" to mean either a directed walk (or) a semi-walk.

ii) A circuit (in the corresponding undirected graph) v0e1v1e2v2 … envn is said

to be a directed circuit if ek is oriented from vk-1 to vk for all 1 k n.

iii) A circuit which is not a directed circuit is a semi-circuit.



Example: Consider the digraph G given 13.1

i) The circuit v1e1v3e6v4e3v1 is a directed circuit.

ii) The circuit v1e1v3e6v4e2v1 is not a directed circuit. So it is a semi-circuit.

In undirected graphs, we define the connectedness by using the notion "path".

As there are two different types of paths (namely directed path, semi-path) in

digraphs, we have two different types of connectedness in the digraphs.

Definitions: Let D be a digraph.

i) D is said to be strongly connected if there exists at least one directed

path from every vertex to every other vertex.

ii) D is said to be weakly connected if its corresponding undirected graph

is connected, but D is not strongly connected.

iii) We say that D is connected if the undirected graph corresponding to D

is connected. So the word "connected digraph" refers to both strongly

connected digraph and weakly connected digraph.

iv) If D is not connected, then we say that D is a disconnected graph.

Example: Consider the digraphs D1 and D2 given in the following.

i) The graph-D1 is a strongly connected graph.

ii) In the graph-D2, there is no directed path from v1 to v4 so it is a weakly

connected digraph.

Graph-D1 Graph-D2

Since there are two types of connectedness in a digraph, we define two

types of components.

Definitions: Let D be a directed graph.

i) Each maximal connected (weakly or strongly) subgraph of a digraph

D, is said to be a component of D.

ii) Within each component of D, the maximal strongly connected

subgraphs are said to be the fragments (or) strongly connected

fragments of D.

e2 e2

v4

v1

e1 e4

e5

e3

e6

e1 e4

e5

e3

e6



Example: Consider the digraph given in the Fig-13.6.

This graph consists of two components H1 and H2.

Fig-13.6

i) The component H1 contains three fragments: {e1, e2}; {e5, e6, e7, e8};

and {e10}.

ii) The component H2 contains a fragment {e11, e12, e13}.

iii) It can be observed that e3, e4 and e9 do not appear in any fragment of

H1.

Definitions: Let D be a digraph. The condensation Dc of D is a digraph

obtained by using the following method:

i) Each strongly connected fragment is to be replaced by a new vertex, and

ii) For any two strongly connected components c1 and c2, the set of all

the directed edges from c1 to c2 is to be replaced by a single directed

edge (from c1 to c2).

iii) Let D be a directed graph and v, u are vertices in D. Then v said to be

accessible (or) reachable from the vertex u, if there is a directed path

from u to v.

Example: Consider the graph D given in the Fig-13.6

i) The components of D are H1, H2.

ii) The fragments of D are C1 = {e1, e2};

C2 = {e5, e6, e7, e8}; C3 = {e10}; and C4 = {e11, e12, e13}.

e1

e3

e8

H1

v1

e9

e13

e12

e14

e11

e10

e2

e5

e4

e6 e7

H2



iii) The fragments C1, C2, C3 and C4 are to be considered as vertices.

iv) There are two edges e3 and e4 from C1 to C2.

These two edges e3 and e4 are to be replaced by an edge (say f1)

from the vertices C1 to C2.

v) There is only one edge e9 from the fragment C2 to C3. So this edge

e9 is to be drawn from the vertex C2 to C3.

vi) There is only one edge (e14) from the fragment C4 to the vertex v1.

So, this edge is to be drawn (in the condensation) from the vertex C4

to v1.

vii) Finally, we get the condensation Dc that is given in the Fig-13.7

Fig. 13.7

Observations:

i) The condensation Dc of a strongly connected digraph D is a vertex.

ii) The condensation Dc of a digraph D contains no directed circuits.

It is clear that a digraph D is strongly connected "v is accessible from u"

for all vertices v and u in D.


4. Let X be a set, and R a relation on X. The relation R is said to be –––––

if a, b, c R, aRb, bRc aRc.

5. Each maximal connected subgraph of a digraph D, is said to be a –––––

of D

13.4 Euler’s Digraphs

Definitions: Let D be a directed graph.

i) A directed walk that starts and ends at the same vertex is called a

closed directed walk.

C3

e9

v1

C4

e14

C2

f1

C1



ii) A closed directed walk which traverses every edge of D exactly once,

is called a directed Euler line.

iii) D is said to be a Euler digraph if it contains a directed Euler line.

Example: Consider the graph given below

Fig. 13.8

The edge sequence f, a, b, c, d, e forms a directed Euler line. Hence, this

graph is a Euler digraph.

Theorem: Let D be a digraph. Then D is an Euler digraph if and only if D is

connected and balanced.

[that is, d+(v) = d-(v) for every vertex v in D].

Definition: A connected digraph containing no circuit (neither a directed

circuit nor a semi-circuit) is said to be a tree.

Note:

i) A tree of n vertices contain n - 1 edges (directed).

ii) Trees in digraphs have additional properties (than those in undirected

graphs) and variations resulting from the relative orientations of the

edges.

Definitions: A digraph D is said to be an arborescence if it satisfies the

following two conditions:

i) D contains no circuit (neither a directed circuit nor a semi-circuit).

ii) There exists exactly one vertex v of zero in-degree (this vertex v is

called the root of the arborescence).

a

fc

e

c

1

2

b

d

c

4

3



Example: The graph given here is arborescence.

Fig. 13.9


6. A closed directed walk which traverses every edge of D exactly once, is

called a –––––––––––.

7. A tree of n vertices contains –––––––– edges.

13.5 Matrix Representation of Digraphs

Definition: Let D be a digraph with p vertices. The adjacency matrix of D is

a p p matrix (aij) with

aij = i j1 if v v is an arc of D

0 otherwise

.

It is denoted by A(D).

Fig. 13.10

v (root)

v2

e5

e7

v1

e8

e6

v4

v3

v5

e1

e4

e3 e2



The adjacency matrix for the digraph given in figure13.10 is

01110

01000

00010

10000

00110

.

The sum of the ith row entries of the adjacency matrix gives d+(vi) and the

sum of the ith column entries gives d-(vi) for every i.

The powers of A(D) give the number of walks from one point to another.

Definitions:

i) Let D be digraph with p vertices. The reachability matrix R = (rij) is the

p p matrix with

.itselffromreachableisvertexeachthatassumew e,otherw ise0

vfromreachableisvif1r ijij

ii) The distance matrix is the p p matrix whose (i,j)th entry gives the

distance from the point vi to the point vj and is infinity if there is no path

from vi to vj.

iii) The detour matrix is the p p matrix whose (i,j)th entry is the length of

any longest vi-vj path and infinity if there is no such path.

Example:

The reachable matrix, distance matrix and the detour matrix respectively for

the matrix of given of 13.10 is given below.

11110 1 1 1 1 1 2

01000

10110 , 2 3 3 1 and 2 3 3 1 .

11110 1 2 2 2 1 2 2 3

11110 2 3 3 1 2 3 3 2


8. Let D be a digraph with p vertices. The adjacency matrix of D is a ––––

matrix



13.6 Summary

In this unit, we have investigated the fundamental features of the

directed graphs.

We have observed that the properties of digraphs are similar to some

types of undirected graphs. The close relation ship between binary

relation and digraphs was given in the unit.

Directed graphs have interesting applications in telecommunications,

flows and networks routing problems.


1. Find the in-degree and out-degree of the following digraphs

2. Find the spanning tree of the digraph given in 1 (ii).

3. Verify whether or not the following digraphs are isomorphic? If so, write

the isomorphisms.

4. Draw the digraphs for the digraph D = {a, b, c, d, e} where the arcs

represented by {(a, c), (a, d), (b, e), (e, c), (d, c)}. Write the in-degree

and out-degree of D.

(i)

c

b a

o

o o

c

a

b

d

e

(ii)

o

o

o o

o

1

2

3

4 o

o

o

o

G:

a

b

c d

G1:

o o

o

o



Find also the converse D1 (i.e., reversing the each arc direction in D)

and find the indegree and outdegree of each point in D1.

5. Give an example of a directed tree with five vertices.

6. Find in-degree and out-degree of each vertex in the following directed

graphs.

13.8 Answers


1. Oriented Graph

2. Out-Degree

3. Zero

4. Transitive

5. Component

6. Directed Euler line

7. n - 1

8. p p

Terminal Questions

1. i) In-degree a = 1

out-degree a = 2

Similarly id (b) = 1, Od (b) = 1

id (c) = 2, Od (c) = 1

ii) id (a) = 1 id (b) = 2 id (c) = 3 id (d) = 1 id (e) = 1

Od (a) = 1 Od ( b) = 2 Od (c) = 2 Od (d) = 1 Od (e) = 1

(i)

v1 v2

v3 v4

v5

o

o

o

o o

(ii)

d

a b

c

o o

o o



2.

3. D and D1 are isomorphic under the isomorphism f(1) = a, f(2) = b, f(3) =

c, f(4) = d.

4. The digraph representing the given arcs is

The converse digraph also can be found in a similar way.

5.

6. i) in-degree v1 =2, out-degree v1 = 1

in-degree v2 = 2, out-degree v2 = 2

Similar for other vertices. (ii). Similar to (i).

c

e

b o

o

o o

o b

d

a

d

e

c a b

o

o

o

o o

a

b c

e

d

aa

a

o

o o

o

o

o



Unit 14 Complexity of Algorithms

Structure

14.1 Introduction

Objectives

14.2 Notations for the Growth Rates of Functions

14.3 Classification of Problems

14.4 Reduction, NP-Complete and NP-Hard Problems

14.5 Establishing NP-Completeness of Problems

14.6 Summary


14.8 Answers

14.1 Introduction

In the previous units, we discussed a number of problems which cannot be

solved by algorithmic means and also discussed a number of issues about

such problems.

In this unit, we will discuss the issue of efficiency of computation of an

algorithm in terms of the amount of time used in its execution. On the basis

of analysis of an algorithm, the amount of time that is estimated to be

required in executing an algorithm will be referred to as the time complexity

of the algorithm. The time complexity of an algorithm is measured in terms

of some (basic) time unit (not second or nano-second). Generally, time

taken in executing one move of a TM, is taken as (basic) time unit for the

purpose. Or, alternatively, time taken in executing some elementary

operation like addition, is taken as one unit. More complex operations like

multiplication etc., are assumed to require an integral number of basic units.

As mentioned earlier, given many algorithms (solutions) for solving a

problem, we would like to choose the most efficient algorithm from amongst

the available ones. For comparing efficiencies of algorithms, that solve a

particular problem, time complexities of algorithms are considered as

functions of the sizes of the problems (to be discussed). The time

complexity functions of the algorithms are compared in terms of their growth

rates (to be defined), as growth rates are considered important measures of

comparative efficiencies.



Objectives


explain the concepts of time complexity, size of a problem and growth

rate of a function

define and explain the well-known notations for growth rates of

functions, viz., O, , , ,

explain criteria for classification of problems into undefinable, definable

but not solvable, solvable but not feasible, P, NP, NP-hard and

NP- Complete etc.

define a number of problems which are known to be NP-Complete

problems

14.2 Notations for the Growth Rates of Functions

The time required by a solution or an algorithm for solving a (solvable)

problem, depends not only on the size of the problem/input and the number

of operations that the algorithm/solution uses, but also on the hardware and

software used to execute the solution. However, the effect of change/

improvement in hardware and software on the time required may be closely

approximated by a constant.

Suppose, a supercomputer executes instructions one million times faster

than another computer. Then irrespective of the size of a (solvable) problem

and the solution used to solve it, the supercomputer solves the problem

roughly million times faster than the computer, if the same solution is used

on both the machines to solve the problem. Thus, we conclude that the time

requirement for execution of a solution, changes roughly by a constant

factor on change in hardware, software and environmental factors.

The Constant Factor in Complexity Measure

An important consequence of the above discussion is that if the time taken

by one machine in executing a solution of a problem is a polynomial

(or exponential) function in the size of the problem, then time taken by every

machine is a polynomial (or exponential) function respectively, in the size of

the problem. Thus, functions differing from each other by constant factors,

when treated as time complexities should not be treated as different, i.e.,

should be treated as complexity-wise equivalent.



Asymptotic Considerations

Computers are generally used to solve problems involving complex

solutions. The complexity of solutions may be either because of the large

number of involved computational steps and / or large size of input data.

The plausibility of the claim apparently follows from the fact that, when

required computers are used generally, not to find the product of two 2 x 2

matrices but to find the product of two n x n matrices for large n running into

hundreds or even thousands.

Similarly, computers, when required, are generally used not to find roots of

quadratic equations but for finding roots of complex equations including

polynomial equations of degrees more than hundreds or sometimes even

thousands.

The above discussion leads to the conclusion that when considering time

complexities f1(n) and f2(n) of (computer) solutions of a problem of size n,

we need to consider and compare the behaviours of the two functions only

for large values of n. If the relative behaviours of two functions for smaller

values conflict with the relative behaviours for larger values, then we may

ignore the conflicting behaviour for smaller values. For example, if the

earlier considered two functions

f1(n) = 1000 n2 and

f2(n) = 5n4

represent time complexities of two solutions of a problem of size n, then

despite the fact that

f1(n) f2(n) for n 14,

we would still prefer the solution having f1(n) as time complexity because

f1 (n) f2(n) for all n 15.


1. Time requirement for execution of a solution, changes roughly by a

constant factor on change in –––––––––– and–––––––––––––––––.

14.3 Classification of Problems

The fact of being engaged in solving problems may be the only sure

indication of a living entity being alive (though, the distinction between



entities being alive and not being alive is getting fuzzier day by day). The

problems, attempted to be solved, may be due to the need for survival in a

hostile and Competitive environment or may be because of intellectual

curiosity of knowing more and more of the nature. In the previous unit, we

studied a number of problems which are not solvable by computational

means. We can go still further and categorize the problems, which we

encounter or may encounter, into the following broad classes:

i) Problems which cannot even be defined formally.

By a formal definition of a problem, we mean expressing in terms of

mathematical entities like sets, relations and functions etc., the

information concerning the problem, in respect of at least

a) Possible inputs

b) Possible outcomes

c) Entities occurring and operations on these entities in the (dynamic)

problem domains.

In this sense of definition of a problem, how to talk of solving, most of

the problems can not even be defined. Think of the following problems.

a) Why the economy is not doing well?

b) Why there is hunger, illiteracy and suffering despite international

efforts to eradicate these?

c) Why some people indulge in corrupt practices despite being

economically well?

These are some of the problems, the definition of each of which

require enumeration of potentially infinite parameters, and hence are

almost impossible to define.

ii) Problems which can be formally defined but can not be solved by

computational means. We discussed some of these problems in the

previous unit.

iii) Problems which, though theoretically can be solved by computational

means, yet are infeasible, i.e., these problems require so large amount

of computational resources that practically is not feasible to solve

these problems by computational means. These problems are called

intractable or infeasible problems. The distinguishing feature of the

problems is that for each of these problems any solution has time

complexity which is exponential, or atleast non-polynomial, function of

the problem size.



iv) Problems that are called feasible or theoretically not difficult to solve

by computational means. The distinguishing feature of the problems is

that for each instance of any of these problems, there exists a

Deterministic Turing Machine that solves the problem having time-

complexity as a polynomial function of the size of the problem. The

class of problem is denoted by P.

v) Last, but probably most interesting class includes large number of

problems, for each of which, it is not known whether it is in P or not

in P.

These problems fall somewhere between class Ill and class IV given

above. However, for each of the problems in the class, it is known that

it is in NP, i.e., each can be solved by at least one Non-Deterministic

Turing Machine, the time complexity of which is a polynomial function

of the size of the problem.

A problem from the class NP can equivalently but in more intuitive

way, be defined as one for which a potential solution, if given, can be

verified in polynomial time whether the potential solution is actually a

solution or not.

The problems in this class are called NP-Complete problems (to be formally

defined later). More explicitly, a problem is NP-complete if it is in NP and for

which no polynomial-time Deterministic TM solution is known so far.

Most interesting aspect of NP-complete problems, is that for each of these

problems neither, so far, it has been possible to design a Deterministic

polynomial-time TM solving the problem nor it has been possible to show

that Deterministic polynomial-time TM solution can not exist.

The idea of NP-completeness was introduced by Stephen Cook* in 1971

and the satisfiability problem defined below is the first problem that was

proved to be NP-complete, of course, by S. Cook.

Next, we enumerate some of the NP-complete problems without justifying

why these problems have been placed in the class.

Problem: Hamiltonian circuit problem (HCP) given an undirected graph

G = (V, E), does G contain a Hamiltonian circuit?

Further Explanation: A Hamiltonian circuit of a graph G = (V, E) is a set of

edges that connects the nodes into a single cycle, with each node appearing



exactly once. We may note that the number of edges on a Hamiltonian

circuit must equal the number of nodes in the graph.

Further, it may be noted that HCP is a special case of TSP in which the cost

between pairs of nodes is the same, say, 1.

Example: Consider the graph

Fig. 14.1

Then, the above graph has one Hamiltonian circuit viz., (1, 2, 4, 3, 1)

Problem: The vertex cover problem (VCP) (also known as Node cover

problem). Given a graph G = (V, E) and an integer k, is there a vertex cover

for G with k vertices?

Explanation: A vertex cover for a graph G is a set C of vertices so that

each edge of G has an endpoint in G. For example, for the graph shown

above, {1, 2, 3} is a vertex cover. It can be easily seen that every superset

of a vertex cover of a graph is also a vertex cover of the graph.

Problem: K-Colourability Problem: Given a graph G and a positive integer

k, is there a k-colouring of G?

Explanation: A k-colouring of G is an assignment to each vertex of one of

the k colours so that no two adjacent vertices have the same color. It may

be recalled that two vertices in a graph are adjacent if there is an edge

between the two vertices.



Fig. 14.2

As the vertices 1, Z 3 are mutually adjacent therefore, we require atleast

three colours for k-colouring problem.

Problem: The complete sub-graph problem (CSP Complete Sub) or clique

Problem: Given a graph G and positive integer k, does G have a complete

sub-graph with k vertices?

Explanation: For a given graph G = (V, E), two vertices v1 and v2 are said

to be adjacent if there is an edge connecting the two vertices in the graph. A

sub-graph H = (V1, E1,) of a graph G = (V, E) is a graph such that V1 V

and E1 E. In other words, each vertex of the sub-graph is a vertex of the

graph and each edge of the sub-graph is an edge of the graph.

Complete Sub-graph of a given graph G is a sub-graph in which every pair

of vertices is adjacent in the graph.

For example, in the above graph, the sub-graph containing the vertices

{1, 2, 3} and the edges (1, 2), (1, 3), (2, 3) is a complete sub-graph or a

clique of the graph. However, the whole graph is not a clique as there is no

edge between vertices 1 and 4.

Problem: Independent set problem: Given a graph G = (V, E) and a positive

integer k, is there an independent set of vertices with atleast k elements?

Explanation: A subset V1 of the set of vertices V of graph G is said to be

independent, if no two distinct vertices in V1 are adjacent. For example, in

the above graph V1 = {1, 4} is an independent set.

Problem: The sub-graph isomorphism problem: Given graph G1 and G2,

does G1 contain a copy of G2 as a subgraph?



Explanation: Two graphs H1 = (V1, E1) and H2 = (V2, E2) are said to be

isomorphic if we can rename the vertices in V2 in such a manner that after

renaming, the graph H1 and H2 look identical (not necessarily pictorially, but

as ordered pairs of sets)

Fig. 14.3

are isomorphic graphs because after mapping 1 a, 2 b, 3 c and

4 d, the two graphs become identical.

Problem: Given a graph g and a positive integer k, does G have an ―edge

cover‖ of k edges?

Explanation: For a given graph G = (V, E), a subset E1 of the set of edges

E of the graph, is said to be an edge cover of G, if every vertex is an end of

at least one of the edges in E1.

Example: For the graph

Fig. 14.4

The two-edge set {(1, 4), (2, 3)} is an edge cover for the graph.



Example: Let S = {1, 2,……, 10}

and P = {S1, S2, S3, S4, S5} such that

S1 = {1, 3, 5}

S2 = {2, 4, 6}

S3 = {1, 2, 3, 4}

S4 = {5, 6, 7, 9, 10}

S5 = {7, 12, 9, 10}

Then Q = {S1, S2, S5} is a set cover for S.


2. Given a graph G and a positive integer k, is there a –––––––––––– of G.

3. Complete Sub-graph of a given graph G is a sub-graph in which every

pair of vertices is ––––––––– in the graph.

14.4 Reduction, NP-Complete and NP-Hard Problems

We know that a problem is called NP-Complete if P has atleast one Non-

Deterministic Polynomial-time solution.

In this section we formally define the concept and then describe a general

technique of establishing the NP-Completeness of problems and finally

apply the technique to show some of the problems as NP-complete.

For the formal definition of NP-completeness, polynomial-time reduction, as

defined below, plays a very important role.

We have already discussed in previous units the reduction technique to

establish some of the problems as undecidable. The method that was used

for establishing undecidability of a language using the technique of

reduction, may be briefly described as follows:

Let P1 be a problem which is already known to be undecidable. We want to

check whether a problem P2 is undecidable or not. If we are able to design

an algorithm which transforms or constructs an instance of P2 for each

instance of P1, then P2 is also undecidable.

The process of transformation of the instances of the problem already

known to the undecidable to instances of the problem, (the undecidability is

to be checked), is called reduction.



Some-what similar, but, slightly different, rather special, reduction called

polynomial-time reduction is used to establish NP-Completeness of

problems.

A Polynomial-time reduction is a polynomial-time algorithm which constructs

the instances of a problem P2 from the instances of some other problems

P1.

A method of establishing the NP-Completeness (to be formally defined later)

of a problem P2 constitutes of designing a Polynomial time reduction that

constructs an instance of P2 for each instance of P1, where P1 is already

known to be NP-Complete.

The direction of the mapping must be clearly understood as shown below.

Reduction

PtimePolynomial

P 21

(Problem already known to be undecidable) (Problem whose NP-

completeness is to be

established)


4. ––––––––––– –––––––––– is a polynomial-time algorithm which

constructs the instances of a problem P2 from the instances of some

other problems P1.

5. A problem L is said to be –––––––––––––– if for any problem L1 in NP,

there is a polynomial-time reduction of L1 to L.

14.5 Establishing NP-Completeness of Problems

In general, the process of establishing a problem as NP-Complete is a two

step process. The first step, which in most of the cases is quite simple,

constitutes of guessing possible solutions of the instances, one instance at a

time, of the problem and then verifying whether the guess actually is a

solution or not.

The second step involves designing a polynomial-time algorithm which

reduces instances of an already known NP-Complete problem to instances

of the problem, which is intended to be shown as NP-Complete.



However, to begin with, there is a major hurdle in execution of the second

step. The above technique of reduction can not be applied unless we

already have established atleast one problem as NP-Complete. Therefore,

for the first NP-Complete problem, the NP-Completeness has to be

established in a different manner.

As mentioned earlier, Stephen Cook (1971) established Satisfiability as the

first NP-Complete problem. The proof was based on explicit reduction of the

language of any non-deterministic, polynomial-time TM to the satisfiability

problem.

The proof of satisfiability problem as the first NP-Complete problem, is quite

lengthy and we skip the proof. Interested readers may consult any of the

texts given in the reference.

Assuming the satisfiability problem as NP-Complete, the rest of the

problems that we establish as NP-Complete, are established by reduction

method as explained above. A diagrammatic notation of the form.

Fig. 14.5

The block diagram in 14.5 indicates that , assuming P as already

established to be NP-Complete, the NP- Completeness of Q is established

by a polynomial-time reduction from P to Q.

A scheme for establishing NP-Completeness of some the problems is

suggested by means of Figure 14.6 given below:



Fig. 14.6: A scheme for establishing NP-Completeness of

some the problems

Example: Show that the Clique problem is an NP-Complete problem.

Proof: The verification of whether every pairs of vertices is connected by an

edge in E, is done for different pairs of vertices by a Non-deterministic TM,

i.e., in parallel. Hence, it takes only polynomial time because for each of n

vertices we need to verify atmost n (n + 1) /2 edges, the maximum number

of edges in a graph with n vertices.

Next, we show that 3-CNF-SAT problem can be transformed to clique

problem in polynomial time.

Take an instance of 3-CNF-SAT. An instance of 3CNF-SAT consists of a set

of n clauses, each consisting of exactly 3 literal, each being either a variable

or negated variable. It is satisfiable if we can choose literals in such a way

that:

• Atleast one literal from each clause is chosen

• If literal of form x is chosen, no literal of form x is considered.



Fig. 14.7: Graph

For each of the literals, create a graph node, and connect each node to

every node in other clauses, except those with the same variable but

different sign.

Example: Show that the Vertex cover problem is an NP-Complete.

A vertex cover of an undirected graph G = (V, E) is a subset V’ of the

vertices of the graph which contains atleast one of the two endpoints of

each edge.

Fig. 14.8: Graph showing vertex cover problem



The vertex cover problem is the optimization problem of finding a vertex

cover of minimum size in a graph. The problem can also be stated decision

problem:

VERTEX-COV = {<G, k> /graph G has a vertex cover of size k}.

A deterministic algorithm to find a vertex cover in a graph is to list all

subsets of vertices of size k and check each one to see whether it forms a

vertex cover. This algorithm is exponential in k.

Proof: To show that Vertex cover problem NP, for a given graph

G= (V, E), we take V’ V and verifies to see if it forms a vertex cover.

Verification can be done by checking for each edge (u, v) E whether u V’

or v V’. This verification can be done in polynomial time.

Now, we show that clique problem can be transformed to vertex cover

problem in polynomial time. This transformation is based on the notion of

the complement of a graph G. Given an undirected graph G = (V, E), we

define the complement of G as G’ = (V, E’), where E’={(u, v)/(u, v) E}. i.e.,

G’ is the graph containing exactly those edges that are not in G. The

transformation takes a graph G and k of the clique problem. It computes the

complement G’ which can be done in polynomial time.

To complete the proof, we can show that this transformation is indeed

reduction: the graph has a clique of size k if and only if the graph G’ has a

vertex cover of size |V| – k.

Suppose that G’ has a clique V’ V with |V’| = k. We claim that V — V’ is a

vertex cover in G’ Let (u, v) be any edge in E’. Then, (u, v) E, which

implies that atleast one of u or v does not belong to V’, since every pair of

vertices in V’ is connected by an edge of E. Equivalently, at least one of u or

v is in V — V’, which means that edge (u, v) is covered by V — V’. Since (u,

v) was chosen arbitrarily from E’, every edge of E’ is covered by a vertex in

V — V’. Hence, the set V — V’, which has size |V| - k, forms a vertex cover

for G’.

Conversely, suppose that G’ has a vertex cover V’ V, where |V’| = |V| — k.

Then, for all u, v V, if (u, v) E’, then u V’ or v V’ or both. The contra

positive of this implication is that for all u, v V, if u V’ and v V’, then (u,

v) E. In other words, V— V’ is a clique, and it has size |V| - |V’| = k. For



example, the graph G(V, E) has a clique {A, B, E}. The complement of graph

G is given by G’ and have independent set given by {C, D, F}.

This proves that finding the vertex cover is NP-Complete.

In short, problems are designated as NP-complete if their solutions can be

quickly checked for corrections, and if the same solving algorithm used can

solve all other NP problems. They are the most difficult problems in NP in

the sense that a deterministic, polynomial-time solution to any NP complete

problem would provide a solution to every other problem in NP. Conversely,

if any one of them provably lacks a deterministic polynomial-time solution

then none of them has one. One example of an NP complete problem is the

subset sum problem which is: given a finite set of integers, determine

whether any non-empty subset of them sums to zero. A supposed answer is

very easy to verify for correctness, but there is no known efficient algorithm

to find an answer, that is, all known algorithms are impractically slow for

large sets of integers.


6. The process of establishing a problem as –––––––––is a two step

process

7. Example of an NP complete problem is the––––– ––––– ––––––.

12.6 Summary

P denotes the class of all problems, for each of which there is atleast

one known polynomial time Deterministic TM solving it.

The notation O provides asymptotic upper bound for a given function.

The notation provides an asymptotic lower bound for a given function

The Notation provides simultaneously both asymptotic lower bound

and asymptotic upper bound for a given function.

A Polynomial-time reduction is a polynomial-time algorithm which

constructs the instances of a problem P2 from the instances of some

other problems P1.


1. Explain the Constant Factor in Complexity Measure

2. Explain the Different Category of Problems



12.8 Answers


1. Hardware, software, environmental factors

2. k-colouring

3. Adjacent

4. A Polynomial-time reduction

5. NP-Hard

6. NP-Complete

7. Subset sum problem

Terminal Questions

1. If the time taken by one machine in executing a solution of a problem is

a polynomial (or exponential) function in the size of the problem, then

time taken by every machine is a polynomial (or exponential) function

respectively, in the size of the problem. Thus, functions differing from

each other by constant factors, when treated as time complexities

should not be treated as different, i.e., should be treated as complexity-

wise equivalent

2. Problems which cannot even be defined formally.

By a formal definition of a problem, we mean expressing in terms of

mathematical entities like sets, relations and functions etc., the

information concerning the problem, in respect of at least

a) Possible inputs

b) Possible outcomes

c) Entitles occurring and operations on these entities in the (dynamic)

problem domains. (Refer Section 14.3)



Acknowledgement, References and Suggested Readings

1. Coremen. T. H., Leiserson. C.E. & Stein. C, (2002), Introduction to

Algorithms (second edition), Prentice-Hall of India.

2. Deo Narsing, 1986, ―Graph Theory with Applications to Engineering and

Computer Science‖, Prentice Hall India, New Delhi.

3. Horowitz. E. & Sahni. S., Fundamentals of Computer Algorithms,

(Galgotia Publications)

4. Kolman Bernard, Busby. R.C., Ross Sharon, 1999, ―Discrete

Mathematical Structures‖ PHI,.

5. Levis. H.R. & Papadimitriou. C.H., (1981), Elements of the Theory of

computation, PHI,

6. www.wikipedia.com.

http://www.wikipedia.com/

bt 0080 fundamentals of algorithms contents · mr. deepak shetty mr. vinayak g pai assistant...

Documents