an efficient tasks scheduling model in distributed...

An Efficient Tasks Scheduling Model in Distributed Processing Systems...F F

International Journal of Advances in Circuits and Systems, Vol. 1, No. 1-2, Jan-Dec 2011 53

An Efficient Tasks Scheduling Model inDistributed Processing Systems UsingANN

Pradeep Kumar YadavCentral Building Research Institute, Roorkee (UK) (India),E-mail: [email protected].

M.P. SinghProfessor, Gurukula Kangri Vishwavidyalaya Hardwar (UK) (India).

Avanish KumarBundelkhand University, Jhansi-284128, (UP) (India)E-mail: [email protected].

Babita AgarwalResearch Scholar, Gurukula Kangri Vishwavidyalaya,Hardwar (UK) (India).

ABSTRACT: The single communication channel is shared by allthe processors for Inter-Processor Communication (IPC) in adistributed system. A program whose execution is distributedamong several processors in a distributed system has the totalprocessing cost equal to the sum of Execution Costs (EC) andInter-Processor Communication, which are function of the amountof data transmitted. An optimal assignment is a distribution oftasks that has lowest total execution cost. The neural networkarchitecture is used to form tasks clusters between maximallylinked tasks and to restrict the clusters size, the average load tobe assigned to each processor. The Artificial Neural Network(ANN) based model has been discussed in this paper. The modelprovides near to optimal solution for assigning a set of “m” tasksof a program to set of “n” processors where “m >> n” in such away that allocated load on all processors is balanced accordingto the average load. For testing the efficiency of the algorithm,several sets of input data have been considered for all program

mailto:[email protected]

mailto:[email protected]

Pradeep Kumar Yadav, M.P. Singh, Avanish Kumar and Babita AgarwalF F

54 International Journal of Advances in Circuits and Systems, Vol. 1, No. 1-2, Jan-Dec 2011

categories, and it is found that our algorithm resulted optimalallocation in most of the cases.

Keywords: Distributed system, inter-processor communication,execution cost, artificial neural network, optimization.

1. INTRODUCTIONDistributed system is a computer system in which multiple processorsconnected together through the high band-width communicationlinks. These links provide a medium for each processor to accessdata and programs on remote processors. Distribution of resourcein distributed system is seen as a way to improve system throughputand availability. An important resource for distributed system is user’sprogram that consists of a set of tasks.

Response time is the important parameter for measuringperformance of distributed system. A distributed system is designedfor solving some specific real time applications. The system isrequired to efficient scheduling of the task in computer communi-cation network and finish a certain task within a specific time limit[15]. The excessive Inter-Processor Communication is always mostcostly and least reliable factor in the loosely coupled distributedsystem. Therefore, an efficient strategy is required for optimalutilization of processor’s capacity in distributed system. Foroptimal utilization of processor’s capacity it is essential to minimizeIPC that arises when the interacting tasks reside on differentprocessors. Only minimizing the IPC alone may not provide a goodresult. A processor’s load indicates the sum of execution times ofthe tasks residing on that processor. Balancing the load on processorsand minimizing the IPC can minimize the response time of thesystem.

Several approaches have been reported for solving the taskallocation problem in distributed system. Program partitioning intothe small tasks and their allocations on the distributed computingsystem for processing play the important role in utilizing the distri-buted system power. The task partitioning, allocation and granularity[1] activities influence the distributed software properties such asInter Processors Communication (IPC) cost and potential for paral-lelism. The manner in which this partitioning is done determinesthe efficiency of a given application when it is executed on distributedcomputing systems.



Therefore an efficient task allocation strategy is required for theproper utilization of computational resources and minimization ofIPC. Several approaches to solve the problem of task assignment indistributed system have been developed but most of these methoddeals with homogenous systems and the complexity of thesealgorithms increase rapidly as the problem size increased. Some ofthe heuristic model for task allocation in distributed computersystems has been discussed by [2-4]. The tasks allocation modelbased on graph theoretic approach has been developed by HeshamH. Ali and Hasham EI-Rewini [5] for NP-hard problem. A modelwas discussed by Chu, T., and Abraham J [6] considering the loadbalancing through systematic scheduling of tasks in distributedcomputing environment. Chu, W., Holloway, L., Lan, M., and Efe,K., [7] discussed a task allocation model for distributed dataprocessing. A tasks allocation model developed by Ma, P., Lee, E.and Tsuchiya, M [8] using branch & bound method. An Algorithmfor solving the Unbalanced Assignment Problems has beendeveloped by [9]. Yadav, P.K., Kumar, Avanish and Gupta, A. R[10] has provided the solution for task allocation in distributedsystem. Cost Based Static Approach for Performance Enhancementof Distributed Networks has been discussed by [11]. ImprovedReliability of Distributed Computing Systems using MathematicalProgramming has been discussed by [12]. Tasks scheduling algorithmfor Multiple Processors system with Dynamic Re-assignment hasbeen recently developed by [13].

By balancing load on the processors, it is possible to get efficientutilization of available computational power. In this paper, authorspresent a task allocation algorithm with the help of ANN workingin union to solve specific problems.

Neural networks process information in a similar way the humanbrain does. The network is composed of a large number of highlyinterconnected processing elements (neurons) working in parallelto solve a specific problem.

An artificial neuron is a device with several inputs and oneoutput. The neuron works in two modes of operation; the trainingmode and the using mode. In the training mode, the neuron can betrained to fire (or not), for particular input patterns. In the using



mode, when a taught input pattern is detected at the input, itsassociated output becomes the current output. If the input patterndoes not belong in the taught list of input patterns, the firing rule isused to determine whether to fire or not.

A Simple Neuron.

Feed-forward ANNs allow signals to travel one way only; frominput to output. There is no feedback (loops) i.e. the output of anylayer does not affect that same layer. Feed-forward ANNs tend tobe straight forward networks that associate inputs with outputs. Inthis paper we use feed-forward ANN to assign tasks to the processorand find the near optimal solution. Neural networks do not performmiracles. But if used sensibly they can produce some amazing results.

The allocation is done before the actual run of the applicationprogram. It is assumed that the execution cost of the subtasks andthe IPC cost, which arise due to the interacting subtasks residing ondifferent processors are known. It allocates the modules or subtasksof an application problem as evenly as possible among the processorsand tries to minimize the IPC cost.

2. TASK ALLOCATION PROBLEMWe conceive of a distributed system as composed as a set of “n”processing nodes P = {p1, p2, ... pn) and an interconnectionstructure providing full connectivity between the nodes and anapplication program consists a set of “m” tasks T = {t1, t2, ... tm} to beexecuted on these processing nodes. An allocation of tasks toprocessors is defined by a function Aalloc, from the set of tasks to theset of processors.



Aalloc: T P, where Aalloc (i) = j If the task ti is assigned toprocessor Pj, 1 ≤ i ≤ m, 1 ≤ j ≤ n.

3. DEFINITIONS

3.1. Execution CostThe execution cost eij, is the amount of the work to be performed bythe executing task ti on the processor pj. The overall execution cost(EC) of a given allocation (Aalloc) and execution time for eachprocessor (PEC) are calculated by using equation (1) and (2) respectively.

EC (Aalloc) =≤ ≤∑ , ( )

1i Aalloc i

i me ...(1)

PEC (Aalloc)j =∈

≤ ≤∑ , ( )

1i TSJ

i Aalloc ii m

e ...(2)

Where TSj = {i : Aalloc (i) = j, j = 1, 2 ... n}

3.2. Inter Processor Communication CostThe inter processor communication cost cik is incurred due the datalimits exchanged between the executing task ti and residing task tkif they are on different processors. The overall inter-processorcommunication (IPC) cost of a given allocation Aalloc is thencalculated by equation (3) and for each Processor Inter-ProcessorCommunication (PIPC) cost is then given by equation (4).

IPC (Aalloc) =+ ≤ ≤

≠

≤ ≤∑

1( ) ( )

( ), ( )1i j m

Aalloc i Aalloc j

Aalloc i Aalloc ji m

c ...(3)

PIPC (Aalloc)j =+ ≤ ≤

= ≠

≤ ≤∑1

( ) ( )

( ), ( )1

i j mAalloc i j Aalloc k

Aalloc i Aalloc ki m

c ...(4)

(Where j = 1, 2, ... n)

3.3. Response Time (RT) of the SystemResponse time of the system is a function of amount computationto be performed by each processor and the computation time. Thisfunction is defined by considering the processor with the heaviest



aggregate computation and communication load. Response time ofthe system for a given assignment is defined by

RT(Aalloc) =≤ ≤1

maxj n

{EC(Aalloc)j + PIPC (Aalloc)j } ...(5)

3.4. Maximally Linked TaskA task tk in a program graph is called a maximally linked task if thesum of IPC costs of tk with other tasks is larger than the sum of IPCcosts of any neighboring task i.e.

>∑ ∑ki hji j

c c for all h ≠ k such that ckh ≠ 0

Where cij is the IPC cost between the tasks ti and tj , when ti and tjare assigned to different processors.

3.5. Average LoadAverage load on a processor is the sum of EC of the tasks assignedto it then the average load that must be assigned to each processorpj is calculated as

Lavg (Pj) = jWm

, j = 1, 2…………..n.

where Wj =≤ ≤∑ .

1,i j

i me j = 1, 2…………..n.

Tlod = ∑1

( )n

jLavg P

4. ASSUMPTIONSTo keep the algorithm reasonable in size, several assumptions havebeen made while designing the algorithm. A program is assumedto be collection of “m” tasks to be executed on a set of “n” processors,which have different processing capabilities. A task may be portionof an executable code or a data file and size of all the tasks are equal.The number of tasks to be allocated is more than the number ofprocessors (m>>n), as normally is the case in the real life. It is assumedthat the execution time of a task on each processor is known, if atask is not executable on any of the processor due to absence ofsome resources. The execution time of that task on that processor is



taken to be (∞) infinite. It is also assumed that once a task has comple-ted its execution on a processor, the processor stores the output dataof the task in its local memory. If the data is needed by some anothertask being computed on the same processor, it reads the data fromthe local memory. Using this fact, the algorithm tries to assignmaximally linked tasks to the same processor. Whenever groupsof tasks or cluster are assigned to the same processor, the datatransfer between them is zero. Completion of a program from compu-tational point of view means that all related tasks have got executed.

5. TASK ALLOCATION MODELIn many heuristic approaches, a search is made for a pair of modulewith maximum communication cost between them. Such a modulepair is assigned to a processor to minimize the IPC cost. But, itincreases when a module is interacting with many other modules.In this paper, authors do not search for such pairs of modules, butsearch maximally linked modules. The flowchart of whole programis shown in Figure 1.

Figure 1: Flow Chart of the Program.



5.1. Maximum Linked TaskIt is three layer feed forward network using neuron having theunipolar activation function given in figure 2. Each of the networklayers is described by the formula O = Γ(WX) At first layer, the inputvector contains the elements of the upper triangular matrix of theIPC matrix T and output contains the m elements. The value of eachweight for the first layer is 1. At the second layer the output of thefirst layer work as an input vector and the output vector hasm(m – 1) elements. In this layer, each element is compared with others(m – 1) elements to search the maximum linked module. Whenith element is compared with jth element then the weight Wij = Sgn(tij) – 0.5) and for others it is equal to one. At third layer each weighthas value one. The output will be one for maximum linked moduleand zero for others. If maximum linked module is not equal to thenumber of processors then we select remaining modules accordingto maximum inter processing communication cost. To obtain them,the max comparator has been used. Thus the module clusters areformed around the maximally linked module and these n modulesare assigned on each processor.

Figure 2: Network for Maximally Linked Task.



Processing loss function: This function is trying to assign ithtaskti to jth processor pj in memory the average load of each processor isstored. Now processing loss function find the difference of theaverage load and total execution cost of tasks which are assignedon processor pj. This difference is called processing loss.

IPC Loss Function: When a ith task is assigned at jth processor,then IPC cost of the ith task with the task which are already assignedat jth processor will becomes zero and it will be added with anothertask which are not assigned at jth processor. Memory contains theprevious IPC cost of assigned tasks at each processor. The differenceof these IPC cost is called IPC loss. Neural network of IPC loss andprocessing loss of both is shown in figure 3. If the sum of theprocessing loss and IPC loss is positive then the ith task can beassigned to jth processor. This process is continued until the processingloss approaches zero/less than zero.

Figure 3: (A)-IPC Loss Task, (B)-Processing Loss Task.



6. EXAMPLEConsider a distributed program consisting of a set T = {t1.t2, t3, ... t9}of 9 tasks to be allocated to a set P = {p1, p2, p3} of 3 processors. Theobjective of task assignment is to minimize the completion cost of adistributed program by properly mapping the tasks to the processors.Authors of [14] have devised a problem and developed a solutionfor it. We have tested our solution on that problem. To illustrate thealgorithm, a program graph has been constructed and is shown infigure 4 with the execution time matrix E being shown in Table 1 forthree processors. Average load on each processor is shown in Table 2.

Figure 4: Example Program Graph.

Table 1Task Execution Time Matrix E.

Task number p1 p2 p3

t1 174 156 110t2 95 15 134t3 196 79 156t4 148 215 143t5 44 234 122t6 241 225 27t7 12 28 192t8 215 13 122t9 211 11 208



Table 2Average Load on Each Processor.

Average Load p1 p2 p3 Total Load

445.333 325.333 404.667 1175.333

In starting first three maximally linked tasks are selected. Whichare t1, t8, t9 assigned to their best processor according to minimumexecution cost on processor? In this case task t1 is assigned to p3, taskt8 to p2 and task t9 to p1 which is the remaining unallocated processor.After this the algorithm proceeds as described in section 4.2 and4.3. It should be noted that after each assignment of a task to theprocessor the total IPC time associated with that processor decreases.Results are shown in section 6.

7. RESULTS & CONCLUSIONThe static task allocation is that when a task is assigned to processor,it remains there while the characteristic of the computation changeand a new assignment is to be computed. The suggested piece ofresearch is in finding the assignment pattern that holds for the lifetime of a task, and result in the optimum value of themeasure of effective-ness. An artificial neural network based solution of static allocationof tasks to processors is used. In distributed computing environmentthe problem of distributing tasks to process is discussed and the resultsobtain are given in Table 3 shows the result obtained by the model.

Table 3Result

Task/ Cumulative Aggregate ProcessingOrder Execution Cost IPC Loss

Processor-P1

t9/3 211 27 234.3333t6/6 452 18 –6.667t7/9 464 15 –18.667

Response time of Processor p1 is = 479Processor-P2

t8/1 13 31 312.333t5/4 247 25 78.333t4/7 462 16 –136.667

Contd. Table 3



Response time of Processor p2 is = 478Processor-P3

t1/2 110 29 294.6667t3/5 266 27 138.6667t2/8 400 15 4.667

Response time of Processor p3 is = 415

As it is not possible to achieve, a distributed system is consideredto be balanced if the load on each processor is equal to the averageload, within a reasonable tolerance that is of 20-30% of average loadis generally chosen. Figure 5 show the maximum busy time of thesystem is 479 which relates to processor p2 after getting the assign-ment from the algorithm is concluded that the total assigned loadon the system is 1372 with the tolerance of 20%. This is very near tothe calculated load given in Table 2. The result obtained by ouralgorithm is similar to [14] but in present algorithm the solutionproposed is highly parallel and can be implemented on chip level.

Figure 5: Maximum Busy Time of the System.

As we have assumed the size of all the tasks are equal and fromthe result it is concluded that all the processors are executing threetasks form the figure 6, the through put of the p3 processor ismaximum because in the execution cost of processor p3 is minimum.

Contd. Table 3



Figure 6: Throughput of the Processors.

The run time complexity of the algorithm suggested by Elsade[14] is O(n2 + m2 + m2 n) and the run complexity of the algorithmpresented in this paper is O(m2n + mn + n2). Figure 6 shows thecomparison between both the method and it concluded that presentmethod is much better than Elsade [14].

Figure 6: Complexity Comparison.



REFERENCES[1] Vetanovic, Z, “The Effects of Problem Partitioning, Allocation and Granularity

on the Performance of Multiple Processor System”, IEEE Trans., 1987, C-36,pp. 421-431.

[2] A.K. Sarje, and G. Sagar, “Heuristic Model for Task Allocation in DistributedComputer Systems”, IEE Proceedings, E., 138, No. 5, September 1991.

[3] Efe. K., “Heuristic Models of Task Assignment Scheduling in DistributedSystems”, IEEE Transaction on Computers, June 1982, pp. 50-56.

[4] Virginia Mary Lo, “Heuristic Algorithm for Task Assignment in DistributedSystems”, IEEE Transaction on Computers,37, No. 11, November 1988, pp. 1384-1397.

[5] Hesham H. Ali and Hasham EI-Rewini, “A Graph Theoretic Approach forTask Allocation”, IEEE Transaction on Computers, 1992.

[6] Chu, T., and Abraham J., “Load Balancing in Distributed Systems”, IEEETransaction on Software Engineering, SE-8, No. 4, July 1981.

[7] Chu, W., Holloway, L., Lan, M., and Efe, K., “Task Allocation in DistributedData Processing”, IEEE Computer, November 1980, pp. 57-69.

[8] Ma, P., Lee, E. and Tsuchiya, M., “A Task Allocation Model for DistributedComputing Systems”, IEEE Trans. Computer., Jan. 1982, pp. 41-47.

[9] Yadav, P.K., Kumar, Avanish and Singh, M.P., “An Algorithm for Solvingthe Unbalanced Assignment Problems”, International Journal of MathematicalSciences, 12(2), pp. 447-461 2004.

[10] Yadav, P.K., Kumar, Avanish and Gupta, A.R., “An Exhaustive Approach ofPerformance Analysis to the Distributed Systems Based on Cost Assignments”,Published in South East Asian Journal of Mathematics and Mathematical Sciences,5, No.1, pp. 29-44, 2006.

[11] Yadav, P.K, Govil, Kapil, and Kumar, Avanish, “Cost Based Static Approach forPerformance Enhancement of Distributed Networks”, Published to theJournal Reflections des ERA Journal of Mathematical Sciences, 2, issue 1, 2007, pp. 23-42.

[12] Kumar, Avanish, Yadav, P.K., and Mudit Bansal, “Evaluation of ImprovedReliability of Distributed Computing Systems using MathematicalProgramming”, Published to the Journal of Mathematical Biosciences, GwaliorAcademy of Mathematical Sciences, 3, No. 1, 2007, pp. 88-96.

[13] Yadav, P.K., Singh, M.P. and Kumar, Harendra, “Scheduling Algorithm: TasksScheduling Algorithm for Multiple Processors with Dynamic Re-assignment“,Journal of Computer Systems, Networks, and Communications, 2008, pp. 1-9, 2008.

[14] Elsade A.A., Wells B.E., “A Heuristic Model for Task Allocation inHeterogeneous Distributed Computing System”, The International Journal ofComputers and Their Applications, 6, no.1, March 1999.

[15] Yadav, P.K., Singh Jumindera and Singh, M.P (2009)., “An Efficient Methodfor Task Scheduling in Computer Communication Network”, InternationalJournal of Intelligent Information Processing, 3(1) pp. 81-89.

an efficient tasks scheduling model in distributed...

Documents