North-Holland Microprocessing and Microprogramming 27 (1989) 489--494 489

HIGHLY PARALLEL DISTRIBUTED COMPUTING SYSTEMS WITH OPTICAL INTERCONNECTIONS

J.R. Just R.S. Romaniuk

Warsaw University of Technology, lnstitute of Electronics Fundamentals ul. Nowowiejska 15/19, 00-665 Warsaw, Poland

The paper deal with the design philosophy of massively parallel distributed computing systems and will give an outline of network topology and the ideas for the operating system basic structure.

i. INTRODUCTION

The paper will deal with the design philosophy of massively.parallel distributed computing systems and will give an outline of the network topology and the ideas for the operating system structure. A possible configuration of a fibre optic LAN will be presented, with the emphasis on the highly parallel computation.

We are aiming here at an all-optical ultra-fast network (LAN~type) capable of servicing at least i0 ~P nodes. During the design and construction process it is nesessary here to use as much as possible the optical signal processing [8]. Also, optically-based signal processing architectures have to be realized. Optical processing of the media acces protocol raises theoretically the networkcapacity, uses more efficiently the available bandwidth of dispersion-optimized single-mode fibre and allows to use more compatible and novel protocols (baseband-expansion- type), spread-spectrum, with dynamically changed redundancy etc..

One of the major aims of this work is to debate the configurations of hybrid and all-optical systems. Changing metal carriers of network's signal to dialectric ones and the distances typical for LANs to I-i0 m for multi-~P network,we fall into completely different class of theoretical and technical confinements. The speed limitations are here at ~ level of up to 1Thz. The bandwidth is {ndependent of the distance between stations. The system works on the edge of optical nonstability due to nonlinear phenomena, The optical network is singlemode with desired high stability of polarization throught all the nodes (coherent).

We have built a computer model of a multi-~P coherent fibreoptic network featuring the following characteristics:

- task-orineted self-reconfiguration capacity; Self-reconfiguration is needed, in the network, for maximalization of parallel processing rate and optimal task partition to

fundamental problems. Here, the self-

reconfiguration process are dynamical and task-driven. These processes are not

a-prjo~q ~oftware driven, - fault tolerability, obtained through programmed redundancy, proper topology and emergency-reconfiguration capacity, - strictly nonblocking character of an optical network [9]; A new connection between a free inlet and a free outlet, in the optical network, can always be made without disturbing existing connections, - optical signal self-routing in the network [7]; The process comparizes of a self-addressing and self-switching in real time during the passage of the signal through the network. For instance, each uP-node in the network is assigned a fixed code sequence witch will serve as its address, any node wishing to transmit data will encode each bit with characteristic code-sequence of a receiving node. All used sequences have to be pseudo-ortogonal to allow big number of nodes to access the bus simultaneously. The label bits open the way for the whole frame through the network.

We will discuss how these major properties can be realized practically in the lightwave technology and applied in massively parallel distributed computing system. The work closes with examples of synthesis of basic control structures and topologies supporting our distributed system. By simulation of any

490 J. Just, R. Romaniuk / Distributed Systems with Optical Interconnections

changes in the optical network's topology and~or signal organization we can trace the values of several quality indicators for coherent fibreoptic multi-~P network like: -maximalization of networks's parallel distributed computing capabilities (maximum network transparency), -minimalization of network's structure - including minimalization of number of modes,minimalization of number of spurious dimensions, -optimization of code sequences for particular network's topology; topology oriented coding and task oriented coding, where the code changes with network's reconfiguration,

-CDMA (code-division-multiple-access)

rate requirement optimization ~or a chosen network BER parameter, -checking of a pseudo-orthogonality of local-codes in dynamically build sub'networks, -optimization of allocation of processes to particular processors in order to minimize intercommunication of coprocesses.

2. BASIC CONCEPT OF NETWORK OF CROSS- CONNECTIONS

This part of the work contains a digest of possible networks of cross-connec- tions for a distributed system. We are interested here only in all- optical, as far as the physical layer of system is concerned. The efficiency of acting depends on whether the physical layer is homogenous or not. Homogeneity is obtained only if the access is realized on an optical way and signal routing does not require any use of electric signals. The general realization of such an optical node is presented in figure 2.1.

routing inpout (optical or none)

!

i-th ¢ ~ ~ optical 0 j-th access I node output port port (optical)

(optical) 1

k-th throughput port

(optical) (for inter-processors links)

Fig.2.1. General diagramme of an optical node

The basic node contain direct access ports, throughput (inpout and output) ports for through-traffic and network's output ports.

Optical access ports require that the

input data be pre-processed and transformed to optical form. The access word contain a leading label - which is coded optically and opens proper route for the data in the optical node. In this sense, the steering process can be referred to as an autonomous optical self-routing or data-routing. A separate node's routing input is not necessary in this case and is distinguished here only for the sake of generality. Figure 2.2 shows an exemplary realization of a chosen optical throgh-connection.

virtual routing (i-~ j)

. optical 1 in I o node

O!t j

Fig.2.2. Realization of i--~ j signal routing in an optical node between i-th and j-th processors.

The self-routing feature facilities gratly splitting the whole network which is composed of loosely coupled nodes, to task-oriented sub-networks. The reconfiguration process is software- driven. This solution requires a considerable redundancy in the network's physical layer.The redundancy is here possible, at practically no additional cost, because of the extremely high bandwidth available in optical links. Thus, a spread spectrum codes are here possible with bit to chip ratio as high as i0 . The bit/chip ratios of this order may be required f~r networks containing more than i0 nondependent processors.

Actually, when a self-routing feature is available in the network the problem of virtual network's subdivisions is very easy. The signals themselves divide dynamically the network to appropriate parts realizing parallel singular task.

J. Just, Ro Romaniuk / Distributed Systems with Optical Interconnections 491

3. LOGICAL AND PHYSICAL STRUCTURES OF

DISTRIBUTED COMPUTING SYSTEMS

The distributed computer systems ( DCS for short ) consists of separate microcomputers linked together by input-output ports. From the software viewpoint it is a distributed facility for the sharing of resources, facilities and information.

To gain an uniform analysis of such a system it will be assumed that certain algorithms are performed by the hardware as well as the software parts of the system. So we can say that from our viewpoint a distributed computer system is a complex system of cooperating algorithms (processes). During the task realization a user of a system creates a virtual network of processes. The virtual network of processes consists of a set of logically connected cooprocesses. Each of the coprocess for the given virtual process is executed in another processor of DCS.

To obtain a meaningful, in such a way understand system behaviour, it is necessary to introduce some kind of the synchronization and communication between the different components.

In this chapter we have presented a framework for the automatic generation of the synchronization and communication mechanisms for loosely coupled DCS, such that execution of processes will be feasible in optimal to certain criteria, manner. More precisely, we propose a new efficient method .of optimal allocation of tasks, to particulat processors, in order to m~n~m~ze ~ntercomm~nfcat~ons o/ coprocesses. This way it allows us to obtain a higher speed of the computing and to create the system higher reliable (the fault-avoidance).

We start with formal specification of processes in DCS - specification based on a modified version of well known traces. Traces were originally developed as a general technique for software specification. There they are modified in order to describe processes in distributed systems.

In this paper we restrict our

considerations, based on the example, to the illust?ation of our method, only.

There are two basic analysis and synthesis layers of multiprocessor networks to be distinguished. These two layers will be the foundation of our considerations.

These layers are the following structures of the system: physical (the hardware of the system) and logical (the software part of the system).

The most important realization aspect od such a system is the connection between circuit part and software part. This connection has to fulfill several assumptions to make the whole system sufficiently effective. These conditions are: optimal (usually maximum possible) speed of acting, fault-tolerance, high reliability, low cost, etc. These positive features are obtained through the mathematical analysis or numerical simulation of a model. The model of a considered system allows to express the most important, from the point of viev of our interests, properties.

The circuit part - physical structure - of a distributed system is represented by set of processors end set of interconnection lines between them in our way of analysis.

The second important part of our system, according to the presented method of analysis, is the software layer. The layer is built by system and user's programs.

The basic task of such a distributed system is a realization of computatlons initiated from arbitrary processor of network. All of the programs residing ane performed by the network will be treated equally as sequences of particular networks states and relations between them. This treatment is necessary for our suggested way of distributed system analysis. These states can be analysed as deep as to the particular physical

components of the network what depends

on the level of abstraction.

Our analysls method assume that the realization of a task is basing on the performance of stated algorithms in different network processors. Controlled relations between different network states are also required. The task defined in this way will be refered to as a virtual process of d~stributed system. The components of this process (succesive parts of the algorithms) can be realized in different Processors. The necessary condltion to realize the whole task is realization of a proper connection, in the physical structure of the network, between talking nodes. The topology of these connections in the set of coprocesses (parts of algorithms performed in particular

492 J. Just, R. Romaniuk / Distributed Systems with Optical Interconnections

processors) can change in time, together with the changes in the task. These connections and the set of a coprocesses create a log~cal~structuPe of the system.

The optimization of the realization process against assumed conditions and limitations consists in a kind of mutual changing balance between these two structures including mutual adaptation.

4. THE EXAMPLE OF LOGICAL CONTROL STRUCTURE SYNTHESIS

-Consider the three processors DCS as depicted in Fig.2 -We have given global job (the virtual process - VP) which DCS should perform (it forms a certain regular formal language)

VP = A(BC) D(EF) G

(A,B,C~D,E,F,G}- the set ofactions. Every action will mean (because of it abstract character) the program, the part of the program, the instruction, activity of hardware, etc.

o e

Fig.l. Graph of VP.

an

- As the result of automatic analysis of both: the word describing VP and the structure of multiprocessors system (the number of microcomputers and the configuration of interconnections). We find the following, an optimal allocation of particular actions of the virtual process to separate processors.

~PI D P:D F i g . 2 .

- On the basis of the above obtained the allocation and the virtual process description (the statement VP) we extend the last one, by adding interaction t..[.l-symbols, separatinf subwords 13

belonging to the different coprocesses (performed by different processing units), and next on the basis of this extended expression we find sequences of actions executed in particular processing units (coprocesses - CP.).

1

Interpretation of the action symbol t..[.] is as follows. The £..[.] denotes

I j 13 an action of transmission from i-th coprocess to j-th one, as well as contain information about a suspension of l-th component and activatlon or action placed in [.] brackets which belonging to j-th component.

Coprocesses:

CPI=A(tI2[B]+tlI[D])tlI[D]D(tI3[E]+tlI[G])

~II[G]GZlI[~] ~

CP2=BC(t21[D]+z22[B])

CP3=EF(t31[G]+t33[E])

Let graphically t-action be represented by double arrow. A resumption point of an activated component is indicated by arrow-head, and a point of the control transfering as well as the point of suspending of the sender is indicated by the arrow-begin. Thus graphically, coprocesses of our the system can be presented by the below graphs.

t13

~i C <~B ~2

21 [D ]

Fig.3.

F <iE ~3

~, t31 [G]

J. Just, R. Romaniuk /Distributed Systems with Optical Intereonnections 493

REFERENCES

[I] Eberbach E., Just J.R., A sell recon/(~urat(on as a basis o~ fau[t-~o[erance in d f s t r i b u t e d computer sNs£em~. Proc. of 6-th Intern. Conf. Fault-Tolerant Systems and Dlagnostlcs, Brno, Czechoslovakla,1983,pp.134-141.

[2] Eberbach E. Just J.R. z . . . . . ' ; ~ approadA tO tAe operatin~ s~stera~ in /a~lt-to[eP~nt dist~-(buted co~puter systems. Sec. Conf,on Fault-Tolerant Computing Systems, Bonn,1984, ~n:Informatik Fachberlchte, voi.84, Sprlnqer Verlag,pp.150-169 .

[3] Eberbach E., Just J.R., Ana[ys(s and syntAesfs o/ /auLt-toLerant computer sWstems. Elektron, Informat~onsverarb. u. Kybern. EIK 22, 1986,pp.385-409.

[6] Just J.R., Eberbach E.,TAe syntAes(s o/ contro[ a[~or~tAras /or /auLt-to[erant distrfbuted comp~te~ s~steras. Microprocessing and M1croprogramm]ng, vol.20, Nord-Holland, 1987,pp.25-32.

[7] Prucnal P.R.,et al, Speed spectrum /fber optic [oca~ area net~)orA usinM optical processi)%8" IEEE JLT, voi.4,1986, Pp.547-554.

[8] Romaniuk R.S.,Ffbre optic pulse sequencers/desequencers. Proc. of SPIE, (Bellingham WA), vol. 722, Sept.1986, pp.125-192.

[9] Smyth C.J,,Nonb[ocNin~ pAoton[c s~itcA netu~rks. IEEE SAC, vol.6, no 7, Aug.1988,pp.1052-1062.

[4] Just J.R., Eberbach E.,TAe synthesis o/ a[Mom(£hras /or /au[r-to[erant d~strfbuted co~putel-sys£ero.s. Microprocessing and Microprogramraing, vol.20, North-Holland,1987,pp.25-32.

[ 5 ] J u s t J . R . , D e s i ~ n i n 8 d f s t r £ b ~ t e d c o m p i l e r s y s t e m s : s c ( e n t i / f c app2"oach to~xzrds an a l g e b r a i c ~ o d e [ based r r ~ t h o d . P r o c . o f 5 - t h I n t e r n . C o n f . M i c r o c o m p . , M ] c r o a r a h . a n d D e v e l o p m e n t s , M i c r o s y s t e m 8 7 , C z e c h o s l o v a k i a , 1 9 8 7 .