DistributedSystems

Communica3onandmodels

RikSarkarSpring2018

UniversityofEdinburgh

Models

•  Expecta3ons/assump3onsaboutthings•  Everyideaorac3onanywhereisbasedonamodel

•  Determineswhatcanorcannothappen

2

Communica3on&modeling•  Modelingdistributedsystems:– Howwecanthinkaboutthem

•  Communica3onbetweennearbynodes•  Communica3onbetweendistantnodes•  Communica3onwithmanynodesSometerminology:•  Oneto“all”:Broadcast– All:insomesetofinterest

•  Onetoone:pointtopoint3

Packets

•  Networkscommunicatedatainmessagesoffixed(bounded)size–calledpackets

•  Moredatarequiresmorepackets

•  NumberofmessagesorpacketstransmiWedisameasureofcommunica3onused

4

Localareanetworks•  Medium:Broadcast

– Messagegoesfromonecomputertoallothercomputers(restrictedtosomeset)•  Forexample,allothercomputersintheLAN,orsomeothersysteminconsidera3on

–  EthernetLANisabroadcastmedium•  Allcomputersareconnectedtoawire.Theytransmitmessagesonthewireandallcanreceive

– WirelessLAN(WiFi)isabroadcastmedium•  Electromagne3cwavesisthecommonmedium

5

Localareanetworks

Advantages:•  Sendingacommonmessagetoeveryoneiseasy•  Findingdes3na3oniseasy– Messagegoestoeveryone–  Justhavea“des3na3on”field

Mainissue:Mediumaccess•  Sincemediumisbroadcast,twopeopletransmiangatthesame3megarblesmessage

6

Mediumaccess•  Onlyonetransmissionata3mecanbeallowed– Mutualexclusionproblem(sharedresourceofcommunica3onwire)

– Wecannotusemessagestosolveit–  Protocols:

•  TDMA:Everyonehasaperiodicslot•  CSMA:Seeifanyoneelseistransmiang.Ifso,defer.•  Usuallyacksarealsousedtoensuretransmission

–  Retransmitifnecessary

–  Bandwidthreduceswithnumberofnodestryingtotransmit.•  OneLANshouldnothavetoomanynodes

7

Mediumaccess

•  Wireless:morecomplicated•  Hiddenterminalproblem•  Morecomplexprotocolusingacks

8

OurmodelsofLAN

•  Graph:Everynodehasanedgetoeveryother

•  Weofenassumethattosendamessage(packet)toanodeonthesamenetworktakesoneunitof3me(or,atmostaconstant)

•  ThismaynotbetrueiftherecanbemanynodesinthesameLAN– Butusuallythenumberisnotverylarge

9

Reallifenetworks

•  LANsconnectedbyrouters

Ethernet/WifiEthernet/Wifi

PointtoPoint

Routers

10

1 4

2 4

3 4

4 4

1 3

2 3

3 3

5 5

1 1

2 2

4 4

5 4

2 2

3 3

4 3

5 3

Rou3ng•  Findingapathinthenetwork•  Everynodehasarou3ngtable•  EquivalenttoaBFStreeforeverynode

1

2

4

3

5

1 1

3 3

4 3

5 3 11

1-4 4

1-3 3

5 5

1 1

2 2

4-5 42 2

3-5 3

Rou3ng:Distributedsearchforapath

•  Smallerrou3ngtablesbycombiningaddresses•  UsedinIP(Internet)rou3ng•  Smallerrou3ngtablesarepreferable

1

2

4

3

5

1 1

3-5 3

12

Rou3ng

•  Realrou3ngismorecomplicated•  Withmorethanonepathtoades3na3on,backupsetc

13

Related:Loca3onbasedrou3ng

•  Geographicalrou3ngusesanode’sloca3ontodiscoverpathtothatnode.

x

y

GreedyRou3ng:Forwardtotheneighborthatisnearesttothedes3na3on

14

Largenetworks•  Communica3onistypicallypoint-to-pointusingrou3ng

•  Broadcastisnotautoma3c–  Ifweneedbroadcast,wewillhavetoarrangeaflood(orsomeothermethod)

Ethernet/WifiEthernet/Wifi

PointtoPoint

Routers

15

Transportmanagement•  UDP:

–  Sendapacket,hopethatthenetworkroutesanddeliversit,in3me

–  NoSequencenumber•  NotnecessarilyFIFO

–  Usefulinstreamingaudio/video.Notforimportantdata.

•  TCP:–  Sendapacket(orfewpackets)–  Packetshavesequencenumber

•  FIFO–  Ifnoacksarrive,resendpackets–  Ifnoacksarefoundafermanytries,returnerror

16

TCP

•  Doesdistributedconges3oncontrol– Whenpacketsdon’tgetdelivered,TCPslowsdownthestream•  Assump3on:routersdroppacketswhentherearetoomany

•  Difficulty– Acksmaynotarriveduetootherfactors•  Someconnec3onfailedtemporarily•  Usermovedfromonenetworktoanother

17

Networkstack

•  Eachlayersolvesadifferentdistributedproblem

18

Formwikipedia

Communica3on:Overlaynetwork•  Wemaysome3mesignorepartsofthenetwork

–  Nodesthatcarrymessagesbutdonotdirectlypar3cipateeg.routers

–  Oredgesthatexistbutwearenotusing–  Orwedon’tknowabout

•  Ofenusedinpeer-to-peernetworks–  Noteverynodeknowsallothernodesinthenetwork–  Butcommunicatestoknownnodesthroughrou3ng

19

Blockingandnon-blockingcommunica3on

•  Blockingcommunica3on–  Sendersendsmessage,waitsun3lreceiverreplies–  Doesnotdoanythinginthemean3me

•  Non–blockingcommunica3on–  Sendersendsmessage,thencon3nuesitsownworkwithoutwai3ng

– Whenreceiverreplies,orsomeothermessagearrives,nodeinterruptscurrentworktohandlemessage

•  Some3mesthesearecalledsynchronous/asynchronous,butwewilltrynottousethat

20

Computa3on•  Synchronous:– Opera3oninrounds–  Inaround,anodeperformssomecomputa3on,andthensendssomemessages

– Allmessagessentattheendofroundxareavailabletorecipientsatstartofroundx+1•  Butnotearlier

21

Computa3on

Communica3on

Round1 Round2 Round3

Communica3on

•  Synchronous– Canbeimplementedifmessagetransmission3meisboundedbysomeconstantsaym

– Computa3on3mesforallnodesareboundedbysomeconstantc

– Clocksaresynchronized(sufficiently)– Thenseteachroundtobem+cindura3on

22

AsynchronousCommunica3on

•  Nosynchroniza3onorrounds– Nodescomputeatdifferentandarbitraryspeeds– Messagesproceedatdifferentspeeds:maybearbitrarilydelayed,maybereceivedatany3me

•  Worstcasemodel– Noassump3onaboutspeedsofprocessesorchannel–  (Butdoesnotincludecommunica3on/computa3onerrors)

23

AsynchronousCommunica3on

•  Hardertomanage– Messagecanarriveatany3meaferbeingsent,mustbehandledsuitably

– Possibletomakesomesimplifyingassump3onsE.g.:•  ChannelsareFIFO:orderofmessagesonachannelarepreserved•  Somecodeblocksareatomic(notinterruptedbymessages)•  Eithercommunica3onorcomputa3on3mesbounded

24

Synchronouscommunica3oninRealsystems

•  Synchronouscommunica3oncanbeafairmodel•  Moderncomputersandnetworksarefast–  (thoughnotarbitrarilyfast)

•  Easiertodesignalgorithmsandanalyze•  Welldesignedalgorithmsarefasterandmoreefficient

•  Ofencanbeadaptedtoasynchronoussystems– Ofenastar3ngpointfordesign

25

Failures

•  Nodesmayfail– Hardwarefailure– Runoutofenergyorpowerfailure– Sofwarefailure(crash)– Permanent– Temporary(whathappenswhenitrestarts?Recoversthestate?Startsfromini3alstate?)

– Modeldependsonsystem.E.g.differenttypesoffailuresoccurwithcorrespondingprobabili3es

26

Nodefailures•  Commonabstractmodels

–  Stoppingfailure:nodejuststopsworking•  Mayneedassump3onsaboutwhichcomputa3on/communica3onitfinishesbeforestopping

•  Mayneedassump3onaboutneighborsknowingoffailure–  Byzan3nefailure:nodebehavesasanadversary

•  Imagineyourenemyhastakencontrolofthenode•  Istryingtospoilyourcomputa3on

•  Nodesmayfailindividually–  E.g.eachnodefailswithprobabilityp

•  Nodesmayhavecorrelatedfailure–  E.g.allnodesfailinaregion(datacenter,sensorfield)

27

Link/communica3onfailure•  Maybetemporary/permanent•  Mayhappendueto

–  Hardwarefailure–  Noise:electronicdevices(microwavesetc)maytransmitradiowavesatsimilarfrequenciesanddisruptcommunica3on

–  Interference:Othercommunica3ngnodesnearbymaydisruptcommunica3on

•  Effects–  Channelsilentandunusable(hardwarefailure)–  Channelac3ve,butunusableduetonoiseandinterference–  Channelac3ve,butmaycontainerroneousmessage(maybedetectedbyerrorcorrec3ngcodes)

28

Security•  Issues:– Unauthorizedaccess,modifica3on.Makingsystemsunavailable(DOS)

– AWackononeormorenodes•  Causingtoitfail•  Readdata•  Takingcontroltoreadfuturedata,disruptopera3on

– AWackoncommunica3onlinks/channel•  Blockcommunica3on•  Readdatainthechannel(easyinwirelesswithoutencyp3on)

•  Corruptdatainthechannel

29

Security

•  Solu3onsusuallyhavespecificassump3onsofwhattheadversarycando

•  E.g.Ifadversaryhasaccesstochannel– Cryptographymaybeabletopreventreading/corrup3ngdata

30

Mobility

•  Movementmakesithardertodesigndistributedsystems–  Communica3onisdifficult

•  Delays,lostmessages•  Edgeweightscanchange

– Applica3onsthatdependonloca3onmustadapttomovement

•  Howdopeoplemove?Whatisamodelofmovement?– Notyetwellunderstood

31

Modelingdistributedsystems

•  Manypossibili3es•  Chooseyourassump3onscarefullyforyourproblem

•  PaycloseaWen3ontowhatisknownaboutcommunica3on/network

•  Startwithsimplermodels– Usuallymoreassump3ons,fewerparameters– Seewhatcanbeachieved– Thentrytodrop/relaxassump3ons

32

33