curs 1 - babeș-bolyai universityvniculescu/didactic/ppd/c1.pdf · curs 1 - ppd - 10 limite ale...
TRANSCRIPT
![Page 1: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/1.jpg)
Curs1
ProgramareParalelasiDistribuita
Curs 1 - PPD - 2
![Page 2: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/2.jpg)
Continutulcursului(realizatsipebazapehttp://grid.cs.gsu.edu/~tcpp/curriculum/?q=home)
Teoretic• Notiuniintroductive:arhitecturi,
concurenta,paralelism• Etapeindezvoltareaprogramelorparalele• Evaluareaperformanteiprogramelor
paralele• Modeledeprogramareparalela
– Diferentaintrecelebazatepememoriepartajatasimemoriedistribuita
• Patterns– Ptprogramareparalela– Ptprogramaredistribuita
Practic• Javathreads(lowlevelAPI)• C++(>=C++11)threads• High-levelAPI:pacheteJava->
java.util.concurrentpackages.• Javastreams• OpenMP(C++)• CUDA(C++)• MPI–MessagePassingInterface
– exemplificariC,C++
Curs 1 - PPD - 3
![Page 3: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/3.jpg)
ProblemaConcurs
Alternativalaexamenulscris• Link:
https://drive.google.com/drive/folders/1qU77HrC-wG96x6EXtP2NeDs20hcpmecW?usp=sharing
• Nuesteobligatoriu!• Sepoateincercadarpentruvalidaretrebuierespectate
conditiile!
Curs 1 - PPD - 4
![Page 4: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/4.jpg)
Bibliografie• IanFoster.DesigningandBuildingParallelPrograms,Addison-Wesley1995.• BernaL.Massingill,TimothyG.Mattson,andBeverlyA.Sanders,AddisonAPatternLanguagefor
ParallelProgramming.WesleySoftwarePatternsSeries,2004.• MichaelMcCool,ArchRobinson,JamesReinders,StructuredParallelProgramming:Patternsfor
EfficientComputation,”MorganKaufmann,,2012.• D.Culler,J.PalSingh,A.Gupta.ParallelComputerArchitecture:AHardware/SoftwareApproach.
MorganKaufmann.1998.• Grama,A.Gupta,G.Karypis,V.Kumar.IntroductiontoParallelComputing,AddisonWesley,2003.• D.Grigoras.CalcululParalel.Delasistemelaprogramareaaplicatiilor.ComputerLibrisAgora,2000.• V.Niculescu.CalculParalel.Proiectaresidezvoltareformalaaprogramelorparalele.PresaUniv.
Clujana,2006.• B.Wilkinson,M.Allen,ParallelProgrammingTechniquesandApplicationsUsingNetworked
WorkstationsandParallelComputers,PrenticeHall,2002• A.Williams.C++ConcurrencyinActionPRACTICALMULTITHREADING.ManningPublisher.2012.• TutorialeJava:http://docs.oracle.com/javase/tutorial/essential/concurrency/further.html• C++11http://en.cppreference.com/w/• OpenMP:http://openmp.org/• MPI:http://www.mpi-forum.org/
Curs 1 - PPD - 5
![Page 5: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/5.jpg)
Evaluare
• Laborator(30%)>=5– Exercitiilaborator– ProgramefolosindJava/C++threads,OpenMP,MPI
• Seminar(10%)– Prezenta(7puncte<–7prezente)– Participareactiva[-1(insuficient),0(suficient),1(bine),2(foartebine)]
• Testpractic(multithreading)– 05%• ProiectCUDA-10%• Examen>=5
– Scris– sesiune40%
• Informatiicurs– http://www.cs.ubbcluj.ro/~vniculescu/didactic/PPD/CursPPD.html
6Curs 1 - PPD -
![Page 6: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/6.jpg)
ProcesareParalela
• Uncalculatorparalelesteuncalculator(sistem)carefolosestemultipleelementedeprocesaresimultanaintr-omanieracooperativapentruarezolvaoproblemacomputationala.
• ProcesareaParalelaincludetehnicisitehnologiicarefacposibilcalcululinparalel– Hardware,retele,SO,biblioteci,limbaje,compilatoare,algoritmi…
• Paralelismulestenatural.
• PERFORMANTA– Parallelismisverymuchaboutperformance!
7Curs 1 - PPD -
![Page 7: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/7.jpg)
Curs 1 - PPD - 8
CalculSerialvs.Paralel(imagesfromIntroductiontoParallelComputingBlaiseBarney)
![Page 8: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/8.jpg)
“Itwouldappearthatwehavereachedthelimitsofwhatitispossibletoachievewithcomputertechnology,althoughoneshouldbecarefulwithsuchstatements,astheytendtosoundprettysillyin5years.“(JohnvonNeumann,1949)
Curs 1 - PPD - 9
![Page 9: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/9.jpg)
Curs 1 - PPD - 10
Limitealeprogramariiseriale• Vitezadetransmisie–
• Vitezaluminii(30cm/nanosecond),limitadetransmisiepefirdecupru(9cm/nanosecond).
• Limitareaminiaturizarii–numardetrazistoripechip.– LegealuiMoore:
număruldetranzistoricarepotfiplasatipeunsingurcircuitintegrat(persquareinchchip)sedubleazalafiecare2ani.
– Darimpunecosturimari.
• Limitarieconomice
![Page 10: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/10.jpg)
Istoric
• CrestereaperformanteiprocesorprincrestereafrecventeiceasuluiCPU(CPUclockfrequency)– RidingMoore’slaw
• Probleme:incalzireaputernicaachipurilor!– Frecventaceasmaimare⇒consumelectricmaimare(Pentium4heatsink¦FryinganeggonaPentium4)
• Solutie–adugaremaimultorcore-uriptaajungelaperformantadorita– Sepastreazafrecventadeceaslafelsauchiarmicsorare– nucresteconsumul.
Curs 1 - PPD - 11
![Page 11: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/11.jpg)
Nivelurideparalelism
1.paralelismlaniveldejob:-intrejoburi;-intrefazealejoburilor;2.paralelismlaniveldeprogram:-intrepărţialeprogramului;-inanumitecicluri;3.paralelismlaniveldeinstrucţiune:
-intrediferitefazedeexecuţiealeuneiinstrucţiuni;4.paralelismlanivelaritmeticşilaniveldebit:- intreelementealeuneioperaţiivectoriale;- intrecircuitelelogiciiaritmetice.
Curs 1 - PPD - 12
![Page 12: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/12.jpg)
• Arhitecturilecurentesebazeazatotmaimultpeparalelismlanivelhardwarepentruaimbunatatiperformanta:
– Multipleexecutionunits– Pipelinedinstructions– Multi-core
Curs 1 - PPD - 13
![Page 13: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/13.jpg)
Paralelism<->Concurenta
• Considerammaimultetaskuricaretrebuieexecutatepeuncalculator• Taskurileseconsideraafipurparaleledaca:
– Sepotexecutainacelasitimp(parallelexecution)• Dependente->executieconcurenta:
– Untaskarenevoiederezultatelealtora;– Untasktrebuiesaseexecutedupaceoanumitaconditieeindeplinita– Maimultetaskuriincearcasafoloseascaaceeasiresursa
=>Formedesincronizaretrebuiefolositepentruasatisfaceconditiile/dependentele
• Concurentaestefundamentalaincomputerscience– Sistemedeoperare,bazededate,networking,…
14Curs 1 - PPD -
![Page 14: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/14.jpg)
Paralelismvs.Concurenta
15 Curs 1 - PPD -
Obs:– Sepotfolosithreadurisauproceseinambelecazuri– Dacauncalculparalelnecesitaacceslaresursecomuneatunci
estenevoiesasegestionezecorectconcurenta=>Paralelismulpoateimplicaconcurenta
Paralelism:Sefolosescmaimulteresursepentruarezolvaoproblemamairapid
resources
Concurenta:Gestiuneacorectasieficientaaaccesuluilaresursecomune
requestswork
resource
![Page 15: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/15.jpg)
ConcurentasiParalelism
• Concurentvs.paralel• ExecutieParalela:
– Taskurileseexecutaefectivinacelasitimp;
– Estenecesaraexistentademultipleresursedecalcul
• Paralelism=concurenta+hardware“paralel”
16Curs 1 - PPD -
![Page 16: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/16.jpg)
Paralelism
• Existamaimultenivelurideparalelism:– Procese,threads,routine,instructiuni,…
• Trebuiesafiesuportatederesurselehardware– Procesoare,nuclee(cores),…(executiainstructiunilor)– Memorii,DMA,retele,…(operatiiasociate)
17Curs 1 - PPD -
![Page 17: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/17.jpg)
oabordaresimplista
Curs 1 - PPD - 18
http://www.java-programming.info/tutorial/pdf/java/11-Java-Multithreaded-Programming.pdf
![Page 18: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/18.jpg)
Decesafolosimprogramareparalela?
• Motiveprimare:– Timpdecalculmairapid(responsetime)– Rezolvareaproblemelor‘mari’decalcul(intimprezonabildecalcul)
• Motivesecundare:– Folosireaefectivaaresurselordecalcul– Costurireduse– Reducereaconstrangerilorasociatememoriei– Limitarilemasinilorseriale
• Paralelism=concurenta+hardware‘paralel’+performanta
19Curs 1 - PPD -
![Page 19: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/19.jpg)
Curs 1 - PPD - 20
• Rezolvareaproblemelordificile,mari:– "GrandChallenge"(en.wikipedia.org/wiki/Grand_Challenge)problemsrequiring
PetaFLOPSandPetaBytesofcomputingresources.– Websearchengines/databasesprocessingmillionsoftransactionspersecond
• Folosirearesurselornon-locale:
– SETI@home(setiathome.berkeley.edu)usesover330,000computersforacomputepowerover528TeraFLOPS(asofAugust04,2008)
– Folding@home(folding.stanford.edu)usesover340,000computersforacomputepowerof4.2PetaFLOPS(asofNovember4,2008)
![Page 20: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/20.jpg)
Directiiinprocesareaparalela
• Arhitecturiparalele– NecesitatiHardware– Computersystemdesign
• Sistemedeoperare(Paralelism/concurenta)• Gestionareaaspectelorsistempentruuncalculatorparalel• Programareparalela
– Biblioteci(low-level,high-level)– Limbaje– Mediidedezvoltare– Software
• AlgoritmiParaleli• Evaluareaperformanteiprogramelorparalele• Testareavs.asigurareacorectitudinii• Paralleltools:
– Performanta,analize,vizualizare,…
21Curs 1 - PPD -
![Page 21: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/21.jpg)
Decesastudiemprogramareparalela?
• Arhitecturidecalcul– Inovatiileconduclanoimodeledeprogramare
• Convergentatehnologica– “killermicro”estepestetot– Laptop-urilesisupercomputeresuntfundamentalsimilare– Trend-urileactualeconduclaconvergentaabordarilordiverse
• Trenduriletehnologicefaccalcululparalelinevitabil– Multi-coreprocessors!– Acumoricesistemdecalculesteparalel
• Intelegereaprincipiilorfundamentale!!!– Programare,comunicatii,memorie,…– Performanta
• “Parallelismisthefutureofcomputing”-BlaiseBarney– M.Andrews,J.S.Walicki.“Concurrencyandparallelism—futureofcomputing”in
ProceedingofACM'85Proceedingsofthe1985ACMannualconferenceonTherangeofcomputing:mid-80'sperspective.pp.224-231.
22Curs 1 - PPD -
![Page 22: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/22.jpg)
InevitabilitateaProcesariiParalele
• Cerinteleptaplicatii– Necesitateauriasadecicluridecalcul
• Trenduritehnologice– Procesaresimemorie
• TrenduriArchitecturale• Factorieconomici• Treduriactuale:
– Today’smicroprocessorshavemultiprocessorsupport– Serversandworkstationsavailableasmultiprocessors– Tomorrow’smicroprocessorsaremultiprocessors– Multi-coreisheretostayand#cores/processorisgrowing– Accelerators(GPUs,gamingsystems)
23Curs 1 - PPD -
![Page 23: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/23.jpg)
exemple…
• ProcesorAMDRyzenThreadripper1950X~1300USD(2017)– MemorieCache40MB– Frecventaprocesor(MHz)3500– TurboBoostpanala 4000MHz– Numarnuclee 16Nuclee=>32Threads
• Intel®Xeon®ProcessorE7v4Family~8000USD(2017)
– #ofCores=24– #ofThreads=48– ProcessorBaseFrequency=2.40GHz– MaxTurboFrequency=3.40GHz– Cache=60MB
Curs 1 - PPD - 24
![Page 24: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/24.jpg)
UBBCLUSTER–IBMIntelligentCluster• Hybridarchitecture
– HPCsystem+– privatecloud
PPD-curs2 25
![Page 25: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/25.jpg)
HPC – IBM NextScale
• Rpeak 62 Tflops, Rmax 40 Tflops • 68 noduri NX360 M5, din care
– 12 nodes with 2 GPU Nvidia K40X, – 6 nodes with Intel Phi
• 2 processors E5-2660 v3 with 10Cores per node • 128 GB RAM per node, 2 HDD SATA de 500 Gb / node • Subscription rate 1:1 between nodes based on Switch: IB Melanox SX6512
with 216 ports • Storage NetApp E5660, 120 HDD SAS cu 600 Gb/Hdd => total 72Tb
– IBM GPFS 4.x -parallel file system
• IBM TS3100 Tape library for data archivation • Operating systems on each node : RedHat Linux 6 with subscription • Management Software: IBM Platform HPC 4.2
PPD-curs2 26
![Page 26: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/26.jpg)
Private Cloud – IBM Flex System
• 10 virtualization servers Flex System x240 – 128 Gb RAM / server – Procesoare 2 x Intel Xeon E5-2640 v2 / server – 2 x SSD SATA 240 Gb / server
• 1 management server • Software for private cloud: IBM cloud manager with OpenStack 4.2 • Software for monitorizing and management: IBM Flex System Manager
software stack • Virtualization software: Vmware vSphere Enterprise 5.1
PPD-curs2 27
![Page 27: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/27.jpg)
Mini cluster – IBM nestscale
~90cores
• 4nodesx2processorsX10cores• 1Managementnode• IP:193.226.40.133
• NOTA– Rmax-MaximalLINPACK(benchmark)performanceachieved– Rpeak-Theoreticalpeakperformance.
• NodeperformanceinGFlops=(CPUspeedinGHz)x(numberofCPUcores)x(CPUinstructionpercycle)x(numberofCPUspernode)
28PPD-curs2
![Page 28: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/28.jpg)
Aplicatiiinformaticeperformante
• Performantaaplicatiilorimpunehardwareperformant(rapid,resursemultiple,etc)
• Hardware-ulavansatgenereazanoiaplicatii• Noileaplicatiiaucererideperformantamaimari
– Crestereexponentialaaperformanteimicroprocesoarelor– Inovatiiinarhitecturileparalelesiinintegrare
• Cerintedeperformanta=>– Performantasistemelortrebuiesaseimbunatateascainansamblu– Schimbari/abordari/reevaluariinSoftwareengineering– Costuri-technologieavansata
applicationsperformance
hardware
29Curs 1 - PPD -
![Page 29: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/29.jpg)
Curs 1 - PPD - 30
Programareparalelavs.Programaredistribuita
INTERCONNECTION NETWORK
P2
P3
P1
P4 P5
Pn . . . .
![Page 30: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/30.jpg)
Curs 1 - PPD - 31
TiPURI DE MULTIPROCESARE PARALLEL DISTRIBUTED
ASPECTE TEHNICE • PARALLEL COMPUTERS (- IN MOD UZUAL ) LUCREAZA BAZAT PE
• CUPLARE STRANSA, • in general bazate pe SINCRONICITATE, • CU UN SISTEM DE COMUNICATIE FOARTE RAPID SI FIABIL • Spatiu unic de adresare (intr-o masura mare)
• DISTRIBUTED COMPUTERS • MAI INDEPENDENTE, • COMUNICATIE MAI PUTIN FRECVENTA SI mai putin RAPIDA (ASINCRONA) • COOPERARE LIMITATA • NU EXISTA CEAS GLOBAL • “Independent failures”
SCOPURI
• PARALLEL COMPUTERS COOPEREAZA PENTRU A REZOLVA MAI EFICIENT PROBLEME DIFICILE
• DISTRIBUTED COMPUTERS AU SCOPURI INDIVIDUALE SI ACTIVITATI PRIVATE. DOAR UNEORI INTERCOMUNICAREA ESTE NECESARA
PARALLEL COMPUTERS: COOPERARE IN SENS “POSITIV”
DISTRIBUTED COMPUTERS: COOPERARE IN SENS “NEGATIV” -- DOAR ATUNCI CAND ESTE NECESARA
![Page 31: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/31.jpg)
Curs 1 - PPD - 32
Aplicatii paralele Suntem interesati sa rezolvam problemele mai rapid in paralel
Aplicatii distribuite
Suntem interesati sa rezolvam anumite probleme specifice :
• COMMUNICATION SERVICES ROUTING BROADCASTING
• MAINTENANCE OF CONTROL STUCTURE TOPOLOGY UPDATE LEADER ELECTION • RESOURCE CONTROL ACTIVITIES LOAD BALANCING MANAGING GLOBAL DIRECTORIES
Ingeneral…
![Page 32: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/32.jpg)
Curs 1 - PPD - 33
Parallelv.s.DistributedSystems(fromM.FUKUDACSS434SystemModels)
Parallel Systems Distributed Systems
Memory Tightly coupled shared memory UMA, NUMA
Distributed memory Message passing, RPC, and/or used of distributed shared memory
Control Global clock control SIMD, MIMD
No global clock control Synchronization algorithms needed
Processor interconnection
Order of Tbps Bus, mesh, tree, mesh of tree, and hypercube (-related) network
Order of Gbps Ethernet(bus), token ring and SCI (ring), myrinet(switching network)
Main focus Performance Ex. - Scientific computing
Performance(cost and scalability) Reliability/availability Information/resource sharing
Unpunctdevedere…
![Page 33: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/33.jpg)
Curs 1 - PPD - 34
SistemeleDistribuite
-potfifolositepentru-
• Aplicatiidistribuiteimplicit– BDDistribuite,rezervaribilieteavion/etc.sistembancar
• Informatiipartajateintreuseri• Partajareresurse• Raportcost/performantamaibunptaplicatiiparalele
– Potfifolositeeficientpt.aplicatiicugranularitatemare(coarse-grained)si/sauptaplicatiiparaleledetipembarrassinglyparallelapplications
• Fiabilitate(Reliability).• Scalabilitate
– Cuplareslaba(Looselycoupledconnection);hotplug-in
• Flexibilitate– Reconfiguraresistemptaintrunicerintele
![Page 34: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/34.jpg)
Curs 1 - PPD - 35
Performanta/Scalabilitate
Spredeosebiredesistemeleparaleleceledistribuiteimplica:- mediumaiputinrapiddetransferaldatelor(reteamaiputinrapida)- HeterogenitateSolutii:-Procesarebatchamesajelor:
• SeevitainterventiaSOptfiecaretransferdemesaj.– Cachedata
• Seevitarepetareatransferuluiaceleiasidate– Evitateaentitatilorsiaalgoritmilorcentralizati
• Evitareasaturariiretelei– Realizareoperatii“post”lanivelulclientului
• Evitareatraficuluiintensintreclientisiservere
– ….
![Page 35: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/35.jpg)
Curs 1 - PPD - 36
Securitate
• Nuexistadoarunsingurpunctdecontrol
• Probleme:– Mesaje,furate,modificate,copiate,…
• Solutie:folosireCriptografie– Failures
• FaultTolerancesolutions
![Page 36: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/36.jpg)
Tipuridesistemedistribuite(Munehiro Fukuda – PDP Fundamentals)
• Modele– Minicomputer– Workstation– Workstation-server– Processor-pool– Cluster– Gridcomputing
Curs 1 - PPD - 37
![Page 37: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/37.jpg)
38
ModelulMinicomputer
• ExtensieasistemuluideTimesharing– Useriitrebuiesaseloghezepepropriulminicomputer.– Apoiselogheazalaaltamasina(remotemachine)prinprogramede
tiptelnet.• Partajareresurse
– DB– High-performancedevices
Mini-computer
Mini-computer
Mini-computer
net
Curs 1 - PPD -
![Page 38: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/38.jpg)
Curs 1 - PPD - 39
ModelulWorkstation
• MigrareProcese– Useriiselogheazamaiintaipestatiadelucrupersonala;– Dacaexistastatii“inasteptare”–unjob“mare”poatemigrala
unadintreele.• Probleme:
– Cumseidentificastatiile“inasteptare”(idle)?– Cummigreazaunjob?– Ceseintampladacaunaltuserselogheazapemasinafolosita?
100MbpsLAN
Workstation
Workstation Workstation
WorkstationWorkstation
![Page 39: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/39.jpg)
Curs 1 - PPD - 40
ModelulWorkstation-Server
• StatiiClient– AplicatiileGrafice/interactiveseproceseazalocal– Ptaltecereridecalculsetrimitcererilaservere.
• Servere(minicomputers)– Fiecareserverestededicatunuiasaumaimultor
tipurideservicii.• Modeldecomunicare:Client-Servermodel
– RPC(RemoteProcedureCall)– RMI(RemoteMethodInvocation)
• Unprocesclientcheamaofunctieaprocesuluiserver.
• Nusefacemigraredeprocese• Examplu:NFS
100GbpsLAN
Workstation
Workstation Workstation
Mini-Computerfileserver
Mini-Computerhttpserver
Mini-Computercycleserver
![Page 40: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/40.jpg)
Curs 1 - PPD - 41
ModelulProcessor-Pool
• Clientii:– Selogheazalaunterminal– Toateserviciilesuntgestionatede
catreservere.• Servere:
– Ptfiecareusersealocanrnecesardeprocesoaredinpool.
• Utilizarebunadarinteractivitateslaba.
Server1
100MbpsLAN
ServerN
![Page 41: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/41.jpg)
Curs 1 - PPD - 42
ClusterModel
• ConstainmaimultePC/workstationsconectatelaoreteadetiphigh-speed.
• Focuspeperformanta.100Mbps
LAN
Workstation
Workstation Workstation
Masternode
Slave1
SlaveN
Slave2
1GbpsSAN
httpserver1
httpserver2
httpserverN
![Page 42: Curs 1 - Babeș-Bolyai Universityvniculescu/didactic/PPD/C1.pdf · Curs 1 - PPD - 10 Limite ale programarii seriale • Viteza de transmisie – • Viteza luminii (30 cm/nanosecond),](https://reader033.vdocuments.site/reader033/viewer/2022041813/5e595158445c09595f554568/html5/thumbnails/42.jpg)
Curs 1 - PPD - 43
High-speedInformationhighway
GridComputing
• Scop– Colectareaputeridecalculamaimultor
supercomputeresiclusteredispersategeografic
• DistributedSupercomputing– Ptproblemwfoartemari.Dificile.
(CPUintensive,memoryintensive).• High-ThroughputComputing
– Folosireamultorresursecarenusuntfolosite
• On-DemandComputing– Resurseladistantaintegrateincalculullocal
• Data-intensiveComputing– distributeddata
• CollaborativeComputing– Suportptcomunicareintremaimulteparti
Super-computer
Cluster
Super-computer Cluster
Mini-computer
Workstation
Workstation Workstation