curs 1vniculescu/didactic/ppd/suport_curs/c... · 2020. 10. 22. · curs 1 - ppd - 5 evaluare •...
TRANSCRIPT
![Page 1: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/1.jpg)
Curs1
ProgramareParalelasiDistribuita
Curs 1 - PPD - 2
![Page 2: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/2.jpg)
Continutulcursului(realizatsipebazapehttp://grid.cs.gsu.edu/~tcpp/curriculum/?q=home)
Teoretic• Notiuniintroductive:arhitecturi,
concurenta,paralelism• Etapeindezvoltareaprogramelorparalele• Evaluareaperformanteiprogramelor
paralele• Modeledeprogramareparalela
– Diferentaintrecelebazatepememoriepartajatasimemoriedistribuita
• Patterns– Ptprogramareparalela– Ptprogramaredistribuita
Practic• Javathreads(lowlevelAPI)• C++(>=C++11)threads• High-levelAPI:pacheteJava->
java.util.concurrentpackages.• Javastreams• OpenMP(C++)• CUDA(C++)• MPI–MessagePassingInterface
– exemplificariC,C++
Curs 1 - PPD - 3
![Page 3: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/3.jpg)
ProblemaConcurs
Alternativalaexamenulscris• Link:
https://drive.google.com/drive/folders/1qU77HrC-wG96x6EXtP2NeDs20hcpmecW?usp=sharing
• Nuesteobligatoriu!• Sepoateincercadarpentruvalidaretrebuierespectate
conditiile!
Curs 1 - PPD - 4
![Page 4: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/4.jpg)
Bibliografie• IanFoster.DesigningandBuildingParallelPrograms,Addison-Wesley1995.• BernaL.Massingill,TimothyG.Mattson,andBeverlyA.Sanders,AddisonAPatternLanguagefor
ParallelProgramming.WesleySoftwarePatternsSeries,2004.• MichaelMcCool,ArchRobinson,JamesReinders,StructuredParallelProgramming:Patternsfor
EfficientComputation,”MorganKaufmann,,2012.• D.Culler,J.PalSingh,A.Gupta.ParallelComputerArchitecture:AHardware/SoftwareApproach.
MorganKaufmann.1998.• Grama,A.Gupta,G.Karypis,V.Kumar.IntroductiontoParallelComputing,AddisonWesley,2003.• D.Grigoras.CalcululParalel.Delasistemelaprogramareaaplicatiilor.ComputerLibrisAgora,2000.• V.Niculescu.CalculParalel.Proiectaresidezvoltareformalaaprogramelorparalele.PresaUniv.
Clujana,2006.• B.Wilkinson,M.Allen,ParallelProgrammingTechniquesandApplicationsUsingNetworked
WorkstationsandParallelComputers,PrenticeHall,2002• A.Williams.C++ConcurrencyinActionPRACTICALMULTITHREADING.ManningPublisher.2012.• TutorialeJava:http://docs.oracle.com/javase/tutorial/essential/concurrency/further.html• C++11http://en.cppreference.com/w/• OpenMP:http://openmp.org/• MPI:http://www.mpi-forum.org/
Curs 1 - PPD - 5
![Page 5: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/5.jpg)
Evaluare
• Laborator(30%)>=5– Exercitiilaborator– ProgramefolosindJava/C++threads,OpenMP,MPI
• Seminar(10%)– Prezenta(7puncte<–7prezente)– Participareactiva[-1(insuficient),0(suficient),1(bine),2(foartebine)]
• Testpractic(multithreading)– 05%• ProiectCUDA-10%• Examen>=5
– Scris– sesiune40%
• Informatiicurs– http://www.cs.ubbcluj.ro/~vniculescu/didactic/PPD/CursPPD.html
6Curs 1 - PPD -
![Page 6: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/6.jpg)
ProcesareParalela
• Uncalculatorparalelesteuncalculator(sistem)carefolosestemultipleelementedeprocesaresimultanaintr-omanieracooperativapentruarezolvaoproblemacomputationala.
• ProcesareaParalelaincludetehnicisitehnologiicarefacposibilcalcululinparalel– Hardware,retele,SO,biblioteci,limbaje,compilatoare,algoritmi…
• Paralelismulestenatural.
• PERFORMANTA– Parallelismisverymuchaboutperformance!
7Curs 1 - PPD -
![Page 7: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/7.jpg)
Curs 1 - PPD - 8
CalculSerialvs.Paralel(imagesfromIntroductiontoParallelComputingBlaiseBarney)
![Page 8: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/8.jpg)
“Itwouldappearthatwehavereachedthelimitsofwhatitispossibletoachievewithcomputertechnology,althoughoneshouldbecarefulwithsuchstatements,astheytendtosoundprettysillyin5years.“(JohnvonNeumann,1949)
Curs 1 - PPD - 9
![Page 9: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/9.jpg)
Curs 1 - PPD - 10
Limitealeprogramariiseriale• Vitezadetransmisie–
• Vitezaluminii(30cm/nanosecond),limitadetransmisiepefirdecupru(9cm/nanosecond).
• Limitareaminiaturizarii–numardetrazistoripechip.– LegealuiMoore:
număruldetranzistoricarepotfiplasatipeunsingurcircuitintegrat(persquareinchchip)sedubleazalafiecare2ani.
– Darimpunecosturimari.
• Limitarieconomice
![Page 10: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/10.jpg)
Istoric
• CrestereaperformanteiprocesorprincrestereafrecventeiceasuluiCPU(CPUclockfrequency)– RidingMoore’slaw
• Probleme:incalzireaputernicaachipurilor!– Frecventaceasmaimare⇒consumelectricmaimare(Pentium4heatsink¦FryinganeggonaPentium4)
• Solutie–adugaremaimultorcore-uriptaajungelaperformantadorita– Sepastreazafrecventadeceaslafelsauchiarmicsorare– nucresteconsumul.
Curs 1 - PPD - 11
![Page 11: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/11.jpg)
Nivelurideparalelism
1.paralelismlaniveldejob:-intrejoburi;-intrefazealejoburilor;2.paralelismlaniveldeprogram:-intrepărţialeprogramului;-inanumitecicluri;3.paralelismlaniveldeinstrucţiune:
-intrediferitefazedeexecuţiealeuneiinstrucţiuni;4.paralelismlanivelaritmeticşilaniveldebit:- intreelementealeuneioperaţiivectoriale;- intrecircuitelelogiciiaritmetice.
Curs 1 - PPD - 12
![Page 12: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/12.jpg)
• Arhitecturilecurentesebazeazatotmaimultpeparalelismlanivelhardwarepentruaimbunatatiperformanta:
– Multipleexecutionunits– Pipelinedinstructions– Multi-core
Curs 1 - PPD - 13
![Page 13: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/13.jpg)
Paralelism<->Concurenta
• Considerammaimultetaskuricaretrebuieexecutatepeuncalculator• Taskurileseconsideraafipurparaleledaca:
– Sepotexecutainacelasitimp(parallelexecution)• Dependente->executieconcurenta:
– Untaskarenevoiederezultatelealtora;– Untasktrebuiesaseexecutedupaceoanumitaconditieeindeplinita– Maimultetaskuriincearcasafoloseascaaceeasiresursa
=>Formedesincronizaretrebuiefolositepentruasatisfaceconditiile/dependentele
• Concurentaestefundamentalaincomputerscience– Sistemedeoperare,bazededate,networking,…
14Curs 1 - PPD -
![Page 14: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/14.jpg)
Paralelismvs.Concurenta
15 Curs 1 - PPD -
Obs:– Sepotfolosithreadurisauproceseinambelecazuri– Dacauncalculparalelnecesitaacceslaresursecomuneatunci
estenevoiesasegestionezecorectconcurenta=>Paralelismulpoateimplicaconcurenta
Paralelism:Sefolosescmaimulteresursepentruarezolvaoproblemamairapid
resources
Concurenta:Gestiuneacorectasieficientaaaccesuluilaresursecomune
requestswork
resource
![Page 15: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/15.jpg)
ConcurentasiParalelism
• Concurentvs.paralel• ExecutieParalela:
– Taskurileseexecutaefectivinacelasitimp;
– Estenecesaraexistentademultipleresursedecalcul
• Paralelism=concurenta+hardware“paralel”
16Curs 1 - PPD -
![Page 16: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/16.jpg)
Paralelism
• Existamaimultenivelurideparalelism:– Procese,threads,routine,instructiuni,…
• Trebuiesafiesuportatederesurselehardware– Procesoare,nuclee(cores),…(executiainstructiunilor)– Memorii,DMA,retele,…(operatiiasociate)
17Curs 1 - PPD -
![Page 17: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/17.jpg)
oabordaresimplista
Curs 1 - PPD - 18
http://www.java-programming.info/tutorial/pdf/java/11-Java-Multithreaded-Programming.pdf
![Page 18: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/18.jpg)
Decesafolosimprogramareparalela?
• Motiveprimare:– Timpdecalculmairapid(responsetime)– Rezolvareaproblemelor‘mari’decalcul(intimprezonabildecalcul)
• Motivesecundare:– Folosireaefectivaaresurselordecalcul– Costurireduse– Reducereaconstrangerilorasociatememoriei– Limitarilemasinilorseriale
• Paralelism=concurenta+hardware‘paralel’+performanta
19Curs 1 - PPD -
![Page 19: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/19.jpg)
Curs 1 - PPD - 20
• Rezolvareaproblemelordificile,mari:– "GrandChallenge"(en.wikipedia.org/wiki/Grand_Challenge)problemsrequiring
PetaFLOPSandPetaBytesofcomputingresources.– Websearchengines/databasesprocessingmillionsoftransactionspersecond
• Folosirearesurselornon-locale:
– SETI@home(setiathome.berkeley.edu)usesover330,000computersforacomputepowerover528TeraFLOPS(asofAugust04,2008)
– Folding@home(folding.stanford.edu)usesover340,000computersforacomputepowerof4.2PetaFLOPS(asofNovember4,2008)
![Page 20: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/20.jpg)
Directiiinprocesareaparalela
• Arhitecturiparalele– NecesitatiHardware– Computersystemdesign
• Sistemedeoperare(Paralelism/concurenta)• Gestionareaaspectelorsistempentruuncalculatorparalel• Programareparalela
– Biblioteci(low-level,high-level)– Limbaje– Mediidedezvoltare– Software
• AlgoritmiParaleli• Evaluareaperformanteiprogramelorparalele• Testareavs.asigurareacorectitudinii• Paralleltools:
– Performanta,analize,vizualizare,…
21Curs 1 - PPD -
![Page 21: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/21.jpg)
Decesastudiemprogramareparalela?
• Arhitecturidecalcul– Inovatiileconduclanoimodeledeprogramare
• Convergentatehnologica– “killermicro”estepestetot– Laptop-urilesisupercomputeresuntfundamentalsimilare– Trend-urileactualeconduclaconvergentaabordarilordiverse
• Trenduriletehnologicefaccalcululparalelinevitabil– Multi-coreprocessors!– Acumoricesistemdecalculesteparalel
• Intelegereaprincipiilorfundamentale!!!– Programare,comunicatii,memorie,…– Performanta
• “Parallelismisthefutureofcomputing”-BlaiseBarney– M.Andrews,J.S.Walicki.“Concurrencyandparallelism—futureofcomputing”in
ProceedingofACM'85Proceedingsofthe1985ACMannualconferenceonTherangeofcomputing:mid-80'sperspective.pp.224-231.
22Curs 1 - PPD -
![Page 22: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/22.jpg)
InevitabilitateaProcesariiParalele
• Cerinteleptaplicatii– Necesitateauriasadecicluridecalcul
• Trenduritehnologice– Procesaresimemorie
• TrenduriArchitecturale• Factorieconomici• Treduriactuale:
– Today’smicroprocessorshavemultiprocessorsupport– Serversandworkstationsavailableasmultiprocessors– Tomorrow’smicroprocessorsaremultiprocessors– Multi-coreisheretostayand#cores/processorisgrowing– Accelerators(GPUs,gamingsystems)
23Curs 1 - PPD -
![Page 23: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/23.jpg)
exemple…
• ProcesorAMDRyzenThreadripper1950X~1300USD(2017)– MemorieCache40MB– Frecventaprocesor(MHz)3500– TurboBoostpanala 4000MHz– Numarnuclee 16Nuclee=>32Threads
• Intel®Xeon®ProcessorE7v4Family~8000USD(2017)
– #ofCores=24– #ofThreads=48– ProcessorBaseFrequency=2.40GHz– MaxTurboFrequency=3.40GHz– Cache=60MB
Curs 1 - PPD - 24
![Page 24: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/24.jpg)
UBBCLUSTER–IBMIntelligentCluster• Hybridarchitecture
– HPCsystem+– privatecloud
PPD-curs2 25
![Page 25: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/25.jpg)
HPC – IBM NextScale
• Rpeak 62 Tflops, Rmax 40 Tflops • 68 noduri NX360 M5, din care
– 12 nodes with 2 GPU Nvidia K40X, – 6 nodes with Intel Phi
• 2 processors E5-2660 v3 with 10Cores per node • 128 GB RAM per node, 2 HDD SATA de 500 Gb / node • Subscription rate 1:1 between nodes based on Switch: IB Melanox SX6512
with 216 ports • Storage NetApp E5660, 120 HDD SAS cu 600 Gb/Hdd => total 72Tb
– IBM GPFS 4.x -parallel file system
• IBM TS3100 Tape library for data archivation • Operating systems on each node : RedHat Linux 6 with subscription • Management Software: IBM Platform HPC 4.2
PPD-curs2 26
![Page 26: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/26.jpg)
Private Cloud – IBM Flex System
• 10 virtualization servers Flex System x240 – 128 Gb RAM / server – Procesoare 2 x Intel Xeon E5-2640 v2 / server – 2 x SSD SATA 240 Gb / server
• 1 management server • Software for private cloud: IBM cloud manager with OpenStack 4.2 • Software for monitorizing and management: IBM Flex System Manager
software stack • Virtualization software: Vmware vSphere Enterprise 5.1
PPD-curs2 27
![Page 27: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/27.jpg)
Mini cluster – IBM nestscale
~90cores
• 4nodesx2processorsX10cores• 1Managementnode• IP:193.226.40.133
• NOTA– Rmax-MaximalLINPACK(benchmark)performanceachieved– Rpeak-Theoreticalpeakperformance.
• NodeperformanceinGFlops=(CPUspeedinGHz)x(numberofCPUcores)x(CPUinstructionpercycle)x(numberofCPUspernode)
28PPD-curs2
![Page 28: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/28.jpg)
Aplicatiiinformaticeperformante
• Performantaaplicatiilorimpunehardwareperformant(rapid,resursemultiple,etc)
• Hardware-ulavansatgenereazanoiaplicatii• Noileaplicatiiaucererideperformantamaimari
– Crestereexponentialaaperformanteimicroprocesoarelor– Inovatiiinarhitecturileparalelesiinintegrare
• Cerintedeperformanta=>– Performantasistemelortrebuiesaseimbunatateascainansamblu– Schimbari/abordari/reevaluariinSoftwareengineering– Costuri-technologieavansata
applicationsperformance
hardware
29Curs 1 - PPD -
![Page 29: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/29.jpg)
Curs 1 - PPD - 30
Programareparalelavs.Programaredistribuita
INTERCONNECTION NETWORK
P2
P3
P1
P4 P5
Pn . . . .
![Page 30: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/30.jpg)
Curs 1 - PPD - 31
TiPURI DE MULTIPROCESARE PARALLEL DISTRIBUTED
ASPECTE TEHNICE • PARALLEL COMPUTERS (- IN MOD UZUAL ) LUCREAZA BAZAT PE
• CUPLARE STRANSA, • in general bazate pe SINCRONICITATE, • CU UN SISTEM DE COMUNICATIE FOARTE RAPID SI FIABIL • Spatiu unic de adresare (intr-o masura mare)
• DISTRIBUTED COMPUTERS • MAI INDEPENDENTE, • COMUNICATIE MAI PUTIN FRECVENTA SI mai putin RAPIDA (ASINCRONA) • COOPERARE LIMITATA • NU EXISTA CEAS GLOBAL • “Independent failures”
SCOPURI
• PARALLEL COMPUTERS COOPEREAZA PENTRU A REZOLVA MAI EFICIENT PROBLEME DIFICILE
• DISTRIBUTED COMPUTERS AU SCOPURI INDIVIDUALE SI ACTIVITATI PRIVATE. DOAR UNEORI INTERCOMUNICAREA ESTE NECESARA
PARALLEL COMPUTERS: COOPERARE IN SENS “POSITIV”
DISTRIBUTED COMPUTERS: COOPERARE IN SENS “NEGATIV” -- DOAR ATUNCI CAND ESTE NECESARA
![Page 31: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/31.jpg)
Curs 1 - PPD - 32
Aplicatii paralele Suntem interesati sa rezolvam problemele mai rapid in paralel
Aplicatii distribuite
Suntem interesati sa rezolvam anumite probleme specifice :
• COMMUNICATION SERVICES ROUTING BROADCASTING
• MAINTENANCE OF CONTROL STUCTURE TOPOLOGY UPDATE LEADER ELECTION • RESOURCE CONTROL ACTIVITIES LOAD BALANCING MANAGING GLOBAL DIRECTORIES
Ingeneral…
![Page 32: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/32.jpg)
Curs 1 - PPD - 33
Parallelv.s.DistributedSystems(fromM.FUKUDACSS434SystemModels)
Parallel Systems Distributed Systems
Memory Tightly coupled shared memory UMA, NUMA
Distributed memory Message passing, RPC, and/or used of distributed shared memory
Control Global clock control SIMD, MIMD
No global clock control Synchronization algorithms needed
Processor interconnection
Order of Tbps Bus, mesh, tree, mesh of tree, and hypercube (-related) network
Order of Gbps Ethernet(bus), token ring and SCI (ring), myrinet(switching network)
Main focus Performance Ex. - Scientific computing
Performance(cost and scalability) Reliability/availability Information/resource sharing
Unpunctdevedere…
![Page 33: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/33.jpg)
Curs 1 - PPD - 34
SistemeleDistribuite
-potfifolositepentru-
• Aplicatiidistribuiteimplicit– BDDistribuite,rezervaribilieteavion/etc.sistembancar
• Informatiipartajateintreuseri• Partajareresurse• Raportcost/performantamaibunptaplicatiiparalele
– Potfifolositeeficientpt.aplicatiicugranularitatemare(coarse-grained)si/sauptaplicatiiparaleledetipembarrassinglyparallelapplications
• Fiabilitate(Reliability).• Scalabilitate
– Cuplareslaba(Looselycoupledconnection);hotplug-in
• Flexibilitate– Reconfiguraresistemptaintrunicerintele
![Page 34: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/34.jpg)
Curs 1 - PPD - 35
Performanta/Scalabilitate
Spredeosebiredesistemeleparaleleceledistribuiteimplica:- mediumaiputinrapiddetransferaldatelor(reteamaiputinrapida)- HeterogenitateSolutii:-Procesarebatchamesajelor:
• SeevitainterventiaSOptfiecaretransferdemesaj.– Cachedata
• Seevitarepetareatransferuluiaceleiasidate– Evitateaentitatilorsiaalgoritmilorcentralizati
• Evitareasaturariiretelei– Realizareoperatii“post”lanivelulclientului
• Evitareatraficuluiintensintreclientisiservere
– ….
![Page 35: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/35.jpg)
Curs 1 - PPD - 36
Securitate
• Nuexistadoarunsingurpunctdecontrol
• Probleme:– Mesaje,furate,modificate,copiate,…
• Solutie:folosireCriptografie– Failures
• FaultTolerancesolutions
![Page 36: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/36.jpg)
Tipuridesistemedistribuite(Munehiro Fukuda – PDP Fundamentals)
• Modele– Minicomputer– Workstation– Workstation-server– Processor-pool– Cluster– Gridcomputing
Curs 1 - PPD - 37
![Page 37: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/37.jpg)
38
ModelulMinicomputer
• ExtensieasistemuluideTimesharing– Useriitrebuiesaseloghezepepropriulminicomputer.– Apoiselogheazalaaltamasina(remotemachine)prinprogramede
tiptelnet.• Partajareresurse
– DB– High-performancedevices
Mini-computer
Mini-computer
Mini-computer
net
Curs 1 - PPD -
![Page 38: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/38.jpg)
Curs 1 - PPD - 39
ModelulWorkstation
• MigrareProcese– Useriiselogheazamaiintaipestatiadelucrupersonala;– Dacaexistastatii“inasteptare”–unjob“mare”poatemigrala
unadintreele.• Probleme:
– Cumseidentificastatiile“inasteptare”(idle)?– Cummigreazaunjob?– Ceseintampladacaunaltuserselogheazapemasinafolosita?
100MbpsLAN
Workstation
Workstation Workstation
WorkstationWorkstation
![Page 39: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/39.jpg)
Curs 1 - PPD - 40
ModelulWorkstation-Server
• StatiiClient– AplicatiileGrafice/interactiveseproceseazalocal– Ptaltecereridecalculsetrimitcererilaservere.
• Servere(minicomputers)– Fiecareserverestededicatunuiasaumaimultor
tipurideservicii.• Modeldecomunicare:Client-Servermodel
– RPC(RemoteProcedureCall)– RMI(RemoteMethodInvocation)
• Unprocesclientcheamaofunctieaprocesuluiserver.
• Nusefacemigraredeprocese• Examplu:NFS
100GbpsLAN
Workstation
Workstation Workstation
Mini-Computerfileserver
Mini-Computerhttpserver
Mini-Computercycleserver
![Page 40: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/40.jpg)
Curs 1 - PPD - 41
ModelulProcessor-Pool
• Clientii:– Selogheazalaunterminal– Toateserviciilesuntgestionatede
catreservere.• Servere:
– Ptfiecareusersealocanrnecesardeprocesoaredinpool.
• Utilizarebunadarinteractivitateslaba.
Server1
100MbpsLAN
ServerN
![Page 41: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/41.jpg)
Curs 1 - PPD - 42
ClusterModel
• ConstainmaimultePC/workstationsconectatelaoreteadetiphigh-speed.
• Focuspeperformanta.100Mbps
LAN
Workstation
Workstation Workstation
Masternode
Slave1
SlaveN
Slave2
1GbpsSAN
httpserver1
httpserver2
httpserverN
![Page 42: Curs 1vniculescu/didactic/PPD/suport_curs/C... · 2020. 10. 22. · Curs 1 - PPD - 5 Evaluare • Laborator (30%) >=5 – Exercitii laborator – Programe folosind Java/C++ threads,](https://reader036.vdocuments.site/reader036/viewer/2022071607/6144287caa0cd638b460acc1/html5/thumbnails/42.jpg)
Curs 1 - PPD - 43
High-speedInformationhighway
GridComputing
• Scop– Colectareaputeridecalculamaimultor
supercomputeresiclusteredispersategeografic
• DistributedSupercomputing– Ptproblemwfoartemari.Dificile.
(CPUintensive,memoryintensive).• High-ThroughputComputing
– Folosireamultorresursecarenusuntfolosite
• On-DemandComputing– Resurseladistantaintegrateincalculullocal
• Data-intensiveComputing– distributeddata
• CollaborativeComputing– Suportptcomunicareintremaimulteparti
Super-computer
Cluster
Super-computer Cluster
Mini-computer
Workstation
Workstation Workstation