hpc challenges for aerothermal predicons in aeronaucal …cscads.rice.edu/franzelli-engines.pdfhpc...
TRANSCRIPT
HPCchallengesforaerothermalpredic5onsin
aeronau5calengines
4thSciDACCScADSSummerWorkshop
B.Franzelli,PhDstudent
CERFACS,France
CERFACS• EuropeanCenterforResearchandAdvancedTraininginScien5ficCompu5ng
• Researchcivilsociety• 7shareholders:CNES,EDF,Météo‐France,ONERA,EADS,SAFRAN,TOTAL
• 110people,90researchersandengineerscomingfrommorethan10countries
• Mainresearchfields:– Aerodynamics– Climateandenvironment– Codecoupling– Combus5on
− Dataassimila5on− Electromagne5sm− Parallelalgorithms
B.Franzelli,CERFACS,France2
Combus5oninaeronau5calengines
3
Compressor
Air‐intake
Turbine
MAINPHYSICALPHENOMENA:1. Fuelinjec5on(two‐phaseflow)2. ChemicalKine5cs3. Thermaltransfers:
‐Wallheattransfer‐Radia5veheattransfer
Burntgases
Combus:onchamber
NoiseSpecificconsump5on Noise
SpecificPowerLongevity
GOALS:• Predic5onofpollutants(CO,Nox,Soot)• Temperatureprofileatcombustorexit(anincreaseof20Kattheexitdividestheshelflifeoftheengineby2)
B.Franzelli,CERFACS,France
MULTIPH
YSICS
Thefullthermalproblem
Theobjec5veistodescribeatwo‐phasereac5veflowinacomplexgeometrytakingintoaccount:• Combus5on&sootproduc5on• Conduc5on• Radia5on• Convec5on
B.Franzelli,CERFACS,France
Thefullthermalproblemisresolvedcouplingthreecodesdedicatedtoeachseparatephenomenonviaaparalleldynamiccoupler.
STRATEGY=MULTIPHYSICSCOUPLING
4
Outline
• Descrip5onofthecodesdevelopped• Examplesofpar55oningproblems
– Efficientpar55oningalgorithmsfortwo‐phaseflowsimula5onsusingaLagrangianapproach
– Storageofhugechemicaldatabasesfortabula5onmethods
– Paralleliza5onoftheradia5vecode
• Tomorrowchallenge:thefullthermalproblem
B.Franzelli,CERFACS,France5
Outline
• Descrip5onofthecodesdevelopped• Examplesofpar55oningproblems
– Efficientpar55oningalgorithmsfortwo‐phaseflowsimula5onsusingaLagrangianapproach
– Storageofhugechemicaldatabasesfortabula5onmethods
– Paralleliza5onoftheradia5vecode
• Tomorrowchallenge:thefullthermalproblem
B.Franzelli,CERFACS,France
AVBPcodemaincharacteris5cs
6
• Startedin1993
• Compressiblereac5veNSequa5ons
• Explicitin5me
• Unstructured/hybridmeshes
• Movingmeshes
• Centredschemes
– FiniteVolume/FiniteElement(2nd/3rdorder)
+controlledlocalar5ficialviscosity
• MassivelyParallel
• MachineIndependent M.Boileau(2007)
‐CrayXT3(Rochester,US)‐ AMDbi‐coreOpteron2.4Ghz‐ 700processors
‐Consump5on:112000hCPU‐Execu5on5me:160h‐Physical5me:50ms
AVBPcodemaincharacteris5cs
6
• Startedin1993
• Compressiblereac5veNSequa5ons
• Explicitin5me
• Unstructured/hybridmeshes
• Movingmeshes
• Centredschemes
– FiniteVolume/FiniteElement(2nd/3rdorder)
+controlledlocalar5ficialviscosity
• MassivelyParallel
• MachineIndependent
AVTPcodemaincharacteris5cs
• HeatEqua5onsolverforanon‐homogenousnon‐isotropicsolidmedium
• DatastructureandnumericalmethodsinheritedbyAVBP
• Unstructured/hybridmeshes
J.Amaya(2010)
B.Franzelli,CERFACS,France7
PRISSMAcodemaincharacteris5cs
• Radia5veTransfertEqua5on(RTE)solver
• BasedontheDiscreteOrdinatesMethod(DOM)
• Unstructured/hybridmeshes
• Differentglobalandnarrow‐bandmodelsforthespectralintegra5on
• Radia5onproper5esofcombus5ongases(CO,CO2,H2O)andsoot J.Amaya(2010)
B.Franzelli,CERFACS,France8
Outline
• Descrip5onofthecodesdevelopped• Examplesofpar55oningproblems
1.Efficientpar55oningalgorithmsfortwo‐phaseflowsimula5onsusingaLagrangianapproach
2.Storageofhugechemicaldatabasesfortabula5onmethods
3.Paralleliza5onoftheradia5vecode
• Tomorrowchallenge:thefullthermalproblem
B.Franzelli,CERFACS,France
1.Two‐phaseflowandHPC
Developmentof2solversinAVBP:
• Gasphase:Eulersolver• Liquidphase:Lagrangesolver
✗ Therearemillionsoffueldropletstobetrackeddownstreamfromtheinjector
✗ Needforefficienttwo‐constraintpar55oningalgorithms
9M.Garcia(2009)B.Franzelli,CERFACS,France
Imbalancedsimula:on
Balancedsimula:on
0
5000
10000
15000
20000
25000
Nprocs
1.Two‐phaseflowandHPC
No particles
0
100000
200000
300000
Nprocs
N° particles
RIB(one‐constraint)
METIS(two‐constraint)
Fortwo‐phaseflowswherepar5clesarenon‐uniformlylocated,two‐constraintpar55oningalgorithmsarerequiredto
avoidloadimbalancing.
B.Franzelli,CERFACS,France
M.Garcia(2009)
1.Two‐phaseflowandHPC
B.Franzelli,CERFACS,France
Duringthecalcula5onofanunsteadycase,par5clesdonotremainatthesameplace:• Howtomodifythepar55oningwith5meinordertopreserveacorrectpar5clesbalance??• Whichcriteriashouldbeusedtodeterminewhenandhowtomodifythepar55oning???
OPENQUESTIONS:
Imbalancedsimula:on
Balancedsimula:on
Fortwo‐phaseflowswherepar5clesarenon‐uniformlylocated,two‐constraintpar55oningalgorithmsarerequiredto
avoidloadimbalancing.
M.Garcia(2009)
2.Chemicalkine5csandHPC• Inordertocorrectlydescribeintermediatespecies,pollutantsandsootinaturbulent
reac5veflow,anaccuratedescrip5onofthechemicalkine5csisrequired.
• Tabula:onmethodsarewidelyworkedon:adatabaseofrelevantchemicalterms(massfrac5onsandtemperature)basedonsimula5onsofsimplecombus5onproblemsisusedtorebuildthechemicalsourcetermsinacomplex3Dcalcula5on.
B.Franzelli,CERFACS,France
Anaprioripar55oningofthedatabaseisimpossible:inanunsteadycomputa5on,eachgridnodemustbeabletoaccessanypartofthedatabaseateach5mestep.
10
• Readthewholedatabaseoneachprocessor:alargeamountofmemoryperprocessoristhenrequired,whichiscontrarytothetendencyofthenewgenera5onmachines.• Reducethedatabasesize:someinforma5oncouldbeneglectedorsingleprecisionfloa5ng‐pointformatcouldbeused,whichwoulddecreasetheaccuracyofthechemicaldescrip5on.• Readdynamicallyandpar:allythedatabaseoneachprocessor:averyquickI/Ocomunica5onisthenrequired.
POTENTIALSOLUTIONS:
3.Radia5onandHPCRadia:vesourcetermdependsonthephoton
frequencyandtheangulardirec:on.
FREQUENCYINTEGRATION:quan55esdependonlocalproper5es=>goodsubdomainparallelism.
ANGULARDIRECTION:includesdatafromALLthepointsofthedomain(longdistanceitera5ons).Sequen5alalgorithm=>problemforsubdomainparallelism.
J.Amaya(2010)
B.Franzelli,CERFACS,France11
Outline
• Descrip5onofthecodesdeveloped• Examplesofpar55oningproblems
1.Efficientpar55oningalgorithmsfortwo‐phaseflowsimula5onsusingaLagrangianapproach
2.Storageofhugechemicaldatabasesfortabula5onmethods
3.Paralleliza5onoftheradia5vecode
• Tomorrowchallenge:thefullthermalproblem
B.Franzelli,CERFACS,France
Thefullthermalproblem
12
Thefullthermalproblemisresolvedusingspecializedcodesforeachtransfermode.
Acoupler(PALM)isusedtoexchangedatabetweenthecodes.
Thefullthermalproblem
B.Franzelli,CERFACS,France
CONDUCTIONAVTP
COMBUSTIONAVBP
RADIATIONPRISSMA
CHARACTERISTICTIME[s]
102
10‐3
10‐9
10‐11
CPUTIME1x
10x
100000x
Differentcomputa5onalresourcesandres5tu5on5mesofcalcula5onsmustbemanagedbythecoupler:difficulttoachievetheop5mumonamassivelyparallelmachine.
Thefullthermalproblemischaracterizedby:• differentphysics(characteris5c5me)• differentmeshes(CPU5me)
13
Thefullthermaldescrip5onandHPC
• Mul:codeproblem:howcouldthecouplermanagethehugeamountofinforma5onrequiredandcalculatedbythethreecodesinamassivelyparallelway(memoryandsynchronisa5onproblems)?
• Mul:machineproblem:howtomanagethecomunica5onof3codesrunningondifferentmachines?
• Dynamicaldistribu:onofprocessors:howcouldprocessorsbedynamicallydistributedbetweenthedifferentcodes(i.e.unsteadycasesormovingmeshes).
B.Franzelli,CERFACS,France14
SpecialThanksto:
J.Amaya,M.Garcia,D.Poitou,E.Riber,O.Vermorel,thePALMTeam
ThisresearchprojecthasbeensupportedbyaMarieCurieEarlyStageResearchTrainingFellowshipoftheEuropeanCommunity’sSixthFrameworkProgrammeundercontractnumberMEST‐CT‐2005‐020426.