in situ flow analysis for mpas-ocean simulations
TRANSCRIPT
OverviewNext generation earth system models that scale to emergent HPChardware systems will need in situ analysis coupled to climatesimulation components. In situ analysis can enable quicker and higherfidelity results than postprocessing.
In this work, we enable in situ analysis for MPAS-Ocean simulationusing Decaf, a data flow system for managing dataflow among tasks ina workflow. We also develop methods to improve load balancing indecoupled analysis tasks.
Coupling MPAS-O with stand alone particle tracer
We move the inbuildMPAS-Ocean particletracer LIGHT to a standalone tracer usingDecaf. This allows forbetter control oversharing of resourcesbetween simulationand analysis (tracer).
Decaf: Decoupled dataflow for in-situ workflows
Dynamic load balancing for unstructured data using constrained graph partitioningState-of-the art methods to loadbalance based on constrained k-dtree do not directly extend tounstructured data. We introduce aconstrained graph partitioning basedload balancing method that naturallyhandles data on both structured andunstructured grids.
Dynamic load balancing for unstructured data using graph distance based embedding The use of graph partitioningto balance workload ischallenged by computationalcost of obtaining balancedpartitions. We propose amethod to address this issueby adapting the constrained k-d tree method for graphdistance based embeddings.
In Situ Flow Analysis for MPAS-Ocean SimulationsMukund Raj (ANL), Hanqi Guo (ANL), Orcun Yildiz(ANL), Phillip Wolfram (LANL), Tom Peterka (ANL)
Dynamic load balancing for parallel particle tracing using workload prediction
Existing methods to dynamically loadbalance parallel particle tracing assumesworkload distribution at upcoming epoch tobe same as the current workload whenpartitioning data for upcoming epoch. Wepropose a method to achieve better loadbalance by partitioning the data based onpredicted workload for upcoming epoch,rather than current workload distribution.
Decaf Dataflow Definition
Transport Layer
Workflow Definition
Swift
Flowcontrol
Nessie Mercury MPI
Python
Datamodel
Datadistribution
Resilience
Decaf Runtime
XML Pegasus
Decaf software organization
MPAS-O
MPAS-O
MPAS-O
MPAS-O
Particle Tracer
Particle Tracer
Particle Tracer
Particle Tracer
Decaf
Coupling MPAS-O with particle tracer using Decaf (left) andpreliminary timing results for MPAS-Ocean built-in particletracer (LIGHT), and data movement using Decaf
Imbalance in particle distribution (left) and advectionworkload (right) for synthetic data in prediction andbaseline cases
Constrained graph partitioning for dynamic load balancingparticle tracing with unstructured data (left) and preliminarytiming results for constrained graph partitioning using BFSexpansion (right)
A workflow is a directed graph of tasks and communicationbetween them. Graph nodes are tasks and graph links arecommunication; both of which can be parallel processes.A dataflow is the communication over links in a workflow.While workflows consists of high level tasks, dataflowsconsists of communication between processes.The Decaf software stack consists of a workflow definitionlayer, a dataflow definition layer, and a transport layer.
Vertex layout in the MPAS-Ocean grid usingintrinsic spatial coordinates (left) and graphdistance based embedding via Pivot MDS (right)