-
Development of Development of CCoastal oastal OOcean cean MModeling odeling IInfrastructure nfrastructure
(COMI) at LSU(COMI) at LSU
Q. Jim Chen
Department of Civil and Environmental Engineering& Center for Computation and Technology
Louisiana State University
-
Acknowledgements
COMI Team:Drs. Gabrielle Allen, Mayank Tyagi (CCT)Drs. Claes Eskilsson, Kelin Hu (CEE/CCT)Yaakoub El-Khamra, Lei Jiang (CS)Qi Fan, Ranjit Jadhav, Qian Zhang, Haihong Zhao (CEE)
Funding:NSF, ONR, NOAA
Gulf Shores, Alabama
-
Motivation
Combined waves and surge
Depth-integrated coastal models
Higher-order finite element methods
Modeling framework and results
Conclusion and future work
Outline
-
Courtesy of Lake Pontchartrain Basin Foundation
Animation of Wave-Structure-Seabed Interaction
Louisiana Coastal Protection Assessment of Multiple-lines of Defense
Chen et al. (2006)
-
Wave overtopping
Bridge damaged by storm waves Combined
Waves & Surge
news.bbc.co.uk/.../uk_enl_1194609737/img/1.jpg
Kevork Djansezian/ AP
Gulfport, Mississippi during Gustav
US -90 bridge over Biloxi Bay, MS
Impact of Katrina on WetlandsUK
-
Challenges
Hurricane protection and coastal restoration require a suite of numerical models for different physical processes.
Different models use different numerical methods to solve different governing equations, e.g. FDM, FEM, FVM, SEM
There is a need for developing a common modeling framework to enable the modeling of multi-physics, multi-scale coastal processes using HPC.
-
CFD Toolkit
ComposingFluid-flow solver
Solvers
SuperLU
Trilinos
PETSc
Numerics
Finite difference
Finite Volume
Spectral Element
Infrastructure
Cartesian
M-B curvilinear
Unstructured
Physics
Navier-Stokes
RANS
Boussinesq
Other coastal models
To develop the capability of modeling coastal circulation and nearshoresurface waves in deltaic environments using high-order numerical methods
To integrate application-oriented coastal modeling systems with massive-processor, high-performance computing facilities and technologies available at LSU
Objectives
-
Depth-Integrated Coastal Models
3-DimensionalEuler Equations
Boussinesq-typeEquations
Nonlinear ShallowWater Equations
Linear Long Wave Equations
Mild SlopeEquations
Wind Stress
Storm Surge
Wave Growth Storm Surge
-
Modeling Surge/Wave Attenuation by Porous Media or Vegetation
Porous Media
Cruz and Chen (2007)
-
Why higher-ordernumerical methods?
-
Unstructured Spectral/hpDG Methods
Solution approximated inside an elementby a pth order polynomial expansion
(note: can be non-uniform).
Elements coupled throughso-called numerical fluxes
(computed by approximate Riemann solvers)
Supports unstructured meshes (conforming or non-conforming) consistingof elements of size h. Note that the solution
is allowed to be discontinuous over theelement boundaries.
-
CACTUS Framework
CACTUS is a freely available, modular, portable and manageable environment
for collaboratively developing parallel, high-performance multi-dimensional simulations
http://www.cactuscode.org/Successful Applications: Astrophysics,Petroleum Engineering,
http://www.cactuscode.org/
-
COMI Architecture
UMDriver provides the underlying parallel layer. At present it uses the Zoltan library to provide mesh partitioning, load balancing and mesh migration
LocalToGlobal provides local re-indexing of elements, edges and vertices The Nektar++ thorn initializes and populates the data structures of the
Nektar++ library MeshReader provides a simple ASCII mesh file reader, and also allows users to
register their own mesh readers (e.g. the mesh reader from Nektar++) The core thorn for the coastal modeling toolkit in Cactus is CoastalWave. This thorn
defines the generic variables, parameters, and methods for coastal models Thorn SWE contains the actual SWE solver
-
Nektar++ library
Nektar++ is an open-source spectral/hp element library presently in the last stages of development (www.nektar.info)
As the name suggests Nektar++ is written in C++
Nektar++ is developed and maintained by Prof. Sherwin (Imperial College London) and Prof. Kirby (University of Utah)
COMI's shallow water codes are based upon Nektar++ and will be distributed as part of the Nektar++ solver library
http://www.nektar.info/
-
Discontinuous GalerkinSpectral Element Model
Flood Wall
Flood wal
Breach
-
Domain distributed over 8 cores
Sixth order triangular elements
COMI Result Eskilsson et al. (ICCS 2009)
-
Parallel DG Model
-
Weak Scaling 900 quadrilateral elements per core 100 time steps Times without I/O, initializing and partitioning of the mesh 128 cores, p = 8 roughly 28 million DoF
-
Hurricane Gustav Animation of the combined H*wind and background winds (NCEP/NCAR Reanalysis)
-
Wind Comparison
0
10
20
30
40Buoy 42040
Win
d sp
eed
(m/s
)
0
10
20
30
40Buoy 42007
Win
d sp
eed
(m/s
)
0
10
20
30
40Dauphin Island
Win
d sp
eed
(m/s
)
Aug.31 00 12 Sep.1 00 12 Sep.2 000
10
20
30
40PSTL1 SW Pass
Win
d sp
eed
(m/s
)
Time (days, Aug.30 ~ Sep.2, 2008)
0
90
180
270
360Buoy 42040
Win
d di
rect
ion
(deg
rees
)
0
90
180
270
360Buoy 42007
Win
d di
rect
ion
(deg
rees
)0
90
180
270
360Dauphin Island
Win
d di
rect
ion
(deg
rees
)
Aug.31 00 12 Sep.1 00 12 Sep.2 000
90
180
270
360PSTL1 SW Pass
Win
d di
rect
ion
(deg
rees
)
Time (days, Aug.30 ~ Sep.2, 2008)
90.5 90.0 89.5 89.0 88.5 88.0 87.5 87.0 86.5 86.0 85.5 85.028.5
29.0
29.5
30.0
30.5
31.0
Longitude (degrees)
Latit
ude
(deg
rees
)
Buoy 42040
Buoy 42039
Buoy 42007
SWP BURL1
CSBF1
Grand Isle
Dauphin Island Perdido PassFort Walton
Panama Beach
Apalachicola
PSTL1 SW Pass
State Docks
Mobile Airport Pensacola AirportDauphin Island Sea Lab StationMeteorology Stations
Tide Gages
C-MAN StationsBuoy Stations
42007
42040
-
Total triangle elements: 63,757 Total nodes: 32,544. Time step: 4 s
The basinThe basin--scale mesh provides the scale mesh provides the open boundary conditions.open boundary conditions.
Both ADCIRC and SWAN are coupled Both ADCIRC and SWAN are coupled using the same regional mesh.using the same regional mesh.
Fine spatial resolution is needed to Fine spatial resolution is needed to resolve topographic and hydrodynamic resolve topographic and hydrodynamic features of flooding and coastal waves.features of flooding and coastal waves.
Nested Domain for Wave-Surge Coupling
Lake Pontchartrain
New Orleans
ADCIRC Mesh
-
Animation of Modeled Significant Wave Heights (m) during Gustav
-
Modeled Maximum Significant Wave Heights (m) during Gustav
-
Wave Comparison
90.5 90.0 89.5 89.0 88.5 88.0 87.5 87.0 86.5 86.0 85.5 85.028.5
29.0
29.5
30.0
30.5
31.0
Longitude (degrees)
Latit
ude
(deg
rees
)
Buoy 42040
Buoy 42039
Buoy 42007
SWP BURL1
CSBF1
Grand Isle
Dauphin Island Perdido PassFort Walton
Panama Beach
Apalachicola
PSTL1 SW Pass
State Docks
Mobile Airport Pensacola AirportDauphin Island Sea Lab StationMeteorology Stations
Tide Gages
C-MAN StationsBuoy Stations
25 30 35 40 45 50 55 60 65 700
2
4
6
8
10
12Buoy 42040 (Red for the observed;Blue for the unstructured modeled;
Hs
(m)
25 30 35 40 45 50 55 60 65 700
2
4
6
8
10
12Black for the rectangular modeled; Green for the unstructured modeled with no elevation change)
Wav
e pe
riod
(s)
25 30 35 40 45 50 55 60 65 700
2
4
6
8
10
12Buoy 42007
Hs
(m)
Hours from 2008-08-30 00h25 30 35 40 45 50 55 60 65 70
0
2
4
6
8
10
12
Wav
e pe
riod
(s)
Hours from 2008-08-30 00h
42007
42040
-
Contribution of Storm Surge to Maximum Significant Wave Heights during Gustav
-
Conclusion and Future Work
A coastal modeling framework has been developed at LSU using spectral/hp DG methods and HPC technologies.
Effects of vegetation and fluid mud are being incorporated into the solvers.
Recent hurricanes provide an excellent testbed for skill assessment of coupled, unstructured surge, wave, salinity and sediment transport models
-
Surge Comparison
20 30 40 50 60 70 800
0.5
1
1.5
2PSTL1 (red for the observed; blue for the calculated)
Sur
ge (m
)
20 30 40 50 60 70 800
0.5
1
1.5
2
Sur
ge (m
)
Dauphin Island (red for the observed; blue for the calculated)
20 30 40 50 60 70 800
0.5
1
1.5
2
Sur
ge (m
)
Panama City
20 30 40 50 60 70 800
0.5
1
1.5
2
Sur
ge (m
)
Pensacola
Hours from 08-30-2008 00:00
20 30 40 50 60 70 800
0.5
1
1.5
2
Sur
ge (m
)
Mobile State Docks
Hours from 08-30-2008 00:0020 30 40 50 60 70 80
0
0.5
1
1.5
2S
urge
(m)
GDIL1
Hours from 08-30-2008 00:00
90.5 90.0 89.5 89.0 88.5 88.0 87.5 87.0 86.5 86.0 85.5 85.028.5
29.0
29.5
30.0
30.5
31.0
Longitude (degrees)
Latit
ude
(deg
rees
)
Buoy 42040
Buoy 42039
Buoy 42007
SWP BURL1
CSBF1
Grand Isle
Dauphin Island Perdido PassFort Walton
Panama Beach
Apalachicola
PSTL1 SW Pass
State Docks
Mobile Airport Pensacola AirportDauphin Island Sea Lab StationMeteorology Stations
Tide Gages
C-MAN StationsBuoy Stations
-
Modeled Maximum SurgeHeights (m, MSL) during Gustav
-
What is the cause of Katrinas record high surge?
250 200 150 100 50-500
-400
-300
-200
-100
0
Ave
rage
dep
th b
elow
MS
L (m
)
Distance from shoreline along the track of hurricane (km)
KatrinaIvanFrederic
50 40 30 20 10 00
2
4
6
8
10
Sur
ge H
eigh
t (m
,MS
L)
Distance from shoreline (km)
Sloping BottomFlat Bottom
Chen et al. (2008)
-
Structure of Cactus
Core Flesh
Plug-In Thorns(modules)
driverdriver
input/outputinput/output
interpolationinterpolation
SOR solverSOR solver
coordinatescoordinates
boundaryboundary conditionsconditions
black holesblack holes
equations of stateequations of state
remote steeringremote steering
wave evolverswave evolvers multigridmultigrid
parametersparameters
gridgrid variablesvariables
errorerror handlinghandling
schedulingscheduling
extensibleextensible APIsAPIs
makemake systemsystem
ANSI CANSI C
Fortran/C/C++Fortran/C/C++
Reservoir simulatorsReservoir simulators
Coastal modellingCoastal modelling
Molecular dynamicsMolecular dynamics
-
A Fully Nonlinear Boussinesq Model for Waves and Currents
Continuity Equation
Momentum Equation
)( 04 ghOM
t =+
)(
)()(
0
04
3212
lghORRR
VVVguut
u
wfi
=+
+++++
Wei et al. (1995) and Chen et al. (2003, 2004, 2006)
= free surface elevationM = volume rate of flow
Boussinesq dispersive terms
Momentummixing term
Wind forcing
Conservation ofpotential vorticity
-
Sea-Dependent Wind Drag Coefficients
A new formulation
Wu (1980)
10213 |)|(10 bUaaC xd ++=
103 065.08.010 UCd +=
a1= 0.2a2 = 18b = 0.065
Surface slope
(Sea-independent)
0 5 10 15 20 25 300
0.5
1
1.5
2
2.5
3
U10(m/s)
Cd
x 10
3
ka0.2
0.15
0.1
0.05
Large and Pond (81)Smith (80)Smith and Banke (75)Geernaert et al.(87)Geernaert et al.(86)Sheppard et al. (72)Donelan (82)Denman and Miyake (73)Pond et al. (71)Graf et al. (84)
Ka=2
(Chen et al, 2004)
Wind Stress = Skin Friction + Form Drag
Challenges Depth-Integrated Coastal Models Modeling Surge/Wave Attenuation by Porous Media or VegetationWhy higher-ordernumerical methods? Unstructured Spectral/hp DG MethodsCOMI ArchitectureNektar++ library Discontinuous Galerkin Spectral Element Model Parallel DG Model Weak ScalingHurricane GustavWind ComparisonNested Domain for Wave-Surge CouplingAnimation of Modeled Significant Wave Heights (m) during GustavModeled Maximum Significant Wave Heights (m) during GustavWave ComparisonContribution of Storm Surge to Maximum Significant Wave Heights during GustavConclusion and Future WorkSurge ComparisonModeled Maximum SurgeHeights (m, MSL) during Gustav What is the cause of Katrinas record high surge?Structure of CactusA Fully Nonlinear Boussinesq Model for Waves and CurrentsSea-Dependent Wind Drag Coefficients