self-attraction and loading in mpas-ocean
TRANSCRIPT
Los Alamos National Laboratory
Self-Attraction and Loading in MPAS-Ocean
Kristin Barton
LA-UR-20-25989
Managed by Triad National Security, LLC for the U.S. Department of Energy's NNSA
6 Aug 2020
Mentors: Mark Petersen, Steven Brus, Brian Arbic
Los Alamos National Laboratory
2
MPAS-Ocean (Model for Prediction Across Scales)
• Ocean simulation model • Unstructured mesh • Allows for variable resolution • Time scales from months to millennia • Spatial scales from < 1 km to global • Functions as ocean component in
climate system models
“MPAS Overview” https://mpas-dev.github.io, Accessed: 26 Jun 2020Petersen, Mark Roger Asay-Davis, Xylar Storm Jacobsen, Douglas William Maltrud, Mathew Einar Ringler, Todd Darwin Van Roekel, Luke Veneziani,
Carmela Wolfram, Phillip Justin Jr. MPAS-Ocean Model User's Guide Version 6.0. Report (2018)Ringler, T., Petersen, M., Higdon, R. L., Jacobsen, D., Jones, P. W., & Maltrud, M. (2013). Ocean Modelling. Ocean Modelling, 69(C), 211–232. doi:10.1016/
j.ocemod.2013.04.010
Los Alamos National Laboratory
3
• Tectonics • Water depth • Shoreline position • Seabed roughness • Extent of sea ice coverage • More!
Haigh, I. D., Pickering, M. D., Green, J. A. M., Arbic, B. K., Arns, A., Dangendorf, S., et al. (2020). The tides they are a-changin': A comprehensive review of past and future nonastronomical changes in tides, their driving mechanisms and future implications. Reviews of Geophysics, 57, e2018RG000636. https://doi.org/10.1029/2018RG000636
Müller, M., Arbic, B. K., and Mitrovica, J. X. (2011), Secular trends in ocean tides: Observations and model results, J. Geophys. Res., 116, C05013, doi:10.1029/2010JC006387.
Tides themselves are not constant, they can be influenced by things like:
Importance of Tides in a global ocean model
Los Alamos National Laboratory
4
Tides account for…
Arbic, Brian & Alford, Matthew & Ansong, Joseph & Buijsman, Maarten & Ciotti, Robert & Farrar, J. & Hallberg, Robert & Henze, Christopher & Hill, Christopher & Luecke, Conrad & Menemenlis, Dimitris & Metzger, E. & Müeller, Malte & Nelson, Arin & Nelson, Bron & Ngodock, Hans & Ponte, Rui & Richman, James & Savage, Anna & Zhao, Zhongxiang. (2018). A Primer on Global Internal Tide and Internal Gravity Wave Continuum Modeling in HYCOM and MITgcm. 10.17125/gov2018.ch13
Astronomical forcing (sun and moon) Self-Attraction and Loading
Damping• SAL includes effects from:
• Tides deforming the earth • Self-gravitation of earth • Self-gravitation of water
Moon
Earth
Los Alamos National Laboratory
5
Importance of SAL• Can be approximated by a
constant or iterative methods
• Not including SAL can affect:
• tidal amplitudes by 10%
• tidal phases by 30deg • Better to calculate using
full spherical harmonic decomposition
Schindelegger, M., Green, J.A.M., Wilmes, S.-B., Haigh, I.D. (2018), Can we model the effect of observed sea level rise on tides? Journal of Geophysical Research: Oceans, 123, 4593–4609. doi: https://doi.org/10.1029/2018JC013959.
Einšpigel, D., Martinec, Z. (2017), Time-domain modeling of global ocean tides generated by the full lunisolar potential. Ocean Dynamics, 67, 165–189. doi: 10.1007/s10236-016-1016-1.
Los Alamos National Laboratory
6
How self-attraction and loading calculation works with spherical harmonic transform
• SHTns = fast spherical harmonic transform package
• Uses OpenMP to perform transform on a single node
• Each coefficient is multiplied by a load love number
Schaeffer, N. (2013), Efficient spherical harmonic transforms aimed at pseudospectral numerical simulations, Geochem. Geophys. Geosyst., 14, 751– 758, doi:10.1002/ggge.20071.
Schindelegger, M., Green, J.A.M., Wilmes, S.-B., Haigh, I.D. (2018), Can we model the effect of observed sea level rise on tides? Journal of Geophysical Research: Oceans, 123, 4593–4609. doi: https://doi.org/10.1029/2018JC013959.
spatial data
spatial data
spherical harmonic transform
coeff(i) = coeff(i) * scale(i)
Los Alamos National Laboratory
7
Simple model: Debot
• Twice-daily lunar tidal amplitudes in meters
• Comparison with a benchmark tidal model TPXO
• Root mean square error: • 3cm for >1000m depths • 14cm for <1000m depths
• This model does not require interpolation
• Provides basis for adding routine to MPAS
Einšpigel, D., Martinec, Z. (2017), Time-domain modeling of global ocean tides generated by the full lunisolar potential. Ocean Dynamics, 67, 165–189. doi: 10.1007/s10236-016-1016-1.
Schindelegger, M., Green, J.A.M., Wilmes, S.-B., Haigh, I.D. (2018), Can we model the effect of observed sea level rise on tides? Journal of Geophysical Research: Oceans, 123, 4593–4609. doi: https://doi.org/10.1029/2018JC013959.
Los Alamos National Laboratory
8
Interpolation
“Gaussian Grids” European Centre for Medium-Range Weather Forecasts. https://confluence.ecmwf.int/display/FCST/Gaussian+grids. Accessed 15 July 2020.
MPAS unstructured mesh Gaussian grid
• Gaussian grid: position of latitudes based on zeros of Legendre polynomials • Useful for doing fast spherical harmonic transforms
Los Alamos National Laboratory
9
Interpolation Process
MPAS mesh
Gaussian grid
Interpolation weight matrix
Los Alamos National Laboratory
10
ESMF (Earth System Modeling Framework)
• Used for coupling models together • Has functions for interpolation • Massive library, so we don’t want to use the whole thing
V. Balaji, Byron Boville, Samson Cheung, Tom Clune, Nancy Collins, Tony Craig, Carlos Cruz, Arlindo da Silva, Cecelia DeLuca, Rosalinda de Fainchtein, Rocky Dunlap, Brian Eaton, Steve Goldhaber, Bob Hallberg, Tom Henderson, Chris Hill, Mark Iredell, Joseph Jacob, Rob Jacob, Phil Jones, Brian Kauffman, Erik Kluzek, Ben Koziol, Jay Larson, Peggy Li, Fei Liu, John Michalakes, Raffaele Montuoro, Sylvia Murphy, David Neckels, Ryan O Kuinghttons, Bob Oehmke, Chuck Panaccione, Daniel Rosen, Jim Rosinski, Mathew Rothstein, Kathy Saint, Will Sawyer, Earl Schwab, Shepard Smithline, Walter Spector, Don Stark, Max Suarez, Spencer Swift, Gerhard Theurich, Atanas Trayanov, Silverio Vasquez, Jon Wolfe, Weiyu Yang, Mike Young, Leonid Zaslavsky. Earth System Modeling Framework: ESMF User Guide Version 8.0.0. 13 Oct. 2019.
ESMF superstructure
ESMF infrastructure
User code
ocean
atmosphere land
ESMF coupler
Los Alamos National Laboratory
11
ESMF (Earth System Modeling Framework) — Options
V. Balaji, Byron Boville, Samson Cheung, Tom Clune, Nancy Collins, Tony Craig, Carlos Cruz, Arlindo da Silva, Cecelia DeLuca, Rosalinda de Fainchtein, Rocky Dunlap, Brian Eaton, Steve Goldhaber, Bob Hallberg, Tom Henderson, Chris Hill, Mark Iredell, Joseph Jacob, Rob Jacob, Phil Jones, Brian Kauffman, Erik Kluzek, Ben Koziol, Jay Larson, Peggy Li, Fei Liu, John Michalakes, Raffaele Montuoro, Sylvia Murphy, David Neckels, Ryan O Kuinghttons, Bob Oehmke, Chuck Panaccione, Daniel Rosen, Jim Rosinski, Mathew Rothstein, Kathy Saint, Will Sawyer, Earl Schwab, Shepard Smithline, Walter Spector, Don Stark, Max Suarez, Spencer Swift, Gerhard Theurich, Atanas Trayanov, Silverio Vasquez, Jon Wolfe, Weiyu Yang, Mike Young, Leonid Zaslavsky. Earth System Modeling Framework: ESMF User Guide Version 8.0.0. 13 Oct. 2019.
Regrid Options
ESMF_RegridWeightGen ESMF_FieldRegrid
• Command line tool • Only outputs weight file • Easy to use
• Can perform entire interpolation
• Requires more restructuring of code
Los Alamos National Laboratory
Pre-generate weight matrices
Interpolate with matrix
multiplication
Do spherical harmonic transform
Perform SAL calculation
Interpolate back to original mesh
Current Progress — Individual pieces are all working!
Interpolate back to original mesh
Los Alamos National Laboratory
Current Challenge — Getting all the pieces working together inside MPAS-O
Master node
local cell ID
local cell ID
local cell ID
local cell ID
global cell ID
MPI gather
• Data in MPAS is spread among many nodes
• Spherical Harmonic routine must be run on single node
Los Alamos National Laboratory
Current Challenge — Getting all the pieces working together inside MPAS-O
Master node
local cell ID
local cell ID
local cell ID
local cell ID
global cell ID
MPI scatter
• Data in MPAS is spread among many nodes
• Spherical Harmonic routine must be run on single node
• After SAL calculation, data must be sent out to original nodes
Los Alamos National Laboratory
15
Thank you!
Los Alamos National Laboratory
16
References:Arbic, Brian & Alford, Matthew & Ansong, Joseph & Buijsman, Maarten & Ciotti, Robert & Farrar, J. & Hallberg, Robert & Henze, Christopher & Hill, Christopher & Luecke,
Conrad & Menemenlis, Dimitris & Metzger, E. & Müeller, Malte & Nelson, Arin & Nelson, Bron & Ngodock, Hans & Ponte, Rui & Richman, James & Savage, Anna & Zhao, Zhongxiang. (2018). A Primer on Global Internal Tide and Internal Gravity Wave Continuum Modeling in HYCOM and MITgcm. 10.17125/gov2018.ch13
Brian K Arbic, Richard H. Karsten, Chris Garrett. On Tidal Resonance in the Open Ocean and the Back-Effect of Coastal Tides upon Open-Ocean Tides. ATMOSPHERE-OCEAN 47 (4) 2009, 239–266 doi:10.3137/OC311.2009
Einšpigel, D., Martinec, Z. (2017), Time-domain modeling of global ocean tides generated by the full lunisolar potential. Ocean Dynamics, 67, 165–189. doi: 10.1007/s10236-016-1016-1.
“Gaussian Grids” European Centre for Medium-Range Weather Forecasts. https://confluence.ecmwf.int/display/FCST/Gaussian+grids. Accessed 15 July 2020.
Haigh, I. D., Pickering, M. D., Green, J. A. M., Arbic, B. K., Arns, A., Dangendorf, S., et al. (2020). The tides they are a-changin': A comprehensive review of past and future nonastronomical changes in tides, their driving mechanisms and future implications. Reviews of Geophysics, 57, e2018RG000636. https://doi.org/10.1029/2018RG000636
“MPAS Overview” https://mpas-dev.github.io, Accessed: 26 Jun 2020
Müller, M., Arbic, B. K., and Mitrovica, J. X. (2011), Secular trends in ocean tides: Observations and model results, J. Geophys. Res., 116, C05013, doi:10.1029/2010JC006387.
Petersen, Mark Roger Asay-Davis, Xylar Storm Jacobsen, Douglas William Maltrud, Mathew Einar Ringler, Todd Darwin Van Roekel, Luke Veneziani, Carmela Wolfram, Phillip Justin Jr. MPAS-Ocean Model User's Guide Version 6.0. Report (2018)
Ringler, T., Petersen, M., Higdon, R. L., Jacobsen, D., Jones, P. W., & Maltrud, M. (2013). Ocean Modelling. Ocean Modelling, 69(C), 211–232. doi:10.1016/j.ocemod.2013.04.010
Schaeffer, N. (2013), Efficient spherical harmonic transforms aimed at pseudospectral numerical simulations, Geochem. Geophys. Geosyst., 14, 751– 758, doi:10.1002/ggge.20071.
Schindelegger, M., Green, J.A.M., Wilmes, S.-B., Haigh, I.D. (2018), Can we model the effect of observed sea level rise on tides? Journal of Geophysical Research: Oceans, 123, 4593–4609. doi: https://doi.org/10.1029/2018JC013959.
V. Balaji, Byron Boville, Samson Cheung, Tom Clune, Nancy Collins, Tony Craig, Carlos Cruz, Arlindo da Silva, Cecelia DeLuca, Rosalinda de Fainchtein, Rocky Dunlap, Brian Eaton, Steve Goldhaber, Bob Hallberg, Tom Henderson, Chris Hill, Mark Iredell, Joseph Jacob, Rob Jacob, Phil Jones, Brian Kauffman, Erik Kluzek, Ben Koziol, Jay Larson, Peggy Li, Fei Liu, John Michalakes, Raffaele Montuoro, Sylvia Murphy, David Neckels, Ryan O Kuinghttons, Bob Oehmke, Chuck Panaccione, Daniel Rosen, Jim Rosinski, Mathew Rothstein, Kathy Saint, Will Sawyer, Earl Schwab, Shepard Smithline, Walter Spector, Don Stark, Max Suarez, Spencer Swift, Gerhard Theurich, Atanas Trayanov, Silverio Vasquez, Jon Wolfe, Weiyu Yang, Mike Young, Leonid Zaslavsky. Earth System Modeling Framework: ESMF User Guide Version 8.0.0. 13 Oct. 2019.