csiro marine and atmospheric research 1 comparing the formulations of ccam and vcam and aspects of...

38
CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric Research Aspendale, Melbourne PDEs on the Sphere Cambridge 28 September 2012

Upload: reynold-berry

Post on 24-Dec-2015

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 1

Comparing the formulations of CCAM and VCAM and aspects of their performance

John McGregor

CSIRO Marine and Atmospheric ResearchAspendale, Melbourne

PDEs on the Sphere Cambridge

28 September 2012

Page 2: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 2

Outline

• CCAM formulation

• VCAM formulation

• Some comparisons

Page 3: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 3

OriginalSadourny (1972)

C20 grid

Equi-angular C20 grid

Alternative cubic grids

Conformal-cubicC20 grid

Page 4: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 4

The conformal-cubic atmospheric model

• CCAM is formulated on the conformal-cubic grid

• Orthogonal• Isotropic

Example of quasi-uniform C48 grid with resolution about 200 km

Page 5: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 5

CCAM dynamics

• atmospheric GCM with variable resolution (using the Schmidt transformation)

• 2-time level semi-Lagrangian, semi-implicit• total-variation-diminishing vertical advection• reversible staggering

- produces good dispersion properties• a posteriori conservation of mass and moisture

CCAM physics• cumulus convection:

- mass-flux scheme, including downdrafts, entrainment, detrainment

- up to 3 simultaneous plumes permitted• includes advection of liquid and ice cloud-water

- used to derive the interactive cloud distributions (Rotstayn 1997)

• stability-dependent boundary layer with non-local vertical mixing• vegetation/canopy scheme (Kowalczyk et al. TR32 1994)

- 6 layers for soil temperatures- 6 layers for soil moisture (Richard's equation)

• enhanced vertical mixing of cloudy air• GFDL parameterization for long and short wave radiation

Page 6: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 6

Location of variables in grid cellsAll variables are located atthe centres of quadrilateralgrid cells.

However, during semi-implicit/gravity-wave calculations, u and v are transformed reversibly to the indicated C-grid locations.

Produces same excellent dispersion properties asspectral method (see McGregor, MWR, 2006), but avoids any problems of Gibbs’ phenomena.

2-grid waves preserved. Gives relatively lively winds, and good wind spectra.

Page 7: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 7

Reversible staggering

Where U is the unstaggered velocity component and u is the staggered value, define (Vandermonde formula)

•accurate at the pivot points for up to 4th order polynomials

•solved iteratively, or by cyclic tridiagonal solver

•excellent dispersion properties for gravity waves, as shown for the linearized shallow-water equations

Page 8: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 8

Dispersion behaviour for linearized shallow-water equations

Typical atmosphere case- large radius deformation

N.B. the asymmetry of the R grid response disappears by alternating the reversing direction each time step,giving the same response as Z (vorticity/divergence) grid

Typical ocean case- small radius deformation

Page 9: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research

Transformation of 2, 3, 4, 6-grid waves

Page 10: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research

The reversible staggering technique allows a very consistent, and thus more accurate, calculation of pressure gradient terms.For example, in the staggered u equation

the RHS pressure gradient term is first evaluated at the staggered position, then transformed to the unstaggered position for calculation of the whole RHS advected value on the unstaggered grid. That whole term is then transformed to the staggered grid, fully consistent with the subsequent implicit evaluation of the LHS on the staggered grid.

Treatment of pressure-gradient terms

Page 11: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research

Treatment of ps advection near terrain

Pressure advection equation

Define an associated variable, similar to MSLP

which varies smoothly, even over terrain. It is thus suitable for evaluation by bi-cubic interpolation, whilst the other term is found “exactly” by bi-linear interpolation (to avoid any overshooting effects). Formally, get

Page 12: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research

Treatment of T advection near terrainSimilarly to surface pressure advection, define an associated variable

which varies relatively smoothly on sigma surfaces over terrain. Again the second term can be found “exactly” by bi-linear interpolation. A suitable function is

Formally, get

This technique effectively avoids the requirement for hybrid coordinates.

Page 13: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research

a posteriori conservation• a posteriori conservation of mass and moisture• “global” scheme• simultaneously ensures non-negative values• during each time step applies correction to changes

occurring during dynamics (including advection)• correction is proportional to the “dynamics” increment,

but the sign of the correction depends on the sign of the increment at each grid point.

The above are all described in the CCAM Tech. Report

Page 14: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 14

MPI implementation

Remapping of off-processor neighbour indices to buffer region

Indirect addressing is used extensively in CCAM - simplifies coding

Original

Remapped region 0

Page 15: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 15

Typical MPI performance

Showing both Face-Centred (FC) and Uniform Decomposition (UD) for global C192 50 km runs, for 1, 6, 12, 24, 48, 72, 96, 144, 192, 288 CPUs

VCAM a little slower, but is still to be fully optimised

Page 16: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 16

An AMIP run 1979-1995

CCAM

Obs

Tuning/selecting physics options:• In CCAM, usually done with 200 km AMIP runs, especially paying

attention to Australian monsoon, Asian monsoon, Amazon region• No special tuning for stretched runs

DJF JJA

Page 17: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 17

Variable-resolution conformal-cubic grid The C-C grid is rotated to locate panel 1 over the region of interestThe Schmidt (1975) transformation is applied•this is a pole-symmetric dilatation, calculated using spherical polar coordinates centred on panel 1•it preserves the orthogonality and isotropy of the grid•same primitive equations, but with modified values of map factorPlot shows a C48 grid (Schmidt factor = 0.3) with resolution about 60 km over Australia

Page 18: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 18

C48 8 km grid over New Zealand

C48 1 km grid over New Zealand

Grid configurations used to support Alinghi in America’s Cup, Olympic sailing

Schmidt transformation can be used to obtain even finer resolution

Page 19: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 19

Quasi-uniform C48 CCAM grid with resolution about 200 km Stretched C48 grid with resolution about 20 km over eastern Australia

• The 200 km run is then downscaled to 20 km (say) by running CCAM with a stretched grid, but applying a digital filter every 6 h to preserve large-scale patterns of the 200 km run

Preferred CCAM downscaling methodology

• Coupled GCMs have coarse resolution, but also possess Sea Surface Temperature (SST) biases

• A common bias is the equatorial “cold tongue”• First run a quasi-uniform 200 km (or modestly

stretched) CCAM run driven by the bias-corrected SSTs

Page 20: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 20

• Uses a sequence of 1D passes over all panels to efficiently evaluate broad-scale digitally-filtered host-model fields (Thatcher and McGregor, MWR, 2009). Very similar results to 2D collocation method.

• These periodically (e.g. 6-hourly or 12-hourly) replace the corresponding broad-scale CCAM fields

• Gaussian filter typically uses a length-scale approximately the width of finest panel

• Suitable for both NWP and regional climate

Digital-filter downscaling method

Page 21: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research

Nonhydrostatic treatment

Being a semi-Lagrangian model, CCAM is able to absorb the extra phi terms into its Helmholtz equation solver, for “zero” cost

The new dynamical core (VCAM) uses a split-explicit treatment, so the Miller-White treatment would need its own Helmholtz solver, so may use Laprise-style nonhydrostatic treatment for VCAM

Page 22: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research

CCAM simulations of cold bubble, 500 m L35 resolution, on highly stretched global grid

Page 23: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 23

Gnomonic grid showing orientation of the contravariant wind components

Illustrates the excellent suitability of the gnomonic grid for reversible interpolation – thanks to smooth changes of orientation

Page 24: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 24

Nonhydrostatic treatment

Being a semi-Lagrangian model, CCAM is able to absorb the extra phi terms into its Helmholtz equation solver, for “zero” cost

The new dynamical core of VCAM uses a split-explicit treatment, so the Miller-White treatment would need its own Helmholtz solver,

Probably will use Laprise-style nonhydrostatic treatment for VCAM

Page 25: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research

New dynamical core for VCAM - Variable Cubic Atmospheric Model

• uses equi-angular gnomonic-cubic grid - provides extremely uniform resolution - less issues for resolution-dependent parameterizations

• reversible staggering transforms the contravariant winds to the edge positions needed for calculating divergence and gravity-wave terms

• flux-conserving form of equations– preferable for trace gas studies– TVD advection can preserve sharp gradients– forward-backward solver for gravity waves– avoids need for Helmholtz solver– linearizing assumptions avoided in gravity-wave terms

Page 26: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 26

Page 27: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 27

Horizontal advection

Flow=qyVj+1/2

Vj-1/2

ucovUi-1/2

vcov

(qx, qy)

q

Flow=qxUi+1/2

Transverse components (to be included in low/high order fluxes)calculated at the centre of the grid cells (loosely following LeVeque)qx: using dt/2 advection from vcov

qy: using dt/2 advection from ucov

Low-order and high-order fluxes combined using Superbee limiter

High order need covariant vels for LW term. Linear interp for edge values of q?

Cartesian components (U,V,W) of horizontal wind are advected

Page 28: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 28

Solution procedure

• Start loop Start Nx(t/N) forward-backward loop Stagger (u, v) +n(t/N) Average ps to (psu, psv) +n(t/N) Calc (div, sdot, omega) +n(t/N) Calc (ps, T) +(n+1)(t/N) Calc phi and staggered pressure gradient terms, then unstagger these Including Coriolis terms, calc unstaggered (u, v) +(n+1)(t/N) End Nx(t/N) loop

• Perform TVD advection (of T, qg, Cartesian_wind_components) using average ps*u, ps*v, sdot from the N substeps

• Calculate physics contributions• End loop

Main MPI overhead is the reversible staggering at each substep, but this just needs nearest neighbours in its iterative tridiagonal solver. Also message passing is needed in the pressure gradient and divergence calcs

Page 29: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research

500 hPa omega (Jan 1979)

Hybrid coordinates introduced

non-hybrid

Page 30: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research

250 hPa windsin 1-year run

30

VCAMCCAM

Page 31: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 31

DJF JJA

VCAM1-year

CCAM1-year

Same physics

Obsclimate

Page 32: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 32

However, can can see some influence of panel edges on rainfall just south of Australia

Page 33: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 33

Problem caused by spurious vertical velocities at vertices!

Eastwards solid body rotation in 900 time stepsUsing superbee limiter

Page 34: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 34

Spurious vertical velocities reduced by factor of 8 by more-careful calculation of pivot velocities near panel edges

Page 35: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 35

With better staggered velocities at panel edges

Page 36: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 36

Comparisons of VCAM and CCAM

VCAM advantages• No Helmholtz equation needed• Includes full gravity-wave terms (no T linearization needed)• Mass and moisture conserving• More modular and code is “simpler”• No semi-Lagrangian resonance issues near steep mountains• Simpler MPI (“computation on demand” not needed)

VCAM disadvantages• Restricted to Courant number of 1, but OK since grid is very

uniform• Some overhead from extra reversible staggering during sub

time-steps (needed for Coriolis terms)• Nonhydrostatic treatment will be more expensive

Page 37: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research

Tentative conclusions

• Reversible interpolation works well for both CCAM and VCAM

• VCAM seems to perform better than CCAM in the tropics- better rain over SPCZ and Indonesia, possibly by

avoiding linearizing ps term in pressure gradients, and better gravity wave adjustment by not using semi-implicit

- rainfall presently not as good in midlatitudes

37

Page 38: CSIRO Marine and Atmospheric Research 1 Comparing the formulations of CCAM and VCAM and aspects of their performance John McGregor CSIRO Marine and Atmospheric

CSIRO Marine and Atmospheric Research 38

Thank you!