1

Douglass E. PostChair, DOE Fusion Simulation Project

Steering Committee(D. Batchelor—ORNL, R. Bramley—U. of Indiana,

J. Cary—U. of Colorado, R. Cohen—LLNL, P. Colella—LBL, S. Jardin—PPPL, D .Post—LANL)

June 4, 2004PASCI meeting, Princeton, NJ

What are the scientific issues for The Fusion Simulation Project?

PRINCETON PLASMA PHYSICS LABORATORY

PPPL

October 14, 2003 2

Fusion Simulation capability has three levels of maturity.

• 3 major task categories—an R&D challenge– “New research”

• Things that we don’t know how to do• In conjunction with existing theory/computational program and SCIDAC,

launch R&D initiatives to begin the process of developing practical modules, will likely take several stages to mature

• Strongest research focus

– “Focused Integration Initiatives”• Things we have a good idea of how to approach, but need some R&D• Fund them to get experience, develop algorithms, deliver capability to

program in short to medium term• Great opportunity for incremental delivery

– “Whole Device modelling”• Things we really know how to do• Fund them to deliver capability in short term, learn and gain experience,

develop prototypes for final simulation code• Good prototypes already exist• EU effort beginning with this emphasis

3

The PSACI asked us to identify the scientific issues the FSP will address.

• FSP will address the overarching question in the DOE Office of Science Strategic Plan: “How can we create and stably control a high-performance fusion plasma?”

• In particular many of the major outstanding scientific questions of vital importance to the future of the program:1. What are the conditions under which a sequence of nonlinear

magnetohydrodynamic (MHD) events lead to a catastrophic disruption of the fusion plasma?

2. What is the physics of the plasma edge whose structure critically affects global confinement of heat and particles?

3. What is the physics of fine scale turbulence on global transport length and time scales?

4. How do we best use heat, particle, and current sources to control plasmas to achieve high levels of fusion power?

4

Planning path

UltimateCapability

GlobalSimulationFor burningPlasmasDischarge controlTheory&Model testingScenarioOptimization

Development path2005 202020152010

PhysicsMathComputer ScienceData analysisActuator controls

2005 202020152010

Ne

w re

sea

rch

ele

me

nts

Extended MHD 1Extended MHD 2Extended MHD 3

Well defined elements:Neutral beam heating/CDEdge neutrals

Well defined FIIs: RF heating CD+MHD stabilityEdge plasma transport +other edge physics

Make useful contributions Gain experienceEstablish credibility

Wh

ole

de

vice

pro

totyp

es

Address unsolved problemsDevelop concepts for solutionsDevelop algorithms/methods

FIIs w

ith sh

ort

Te

rm a

nd

in

term

ed

iate

term

b

en

efit

Well defined FIIs

Core Transport 1Core Transport 2Core Transport 3 Core Transport

Extended MHD

5

Multiple times are a challenge.

6

FSP has always recognized the central role of R&D.

7

In 2002, US Fusion Community proposed a “Fusion Simulation Project”

• In 2002, FESAC and DOE proposed a “Fusion Simulation Project”

• Ramp up to $20 M per year in 3 to 4 years, begin with $4M in FY05

Fusion Simulation Project, Integrated Simulation and Optimization of Fusion Systems

Jill Dahlburg, General Atomics (Chair)James Corones, Krell Institute, (Vice-Chair)Donald Batchelor, Oak Ridge National LaboratoryRandall Bramley, Indiana UniversityMartin Greenwald, Massachusetts Institute of TechnologyStephen Jardin, Princeton Plasma Physics LaboratorySergei Krasheninnikov, University of California - San DiegoAlan Laub, University of California - DavisJean-Noel Leboeuf, University of California - Los AngelesJohn Lindl, Lawrence Livermore National LaboratoryWilliam Lokke, Lawrence Livermore National LaboratoryMarshall Rosenbluth, University of California - San DiegoDavid Ross, University of Texas - AustinDalton Schnack, Science Applications International Corporation

Office of Science

U.S. Department of Energy

On the Status of MHD Modeling

S.C. JardinPPPL

meeting of the FSP planning steering committeeMay 18-20, 2004 Boulder, Colorado

Office of Science

U.S. Department of Energy

Outstanding Problems in MHD

Full nonlinear sawtooth oscillation modeling in fusion-grade plasmas

Tearing mode and neoclassical tearing mode excitation in high-beta plasmas

Nonlinear evolution and control of resistive wall modes, including toroidal flows.

Effects of Fast Ions

Edge MHD-type instabilities

Disruptions and Vertical Displacement Events

Pellet Injection fueling of a Burning Plasma

For each of these, the goal is to have both a detailed 3D complete model that can be invoked, and a simplified approximate model for the WDM code

Office of Science

U.S. Department of Energy Full nonlinear sawtooth oscillation modeling in fusion-grade plasmas

What will be the period and inversion radius of the sawtooth oscillation in an ITER-class burning plasma, as a function of plasma current and pressure?

What physics underlies complete and incomplete reconnection during the sawtooth?

Under what conditions will the sawtooth instability trigger the onset of a metastable island (neoclassical tearing mode) or lead to a disruption?

Can RF be used effectively to stimulate small sawteeth in ITER?

• Requires Extended MHD model (more terms than in resistive MHD) to get correct onset criteria, crash time

• Requires Hybrid description including energetic alpha particles in ITER

• Multiple space-scales due to small reconnection regions

• Free boundary with vacuum region required.

• Extensive validation program required.

• Coupling to RF modeling desired.

Cohen FSP_edge 5/18-20/2004 -11-

EDGE SIMULATION ISSUES FOR THE FSP

Ronald H. Cohen

Presentation to

FSP Workshop, Boulder

May 18-20, 2004

Acknowledgments: W. Nevins, T. Rognlien, M. Umansky, X. Xu

Cohen FSP_edge 5/18-20/2004 -12-

Integrated edge simulation vital for ITER/BP

• Edge Pedestal establishes T, n at core-edge interface– Dominant control parameters for core confinement– Important transient phenomena (see next slide)

• Transport through edge region strongly impacts ability to handle power & He exhaust and particle fueling

Projections of fusion gain for ITER

Cohen FSP_edge 5/18-20/2004 -13-

What is needed for a predictive capability

• Need to model a range of phenomena/processes:– Microturbulence– Intermittant meso-scale transport (“blobs”)– Formation of “transport barriers” (“L-H transition”)– Edge-Localized Modes (ELMs)– Bootstrap current– Neoclassical (drift-induced) transport– Dust generation/transport– Neutral-particle and impurity transport– Wall erosion processes

Cohen FSP_edge 5/18-20/2004 -14-

What is needed for a predictive capability(cont)

• Essential simulation ingredients– 3-D multi-species plasma model; kinetic– Electromagnetic (including MHD)– Field-line-aligned grid with X-point geometry– Neutrals– Impurities– Plasma-wall interactions– Spatial domain spanning from material wall in through the

pedestal– Temporal evolution spanning scales from turbulence

(µsec) through multiple ELM cycles, macroscopic evolution in presence of recycling (100’s of ms)

• Of necessity, this is integrated simulation!

Cohen FSP_edge 5/18-20/2004 -15-

Roadmap to Predictive Edge Simulation

Where we’ve been: mostly isolated pieces, limited parallelism

UEDGE, B2 (2-D transport)

BOUT (3-D 2-fluid turbulence)

WBC, Redep (near-surface transp/ chemistry)

Filling in the gaps

Coupled fluid turbulence/transport (LLNL-LDRD)

Comprehensive fluid turbulence (community SciDAC proposal)

Kinetic edge code (turbulence timescales) (LLNL, NYU, Helsinki)

Fusion Simulation Project:

EDGE FII

Where it would come together

DIVIMP, MCI (impurity transport)

DEGAS, NUT (kinetic neutral transport)

MD-based models of plasma-wall interaction (Argonne, U. Ill, LLNL)ELITE (Edge MHD stability)

etc

Fusion Simulation Project:

Global model

Cohen FSP_edge 5/18-20/2004 -16-

We would bring it all together in the FSP

• Edge is a logical choice for an FII– Bring together:

• Multiple-timescale techniques (to extend to discharge timescales)• Full edge plasma physics

– Initially fluid (from SciDAC)– Add kinetic option when available

• Physics-based (ultimately, MD-based) plasma-wall interactions • Monte-Carlo neutrals• Evolving MHD edge equilibrium

Cohen FSP_edge 5/18-20/2004 -17-

An edge FII must address computational challenges

• Here are a few:– Multiple timescales– Robust solution algorithms– Efficient utilization of many CPU’s for heterogeneous

physics– Open-closed flux surface topology change

• Singularity at x point• Different b.c.’s

– Spatial scales -- broad range but much overlap• High-order discretizations• Preconditioners• Multi-dimensional parallelization• AMR

Cohen FSP_edge 5/18-20/2004 -18-

Plausible stages of computational development-1

• Baseline approach (SciDAC?)– Multi-fluid 3-D plasma with:

• Static field-line-aligned grid• Fully implicit advance (Krylov/PVODE)• Spatial-discretization-independent representation of physics • Finite difference as initial spatial discretization, other options

as they are available

– Fluid neutrals– Fluid impurities– Phenomenological plasma-wall interaction models– Component framework (e.g. CCA)– Science applications: Dynamics for single ELM crash;

turbulence and neoclassical transport between crashes

Cohen FSP_edge 5/18-20/2004 -19-

Plausible stages of computational development-2

• Extension to long timescales– Coupling of turbulence and transport timescales via

relaxed iteration– 3-D turbulence; 2-D transport equations– Improved preconditioning– AMR, if warranted– Science application: pedestal evolution on transport

timescale

Cohen FSP_edge 5/18-20/2004 -20-

Plausible stages of computational development-3

• Extension to kinetic models and physics-based plasma-wall interactions:– Complete development of 3-D edge gyrokinetic turbulence

model (begun under other funding). Nominally: 5-D continuum (alternate: PIC)

– Substitute gyrokinetic turbulence model and its 2-D reduction for transport for fluid-based turbulence and transport models

– Replace fluid neutrals model by Monte-Carlo neutrals, or hybrid– Include radiation transport, especially for hydrogen lines– Extend gyrokinetic plasma model to include kinetic impurities– Retain fluid model option, with added closure terms to model

kinetic effects (fast-running option)– Replace phenomenological plasma-wall models by models

based on physics computations (e.g. molecular dynamics and near-sheath plasma) of wall interactions

– Science applications: previous problems at higher temperature and/or lower density

21

These issues are driven by the requirement to simulate all aspects of a burning plasma.

• Solve conservation equations for density, momentum, and energy on a set of grids– Calculate source and sink terms– Calculate coefficients

• Individual elements– MHD– Transport– Edge– RF, CD, fuelling, impurities, NBI, plasma control,

diagnostics, physical data

• Integrate them

22

We are asking for $2M to get started. Why?

• We are asking for $2M to get started. What will we spend it on?– It’s $1M from OFES, $1M from OSACR– An FTE at a lab costs roughly $330k fully burdened. Universities are a little

less. $1M = 3 FTEs

• 3 FTE’s ~ 6 half-time staff– OFES: Project leader, MHD, transport, edge, whole device modelling,

sources, all at 0.5– OSACR: 0.5 Project leader, 1 computational mathematicians, 0.5

parallelization, 0.5 computational infrastructure, 0.5 collaboration infrastructure

• Essential to leverage base program• $2M may seem like a huge sum. It’s actually $1M from OFES.

OASCR will fund “computer science” activities, not physics. – It’s actually small for planning a $300M project. The usual rule of

thumb is that the design phase of a project costs about 10% of the total project.

– One reason that $1M is a substantial sum on the SciDac scale is that SciDac is highly leveraged. I don’t have the exact numbers for the MHD SciDac effort, but if 20 people are working on it and the funding is about $500k, that’s an average of $25k per person, about 8% of what that person actually costs.

23

Strawman WBS for pre-conceptual design

• OFES ( all at about 1/2 FTE of senior staff from many different institutions): – Project leader—overall coordination, ensure that vision is developed, interface with

DOE, ITER, community, ….– MHD—assess state of art, develop plans and requirements for FII’s and R&D, gather

requirements from experimentalists, ITER, etc., coordinate with comp math community on new opportunities,

– Transport, Edge, sources - similar– Whole device modelling—assess state of art, data bases, code libraries, investigate

opportunities for improvement, gather requirements from experimentalists

• OSACR—OSACR community needs to involved from the beginning: – 0.5 Project leader—same as above – 1 FTE of computational mathematicians—work with fusion community to identify

opportunities for improved algorithms, liaison with applications and comp math communities

– 0.5 parallelization and computer infrastructure—assess state of parallelization, advantages and opportunities, develop appropriate strategies, interface with platform issues

– 0.5 computational and software infrastructure—assess state of the art, identify issues and opportunities for fusion community, develop plan, begin prototyping

– 0.5 collaboration infrastructure—assess state of the art, identify issues and opportunities, begin prototyping using pre-conceptual design team,

24

Senior people are interested in getting involved

• Most of the present FSP committee is interested in a half time involvement– E.g. Phil Colella has expressed a strong interest, other comp

math people are interested, liaison groups like D. Schissel’s group are interested, but project must have a good chance of becoming real!!!

• Many others in the OFES and OASCR communities have expressed interest

• FSP is a major new activity, and a path for the future• EU and Japanese groups have expressed a strong

interest in collaboration• ITER physics groups have also expressed a strong

interest

October 14, 2003 25

ASCI had a large R&D effort in their large Program ~ $747 M/year (proposed FY02)

Program element Budget($M)

Applications—Development of applications and algorithms 147MPM—Materials and Physics Models 68PSE-Problem Solving Environment 54DisCom—High Speed Classified Network 29Views—visualization 80Ongoing Computing—Operation of present computers 97Platforms—New computers 153Institutes—Computer Science and Math Research 16Path Forward—Enabling Technologies 24Verification and Validation—V&V 42ASCI Alliances and University Programs 25 + 12

October 14, 2003 26

Summary Requirements for ICF/HEDP Simulation Capabilities and Features, From LA-UR-03-0960, D. Post, May 26, 2002 Challenging but easier than MFE

27

An appropriate balance between a strong R&D effort and delivering a product must be maintained!

• Tension between: – Sufficient freedom to innovate and do research to develop methods to

solve new problems we can’t yet solve– Sufficient structure and accountability to ensure that a useful product is

delivered on time and on budget

• In particular:– Sufficient local authority within the project elements to innovate and

develop the new methods and physics understanding essential for successAnd – Sufficient central authority to make binding decisions on technical merit

rather then political considerations– Sufficient structure and authority to ensure adequate coordination among

the various collocated and non-collocated project elements. – Sufficient structure that the project sponsors (DOE and the fusion

community) can hold the project and the project team members accountable

28

The FSP can use the “lessons learned” from other large scale computing projects.

• Lessons learned are available from other large scale scientific computing projects—we can be more efficient by building on them– ASCI, NNSA– ASCI Alliances– DARPA HPCS– ESMF, climate modelling, environmental sciences,– DOD, NOAA, industry, astrophysics, atomic physics,

hydrology and oil reservoir, CFD, Combustion and fire modelling, Geophysics, engineering, chemistry, solar and planetary physics, …

• DARPA HPCS

29

FSP is an opportunity, perhaps our best opportunity, to get the additional resources we need to make progress.

• The reality of the present budgeting process at DOE is that we need to have a “new, exciting, innovative initiative” to get new funding.

• Otherwise, we will be looking at constant funding or worse for the foreseeable future

• The FSP offers us a chance to reverse that trend. Let’s go for it. • It will not only give us additional plasma physics resources, but

will give us leverage with the comp sci community• OASCR is really enthusiastic about the FSP. Not only do they

want to help us with funding, but are giving us priority for computer time and access to their best people. Let’s let them help us.

• All of us involved in the FSP understand the prime importance of R&D