VERA-CS V&VWorkshop on Multi-physics Model Validation

Dave Kropaczek *North Carolina State UniversityCASL Chief ScientistJune 2017

* With input from the CASL RTM, PHI and AMA teams:T. Downar, K. Clarno, B. Collins, B. Kochunas, K.S. Kim, B. Martin, S. Wang, Lee, S. Palmtag, G. Godfrey, U. Mertyurek

2

VERA and VERA-CS

3

Introduction

• Several government and commercial institutions are now developing large-scale computational simulations for massively parallel platforms to simulate the performance of complex, coupled multi-physics phenomena similar to the CASL efforts on nuclear reactor simulation (e.g. Climate / Weather predictions, Biochemical Systems, etc) [Post, 2014]

• Approach:– Exploit the hierarchical nature of the multi-physics solution and first

performing uniformly acceptable V&V on the single physics codes before moving to the coupled physics solutions.

– Identify clear “Quantity of Interest” (QOI) • Challenge problems QOI CRUD, PCI, DNB, RIA, LOCA• QOI for VERA-CS V&V is Core Follow

– The essential distinction between V&V for single and multi-physics codes is the presence of an additional code or module required to do code coupling. • MPACT-ORIGEN-CTF-MAMBA• TIAMAT: BISON-VERA-CS• CICCADA: MAMBA-CFD (CCM+).

4

VERA-CS Validation Plan

A comprehensive validation plan was developed for VERA-CS [Godfrey, 2014] and presented in detail in CASL-U-2014-0185-000.

•3D Core Pin Powers•Intra-Pin Distributions

•Depleted Isotopics•Gamma Transport

•Gamma Scans•Burnup•Radiochemical Assays

•CRUD Deposition

•Criticality•BOL Pin Powers•Temperature Worth

•Critical Boron•Rod Worths•ITC•Flux Maps•T/H Feedback

Operating Power Plants

Critical Experiments

CE Monte Carlo

Fuel Rod PIEs

5

• As part of a FY17 the current status of VERA-CS Verification and Validation for PWR Core Follow operation was assessed and a multi-phase plan was proposed for continuing VERA-CS V&V in FY17 and FY18

• The V&V status was assessed for each single physics code (ie. MPACT, CTF, Multi-group Cross Section Generation, BISON/Fuel Temperature Tables, MAMBA3D)

• The V&V status was assessed for code coupling and a multi-phase VERA-CS VYV plan for FY18 ff. is proposed.

VERA-CS V&V Plan

6

VVI: Predictive Code Maturity Model

7

Single Physics Code V&V

• XSEC Generation– ORNL 51-Grp Library

• MPACT– ORIGEN

• CTF

• BISON / Fuel Temperature Tables

• Other VERA-CS Codes

8

XSEC Library V&V

9

MPACT Verification & Validation PlanVerification (MPACT)

– Source Code Verification• Unit Testing• Regression Testing

– Solution Verification• Mesh Convergence Analysis• Method of Manufactured Solutions

Validation– Critical Experiments

• Babcock & Wilcox Critical Experiments• DIMPLE Critical Experiments

– Transient Tests• Special Power Excursion Reactor Test (SPERT)

– Operating Power Plants (VERA-CS)• Watts Bar Nuclear Plant• BEAVRS

– Post Irradiation Examinations (VERA-CS)

10

Verification - Source Code Unit TestingMETRIC M_libs M_Drivers M-exe MPACT_Utils

Unit Tests 120 6 0 44Regression Tests 0 0 97 0

Coverage 74.1% 70.9% 67.4% 85.2%Lines of Code 88,556 3,942 417 20,587

*Data as of 9/27/16

Note: Previous rev0 M_libs was split into MPACT_Utils and MPACT_libs

1 - Test Server checks for changes every 10 minutes and tests two configurations2 - Tests many more regression tests, performed by CASL and UM test machines3 - Test GCC 4.6.1, 4.7.2, 4.8.1, Intel 12.1.5 with and without MPI & other TPLs4 - Unit tests for solver kernels test against analytic solutions.

Some regression tests compare against analytic solutions.5 - Depletion solver is compared to experimental results.6 - This means analyzing program with Valgrind7 - This means running "gcov" on all tests.

11

MPACT Test Harness (B. Collins)• A Python script MPACTdiff was developed in order to provide the

capability to simplify the process. • The MPACTdiff script works with the MPACTdiff executable which is

used by the regression and validation test suite to ensure that the MPACT solution is unchanged between commits.

12

Expansion of Regression Test Suite

• A suite of regression tests was developed that mimic the capability required to perform the VERA Progression Problems but with minimal computational expense.

• This significantly adds to the coverage of the capabilities in the MPACT test matrix and the matrix has been updated based on these features.

• The motivation for this is that problems 4, 5, 7, 8, 9, and 10 of the VERA Progression Problems all require significant computational resources to complete. Therefore it is not feasible to run these cases on a nightly or weekly basis with the computing resources available.

• Target: 32 cores or less and run as part of the automated test suite including a full cycle depletion and shuffle.

13

Expansion of Regression Test Suite

• Mini-Regression Test Suite– The standard 47 group library

with an 8 group test library– Standard 17x17 pin

assemblies with 5x5 pin assemblies

– A standard 193 assembly core with a 69 assembly core

– Shortened core from 12 feet to 8 feet.

– MPACT parameters are coarsened to 0.08 cm ray spacing, 4 azimuthal angles per quadrant and one polar.

• 32 cores or less and run as part of the automated test suite including a full cycle depletion and shuffle.

Mini Progression Problem 5 (Center Slice)

14

Solution Verification -Method of Manufactured Solutions

• Mesh convergence analysis is limited, but exact solutions are generally possible for a limited set of problems.

• The Method of Manufactured Solutions (MMS) provides an alternative by specifying the solution beforehand and substitute the solution into the equation which the solver claims to have solved. This results in an extra analytical source (Manufactured Source) which can be used in the code as a fixed source problem.

• By comparing the numerical solution from the solver with the manufactured analytical solution and observing the expected rate of convergence in the successive refinements, the numerical code can be verified.

• Oberkampf, 2008: ”the Method of Manufactured Solutions is capable of verifying several numerical aspects in the code, such as the mathematical correctness of the numerical algorithms, the grid-spacing technique, and the absence of coding errors in the software implementation”.

15

MPACT MMS – Fixed Source Problem (J. Wang / B. Martin / B. Collins)

In FY16 [Wang, 2016b], a test suite of three problems involving different solution structures were developed for verification of fixed source problems:

Table 2.4. Testing Suite for Fixed Source Problems.

16

Decouping of Angular and Spatial Error

A method was developed which allows the angular error to be decoupled from the spatial error, enabling the assessment of the convergence with spatial resolution. The method can be generalized to higher dimensions. The steps and equations to remove the angular error are derived in [Wang, 2016a].

( )

( )

( ) ( )

, 4

,

4

angular, spatial,

1 ,

1

1 1 ,

j

j

j j

j m j m Vm j

m j m m mVm mj

m mV Vmj j

j j

E w r d dVV

w r w dVV

r w dV r d dVV V

E E

π

π

ψ ψ

ψ ψ

ψ ψ

= ⋅ − Ω Ω∆

= ⋅ − ⋅ ∆

+ − Ω Ω ∆ ∆ = +

∑ ∫ ∫

∑ ∑∫

∑∫ ∫ ∫

Grid Points per mfp Overall Error Angular Error Spatial Error

16 0.05613 -0.00706 0.0631848 0.01395 -0.00808 0.02203

144 -0.00061 -0.00812 0.00750432 -0.00560 -0.00812 0.002521296 -0.00728 -0.00812 0.000843888 -0.00784 -0.00812 0.00028

17

MMS for Eigenvalue ProblemApplication of MMS to eigenvalue problems was also investigated. Two approaches were developed, one using an inhomogeneous manufactured source (MS) and the other a manufactured cross section (MXS), which were shown to be consistent, albeit being two independent approaches.

( )MMS

1 , 0sin2 2

,1 2sin , 02 2

DDD

τ µπψ τ µ

τπ µ

> Τ + = + < Τ +

It was shown that both k and the cell-averaged and cell-edged scalar flux exhibit the same order of accuracy with spatial grid refinement, consistent with the expected order of accuracy:

18

MPACT Validation - B&W 1484 SummaryEigenvalue Delta-K

Core 1 Core 2 Core 1 Core 2 RangeHelios (2009) 0.99831 0.99634 -169 -366 197

CASMO-5 TCP0 (2009) 0.99957 0.99923 -43 -77 34

CASMO-5M P3 (2009) 0.99998 1.00044 -2 44 46

Kulesza 47g P2 1.00132 0.99910 132 -90 222

Kulesza 47g TCP0 NLC 0.99977 0.99746 -23 -254 231

Kulesza 47g TCPO limited 1.00192 0.99844 192 -156 348

Kulesza 47g TCP0 out-scat 0.99292 0.99497 -708 -503 205

MPACT Input 47g 0.99772 0.99595 -228 -405 178

MPACT 51g E71 TCP0 0.99911 0.99918 -89 -82 7

MPACT 51g E71 P2 1.00080 1.00092 80 92 13

Are we really this good? B&W 1484 does not have measured pin powers

19

Code-to-Code Comparisons

• Created new model that is similar to 1484 core where we can compare MCNP and MPACT– 2D problem, no buckling– Temperature set to MCNP library value– 2 rods added to make the core octant symmetric

• Allows us to evaluate pin powers

20

MPACT/MCNP - Power and DifferencesCore0c Power

1.3387 1.3244 1.2959 1.2538 1.1986 1.1315 1.0541 0.9695 0.8843 0.8140 0.7975 0.95031.3244 1.3101 1.2818 1.2398 1.1849 1.1182 1.0415 0.9583 0.8760 0.8115 0.8050 0.96791.2959 1.2818 1.2536 1.2121 1.1578 1.0919 1.0168 0.9368 0.8613 0.8124 0.8423 1.06671.2538 1.2398 1.2121 1.1711 1.1178 1.0535 0.9812 0.9070 0.8442 0.8258 0.94511.1986 1.1849 1.1578 1.1178 1.0660 1.0043 0.9368 0.8721 0.8305 0.8632 1.08941.1315 1.1182 1.0919 1.0535 1.0043 0.9472 0.8880 0.8394 0.8323 0.95941.0541 1.0415 1.0168 0.9812 0.9368 0.8880 0.8439 0.8247 0.8752 1.09980.9695 0.9583 0.9368 0.9070 0.8721 0.8394 0.8247 0.8678 1.05430.8843 0.8760 0.8613 0.8442 0.8305 0.8323 0.8752 1.05430.8140 0.8115 0.8124 0.8258 0.8632 0.9594 1.09980.7975 0.8050 0.8423 0.9451 1.08940.9503 0.9679 1.0667

Difference (%)0.62 0.71 0.57 0.52 0.55 0.37 0.28 0.25 0.01 -0.20 -0.37 -0.330.71 0.71 0.65 0.60 0.55 0.45 0.35 0.23 0.04 -0.19 -0.44 -0.280.57 0.65 0.51 0.47 0.50 0.32 0.22 0.17 -0.10 -0.37 -0.62 -0.380.52 0.60 0.47 0.42 0.44 0.27 0.17 0.09 -0.21 -0.51 -0.680.55 0.55 0.50 0.44 0.39 0.29 0.16 -0.01 -0.29 -0.65 -0.590.37 0.45 0.32 0.27 0.29 0.09 -0.05 -0.18 -0.54 -0.780.28 0.35 0.22 0.17 0.16 -0.05 -0.22 -0.39 -0.77 -0.710.25 0.23 0.17 0.09 -0.01 -0.18 -0.39 -0.64 -0.550.01 0.04 -0.10 -0.21 -0.29 -0.54 -0.77 -0.55

-0.20 -0.19 -0.37 -0.51 -0.65 -0.78 -0.71-0.37 -0.44 -0.62 -0.68 -0.59-0.33 -0.28 -0.38

Error in converged solution Max=0.71%

21

MPACT Validation - DIMPLE (Yee/Kochunas)

The DIMPLE experimental program considered critical experiments with low enriched uranium dioxide fuel rods containing 3.0 wt.% 235U with light water moderation and reflection. These experiments were performed in the DIMPLE low power reactor at U.K.A.E.A’s Winfrith site during 1983

Config. Code keff

Dev. from Critical (pcm)

S06AMPACT 1.00137 +137

WIMS1 1.00039 +39

S06BMPACT 1.00171 +171

WIMS1 0.99952 -48

22

MPACT Results for DIMPLEThermal Flux

Group 45Fast FluxGroup 5

23

MPACT Results for DIMPLE

ConfigurationAxial B2

[m-2]Code keff

Deviation from Critical

(pcm)

S06A 24.7MPACT (fine) 1.00137 +137

WIMS1 1.00039 +39

S06B 21.1MPACT (fine) 1.00171 +171

WIMS1 0.99952 -48

24

VERA-CS Validation - BEAVRS Cycle 1 Results (Collins 10/5)

0

20

40

60

80

100

0 100 200 300 400 500 600

Perc

ent R

ated

Pow

er

Days

Power Average Power Critical Boron Measurement Flux Map Measurement

25

BEAVRS Cycle 1 Depletion (Collins 10/3)

26

Exposure Power Boron Flux Map Comparisons

GWD/MT EFPD [%] Meas. Calc. Diff. Radial RMS 3D RMS Delta A/O0 0 0 975 958 17 2.44% 5.14% -2.23%

0.268 6.4 48.69 703 696 7 -- -- --1.023 24.5 98.67 626 601 25 1.71% 4.54% -0.46%1.184 28.4 0 633 -- -- -- -- --1.187 28.5 0 633 -- -- -- -- --1.296 31.1 62.78 638 652 -- 3.49% 5.46% -2.20%1.507 36.1 99.78 610 601 9 0.99% 3.14% -0.52%2.163 51.9 99.98 623 582 40 1.35% 3.02% -0.22%3.297 79.1 93.78 580 563 17 0.85% 3.11% 1.22%4.614 110.6 99.6 532 503 29 0.91% 3.79% -1.63%5.713 137 98.9 479 452 27 -- -- --5.734 137.5 0 478 -- -- -- -- --5.779 138.6 0 476 -- -- -- -- --

BEAVRS Results:

BOC -> MOC

27

Exposure Power Boron Flux Map Comparisons

GWD/MT EFPD [%] Meas. Calc. Diff. Radial RMS 3D RMS Delta A/O6.013 144.2 63.65 461 -- -- 1.08% 4.46% -2.65%6.491 155.7 99.7 444 415 30 1.28% 4.36% 1.55%7.508 180.1 99.3 384 353 31 0.90% 3.47% 0.95%8.701 208.7 99.86 310 284 26 1.00% 3.55% 0.98%9.804 235.1 99.51 248 218 30 -- -- --11.085 265.8 99.91 162 135 27 1.21% 4.01% -0.75%12.342 296 99.79 70 50 20 1.27% 3.73% -1.01%12.677 304 0 52 -- -- -- -- --12.694 304.4 0 51 -- -- -- -- --12.74 305.5 84.1 49 51 -2 -- -- --12.916 309.7 84.48 39 53 -14 1.45% 4.34% 1.79%13.31 319.2 84.94 18 17 1 -- -- --13.411 321.6 70 13 40 -27 -- -- --13.604 326.2 69.86 2 28 -26 1.48% 4.59% 1.00%13.645 327 2 0 0 -- -- -- -- --

BEAVRS Results:

MOC -> EOC

28

VERA-CS Validation - Watts Bar Cycles 1-12 (Godfrey, 2015)

Proposed Metrics for “Red Flagging” Core Follow Calculations:

At startup:o HZP boron: ± 20 𝒑𝒑𝒑𝒑𝒎𝒎o Rodworth: ± 7 %o ITC: ±1 𝑝𝑝𝑐𝑐𝑚𝑚/𝐹𝐹

At every statepoint:o HFP boron: ±35𝒑𝒑𝒑𝒑𝒎𝒎

o AO: ±3%o Pin Power Distribution/ Peaking factors: ± 2 %

29

VERA-CS Validation - Post Irradiation Exams(U. Mertyurek / Kevin Clarno)

• The purpose of the Post Irradiation Examination portion of the validation plan proposed in [Godfrey, 2014] is to demonstrate the accuracy of the isotopic depletion and decay calculations in VERA-CS

• The work performed in FY16 focused on a publically available set of PWR post irradiation examination measurements [Mertyurek, 2016].

ReactorAssembly

TypeNumber of

SamplesEnrichment

(235U %)Fuel Type Burnup

(GWd/MTU)

Takahama-3a 17×17 WE 3 4.11-2.63(Gd2O3)

UO2, UO2-Gd2O3

16.44-47.5

Calvert Cliffs-1b

14×14 CE 2 3.04 UO2, 27.35-44.34

Göesgen15x15 WE 3 4.3 UO2, 46-67.9

30

Takahama PIEs

Solution ParameterPWR

Guide Tube/Water Rod Mesh 3Fuel Mesh 3

Number of Inner iterations 1Up scatter 1

Max. Number of Outer iterations

500

Cross Section Library mpact47g_70s_v4.0_11032014.fmt

The Takahama-3 data [Nakahara, 2002] was taken from three fuel rods (SF95, SF96, SF97) which were removed from two 17×17 WE fuel assemblies. Sixteen samples were measured from the three fuel rods, but only three samples were modeled for this initial validation.

31

-40%

-30%

-20%

-10%

0%

10%

20%

30%

40%

50%

C/M-

1

SF96-2

SF96-2(MPACT) SF96-2(TRITON)

Takahama: SF96-2

32

-30%

-20%

-10%

0%

10%

20%

30%

C/M-

1

SF96-4

SF96-4(MPACT) SF96-4(TRITON)

Takahama: SF96-4

33

Takahama: SF97-5

-30%

-20%

-10%

0%

10%

20%

30%

C/M-

1

SF97-5

SF97-5(MPACT) SF97-5(TRITON)

34

Summary of Results• Takahama: VERA-CS results show good agreement with the

measurement for important actinides. 235U, 239Pu, 241Pu, 240Pu all show less than 5% difference for SF96 samples. Although relative differences for SF97 sample isotopics are relatively higher, the only significant difference is in 241Pu (7%).

• Calvert Cliffs: Again, VERA-CS isotopic predictions are in good agreement with measurements for all U and Pu vectors except 238Pu for which the difference was as large as 10% but is similar to SCALE 6.1 TRITON results.

• Göesgen:The GGU1 and GGU2-2 results are consistent with Takahama-3 and Calvert Cliffs-1 results. U and Pu vectors except 238Pu show less than 5 % difference with measurements. GGU2-1 results show elevated differences for 235U(-8%) and 239Pu (-13%) isotopes. These are the only measurements in which the SCALE 6.1 TRITON/NEWT isotope predictions are better than MPACT. Future work will include a detailed assessment of the MPACT model for this case.

35

Single Physics Code V&V

• XSEC Generation– ORNL 51-Grp Library– HELIOS v2.0

• MPACT– ORIGEN

• CTF

• BISON / Fuel Temperature Tables

• Other VERA-CS Codes

36

VVI Report: Single Physics Code V&V I0=Low 3=high

PCMM Scoring for CIPS (Jones, 2017)

37

V&V Report: Single Physics Code V&V II

PCMM Scoring for PCI (Jones, 2017)

38

Summary• The V&V status of each of the single physics

codes systems currently used for core follow analysis (ie MPACT, CTF, MultigroupCross Section Generation, and BISON / Fuel Temperature Tables) appear to have achieved an acceptable level of V&V, but specific actions are proposed in Phase I(FY17) to achieve a uniformly acceptable level of V&V.

• V&V activities were identified for other VERA-CS codes which are currently ongoing development in FY17 (e.g. TIAMAT, MAMBA3D) and proposed Phase II(FY18) in which those codes will also reach an acceptable level of V&V, and then be used to perform core follow calculations as part of VERA-CS.

• The possible extension of VERA-CS to other reactor types (e.g. BWR) and the corresponding V&V would be addressed in Phase III

39