54 4 Research Groups’ Essentials

4.3 Research Group 3 “Numerical Mathematics and Scientific

Computing”

RG 3 studies the development of numerical methods, their numerical analysis, and it works atimplementing software for the numerical solution of partial differential equations. Many of theresearch topics have been inspired by problems from applications. Below, a selection of topicsof the group is briefly described. Further topics include discretizations of convection-diffusionequations, which are at the same time accurate, efficient, and free of unphysical oscillations, andconservation law preserving finite element methods for equations from fluid mechanics. Researchsoftware based on the developed methods is used in applications like semi-conductor devicesimulation (in collaboration with RG 1 Partial Differential Equations and RG 2 Laser DynamicsLink?), the simulation of problems from hemodynamics and cancer growth, of electrolytes andelectrochemical systems (in collaboration with RG 7 Thermodynamic Modeling and Analysis ofPhase Transitions; see also the Scientific Highlights article on page 21, and the development ofalgorithms for reduced-order modeling.

A novel coupled simulation method for stochastic particle systems

Stochastic particle systems can be modeled by population balance systems (PBS). PBSs areencountered in chemical engineering, meteorology, oceanography, or biomedicine. The type ofparticles depends on the application area. In chemical engineering, particles are, e.g., crystals,in meteorology, atmospheric pollutants, or in oceanography, sediment particles that are trans-ported by marine currents. PBSs describe the development of the particle population itself, aswell as of the surrounding flow field, its temperature, and the concentration of transported dis-solved species. Thus, PBSs comprise multiple interaction phenomena, and they pose severalnumerical challenges; see, e.g., [5].

Fig. 1: Snapshot of themass of crystalline aspirin ina flow tube crystallizer, dueto attachment growth ofdissolved aspirin from thesurrounding fluid. The flowis from left to right, theupper boundary is a solidwall, and the lower boundaryis the symmetry axis.

Together with RG 5 Interacting Random Systems, stochastic-deterministic methods for the solu-tion of PBSs were developed. The temperature, concentration, and flow fields are modeled withpartial differential equations, using advanced finite element methods for their simulation. Highlydeveloped stochastic methods (kinetic Monte Carlo methods) are utilized for simulating the parti-cle population. The coupled simulation method is based on a suitable splitting strategy for PBSsand on using two specialized in-house research codes: PARMOON (finite element fluid dynamics,

Annual Research Report 2017 (Draftcopy of March 1, 2018, 14:10)

4.3 RG 3 Numerical Mathematics and Scientific Computing 55

RG 3) and Brush (stochastic particle methods, RG 5). An interface between these codes wasimplemented, interchanging information like velocity, particle positions, or particle properties. Anefficient communication between both codes was achieved.

In the first step, an application from chemical engineering was studied: a flow crystallizer for

Fig. 2: Particle sizedistribution of inlet andoutlet crystal fraction of anaspirin flow crystallizer,computed with the newcoupled simulation method

the production of aspirin. The considered problem can be modeled with an axisymmetric setup;see Figure 1. In an experimental paper, well-controlled setups are reported, exploiting surfacegrowth and particle collision growth. These particle interaction phenomena entered the stochas-tic simulation. A temperature and mass balance equation and the flow field were dealt with bythe continuous part of the simulation. This combination of methods enabled good reproductionof the experimental results for four different setups, in reasonable computing time, compare Fig-ure 2. The blue histogram shows the inlet crystal size distribution, i.e., the crystal fraction thatwas pumped into the crystallizer. The flatter, pink histogram is the simulation result, which showsdistinctly the effect of particle growth in the tube crystallizer.

The newly developed method is well suited for systems of particles with multiple inner coordinates.Simulations of applications with such particles are future work. In addition, the extension of themethod to three dimensions and its implementation on parallel computers are planned.

Tetrahedral mesh improvement using moving mesh smoothing, lazy searching flips, andradial basis functions (RBF) for surface reconstruction

TETGEN is a C++ program and library for generating tetrahedral meshes of 3D domains [4]. It isa long-term research project of WIAS. Its goals are to investigate the mathematical problems, todevelop theoretically guaranteed algorithms, and to implement robust, efficient, and easy-to-usesoftware. Recently, a new algorithm on mesh improvement was developed [2].

Given a tetrahedral mesh and objective functionals measuring the mesh quality, which take intoaccount the shape, size, and orientation of the mesh elements, the aim is to improve the meshquality as much as possible. In this new algorithm, the recently developed flipping and smoothingmethods were combined into one mesh improvement scheme and applied in combination with asmooth boundary reconstruction via radial basis functions; see Figure 3.

Smoothing

SmoothingChange position, keep connectivity

=)

Dassi, Kamenski, Si. Mesh improvement: MMPDE-smoothing & lazy flip · NUMGRID2016 · Page 4/27

Moving mesh smoothing

Mesh definition by a mapping (mesh metric):

K̂K

JK :=�F�1

K

�0

|K| = det JK�1 · |K̂|

F�1K

FK

Minimize the meshing functional (moving mesh smoothing):

Ih =X

K

|K| G (JK , det JK) ⇡Z

⌦G (J, det J)

with G(J, det J) = ✓�tr�JJT

�� dp2 + (1 � 2✓) d

dp2 (det J)p.

Then, the vertex velocities are

dxi

dt= �

✓@Ih

@xi

◆T

.

Dassi, Kamenski, Si. Mesh improvement: MMPDE-smoothing & lazy flip · NUMGRID2016 · Page 5/27

Edge flipping

Edge flippingKeep position, change connectivity

d

a

bc

e

3-2-flip

a

b

a

b

p1

p0

p3

p2

p1

p0

p3

p2

4-4-flip

Dassi, Kamenski, Si. Mesh improvement: MMPDE-smoothing & lazy flip · NUMGRID2016 · Page 8/27

Lazy flip

The whole process as a tree:

Important points:

⌅ we only remove the edge ab if we arrive a leaf,

⌅ once we arrive at the leaf, we are done.

This flip is “lazy”: it does not explore other branches.

Dassi, Kamenski, Si. Mesh improvement: MMPDE-smoothing & lazy flip · NUMGRID2016 · Page 14/27

flipping — keep position, change connectivity

``Lazy-searching” flipping

Smoothing — change position, keep connectivity

Moving mesh smoothingFig. 3: Mesh improvementoperations

Numerical studies show that the combination of these techniques into a mesh improvement frame-work achieves results that are comparable and even better than the previously reported ones; see

Annual Research Report 2017 (Draftcopy of March 1, 2018, 14:10)

56 4 Research Groups’ Essentials

examples in Figure 4.

Fig. 4: Numerical studiesthat show the improvementof the mesh quality with thenew algorithm proposedin [2]

Rand1 example

Initial Optimized

Dassi, Kamenski, Si. Mesh improvement: MMPDE-smoothing & lazy flip · NUMGRID2016 · Page 23/27

initial improved

Rand1 example (initial mesh: #Th = 5 104)

New Method Stellar

#Th = 3 528

✓min,Th= 40�

✓max,Th= 130�

µTh= 69.70�

�Th= 15.74

✓min,Th= 39�

✓max,Th= 138�

µTh= 70.30�

�Th= 20.18 #Th = 1 186

#Th = 3 897

✓min,Th= 12�

✓max,Th= 156�

µTh= 69.25�

�Th= 23.40

✓min,Th= 8�

✓max,Th= 165�

µTh= 69.98�

�Th= 25.60 #Th = 5 733

CGAL mmg3d

Dassi, Kamenski, Si. Mesh improvement: MMPDE-smoothing & lazy flip · NUMGRID2016 · Page 24/27

aspect ratio

Initial Final Cross section✓min,Th = 31�

✓max,Th = 138�

µTh = 70.69�

�Th = 18.16

#Th = 17 999

Figure 10: Example of tetrahedral mesh improvement with a curved (reconstructed) surface.

Surface mesh Cross section New Method

✓min,Th = 31�

✓max,Th = 138�

µTh = 70.51�

�Th = 17.84

#Th = 640 993

Stellar

✓min,Th = 26�

✓max,Th = 148�

µTh = 70.78�

�Th = 27.50

#Th = 442 838

Figure 11: Spine example: the initial mesh with #Th = 688 420 and the final optimized mesh.

Surface mesh Cross sectionNew Method

✓min,Th = 16�

✓max,Th = 162�

µTh = 71.02�

�Th = 20.10

#Th = 246 203

Stellar

✓min,Th = 13�

✓max,Th = 163�

µTh = 71.46�

�Th = 24.99

#Th = 196 450

Figure 12: Elephant example: the initial mesh with #Th = 260 401 and the final optimized mesh.

12

Initial Final Cross section✓min,Th = 31�

✓max,Th = 138�

µTh = 70.69�

�Th = 18.16

#Th = 17 999

Figure 10: Example of tetrahedral mesh improvement with a curved (reconstructed) surface.

Surface mesh Cross section New Method

✓min,Th = 31�

✓max,Th = 138�

µTh = 70.51�

�Th = 17.84

#Th = 640 993

Stellar

✓min,Th = 26�

✓max,Th = 148�

µTh = 70.78�

�Th = 27.50

#Th = 442 838

Figure 11: Spine example: the initial mesh with #Th = 688 420 and the final optimized mesh.

Surface mesh Cross sectionNew Method

✓min,Th = 16�

✓max,Th = 162�

µTh = 71.02�

�Th = 20.10

#Th = 246 203

Stellar

✓min,Th = 13�

✓max,Th = 163�

µTh = 71.46�

�Th = 24.99

#Th = 196 450

Figure 12: Elephant example: the initial mesh with #Th = 260 401 and the final optimized mesh.

12

aspect ratioimproved mesh using RBF

PARMOON – A software platform for problems from fluid dynamics

Fig. 5: Solution to the Na-vier–Stokes–Darcy coupledproblem for a “river bed”computed by PARMOON

PARMOON is a flexible finite element package for the solution of steady-state and time-dependentconvection-diffusion-reaction equations, incompressible Navier–Stokes equations, and coupledsystems consisting of these types of equations, like population balance systems or systems cou-pling free flow and flows in porous media. PARMOON abbreviates Parallel Mathematics and objectoriented Numerics and is developed in cooperation with the Computational Mathematics Groupof Prof. Sashikumaar Ganesan at the Department of Computational and Data Sciences (IndianInstitute of Science, Bangalore) and the group of Prof. Gunar Matthies (TU Dresden).

PARMOON is the successor of MOONMD, whose development started in 1996 in Magdeburgand whose reference paper has more than 100 citations. Starting in 2013, the parallelization fordistributed memory systems and the re-implementation of large parts of the code base led to thenew name PARMOON.

One of the main features of both is the clear separation of geometry and finite elements as one canfind them in textbooks. Well over 100 finite elements are implemented in one, two, and three spa-

Fig. 6: Pressureisosurfaces of a flowcomputed by PARMOON

tial dimensions, including conforming, non-conforming, discontinuous, higher-order, vector-valued,and isoparametric ones as well as finite elements with bubbles. A number of time stepping meth-ods, such as θ -, diagonally implicit Runge–Kutta, and Rosenbrock–Wanner schemes, can beemployed. A wide variety of spatial discretizations is available, especially many stabilizations forconvection-dominated convection-diffusion-reaction equations and for finite element pairs for theNavier–Stokes equations that are not inf–sup stable. Furthermore, turbulence models can be

Annual Research Report 2017 (Draftcopy of March 1, 2018, 14:10)

4.3 RG 3 Numerical Mathematics and Scientific Computing 57

used. PARMOON has interfaces to external libraries to solve the resulting linear systems of equa-

Fig. 7: Parallel efficiency forthe heat equation with135,005,697 degrees offreedom

tions. These include direct solvers (UMFPACK, PARDISO, MUMPS) as well as many iterative ones(through the portable, extensible toolkit for scientific computation PETSc). Additionally, PARMOONhas built in a fully parallelized geometric multigrid solver/preconditioner. It outperformed externalsolvers in the context of incompressible Navier–Stokes equations by up to 24 processes; see [6].And a very good speedup of the multigrid preconditioner for the heat equation was obtained of upto 960 processes; see Figure 7 and [3].

The code is continually extended and revised to address both new software and architecturaldevelopments (for examples concerning the build system, compilers, and debugging tools), aswell as new discretizations and stabilizations. There are, e.g., several solvers/preconditioners for

104 105 106 107

# dof

101

102

103

104

computing tim

e in sec.

Q2/Pdisc1

UMFPACK, slope 2.63FGMRES + MG(cell), slope 0.77FGMRES + MDML(cell), slope 0.89FGMRES + boundary-corr. LSC(dir), slope 2.03

Fig. 8: Different solvers fora steady-state flow around acylinder in 3D: computingtimes and slope of best-fitline for Q2/Pdisc

1

incompressible flow simulations available in PARMOON that are quite different in nature. A studyassessing a direct solver (UMFPACK) with the FGMRES method preconditioned with a coupledmultigrid scheme or the least-squares commutator (LSC) method was conducted in [1]. None ofthese solvers was superior in any of the considered cases. In fact, the efficiency rather depends onthe pair of inf-sup stable finite element spaces, the fineness of the spatial mesh, and the length ofthe time step. While direct solvers are feasible in two space dimensions for small to medium-sizeproblems, in all other situations, iterative methods are the only option. It furthermore turned outthat for steady-state problems a coupled multigrid preconditioner (using Vanka-type smoothers)was generally the most efficient approach; see Figure 8. The LSC preconditioner, on the otherhand, was fastest whenever time-dependent problems were discretized with small time steps.

References

[1] N. AHMED, C. BARTSCH, V. JOHN, U. WILBRANDT, An assessment of some solvers for saddlepoint problems emerging from the incompressible Navier–Stokes equations, Comput. Meth-ods Appl. Mech. Eng., 331 (2018), pp. 492–513.

[2] F. DASSI, L. KAMENSKI, P. FARRELL, H. SI, Tetrahedral mesh improvement using movingmesh smoothing, lazy searching flips, and RBF surface reconstruction, Computer-Aided De-sign, published online on 13.12.2017, DOI: 10.1016/j.cad.2017.11.010.

[3] S. GANESAN, V. JOHN, G. MATTHIES, R. MEESALA, A. SHAMIM, U. WILBRANDT, An objectoriented parallel finite element scheme for computation of PDEs: Design and implementation,in: 2016 IEEE 23rd International Conference on High Performance Computing Workshops,IEEE, 2016, pp. 106–115. DOI: 10.1109/HiPCW.2016.19.

[4] H. SI, TetGen, a Delaunay-based quality tetrahedral mesh generator, ACM Trans. Math. Softw.,41:2 (2015), pp. 11/1–11/36.

[5] V. WIEDMEYER, F. ANKER, C. BARTSCH, A. VOIGT, V. JOHN, K. SUNDMACHER, Continuouscrystallization in a helically coiled flow tube: Analysis of flow field, residence time behavior,and crystal growth, Ind. Eng. Chem. Res., 56:13 (2017), pp. 3699–3712.

[6] U. WILBRANDT, C. BARTSCH, N. AHMED, N. ALIA, F. ANKER, L. BLANK, A. CAIAZZO, S.GANESAN, S. GIERE, G. MATTHIES, R. MEESALA, A. SHAMIM, J. VENKATESAN, V. JOHN,ParMooN—A modernized program package based on mapped finite elements, Comput. Math.Appl., 74:1 (2017), pp. 74–88.

Annual Research Report 2017 (Draftcopy of March 1, 2018, 14:10)