Numerical simulation of solute transport

in heterogeneous porous media

A. Beaudoin, J.-R. de Dreuzy, J. ErhelA. Beaudoin, J.-R. de Dreuzy, J. Erhel Workshop High Performance Computing at LAMSINENIT-LAMSIN, Tunisia, November 27 - December 1st, 2006

2D Heterogeneous 2D Heterogeneous permeability fieldpermeability fieldStochastic model Y = ln(K)Stochastic model Y = ln(K)with correlation functionwith correlation function

2( ) expY YY

C

rr

31 Y

Physical modelPhysical model

Flow modelFlow model

v = - K*grad (h)

div (v) = 0

Fix

ed

head

Fix

ed

head

Nul flux

Nul flux

Steady-state caseDarcy equationMass conservation equationBoundary conditions

Transport modelTransport modelF

ixed

head

an

d C

=0

Fix

ed

head

an

d d

C/d

n=

0

Nul flux and C=0

Nul flux and C=0

Advection-dispersion equationBoundary conditionsInitial condition

dC/dt + div(v C - d gradC) = f

injection

Numerical flow Numerical flow simultionssimultions

Finite Volume Method with a regular mesh ; N =Nx Ny cells

Large sparse structured matrix A of order N with 5 entries per row

Linear system Ax=b

Numerical transport simulationNumerical transport simulation

Particle tracker

rZdtddttMvtMdttM 2

Many independent particlesBilinear interpolation for v

injection

d

l

vvvv

ldt m

yyxx

m

2,

,,,maxmin

2

Examples of simulations with Examples of simulations with σσ=2=2

Pe=∞ Pe=10

Sparse direct solverSparse direct solver

memory size and CPU time with memory size and CPU time with PSPASESPSPASES

Theory : NZ(L) = O(N logN) Theory : Time = O(N1.5)

variance = 1, number of processors = 2

Multigrid sparse solverMultigrid sparse solver

CPU time with HYPRE/AMGCPU time with HYPRE/AMG

variance = 1, number of processors = 4residual=10-8

Linear complexity of BoomerAMG

Transport with particle trackerTransport with particle tracker

CPU timeCPU time

Linear complexity of particle tracker

variance = 1, number of processors = 4

Sparse linear solversSparse linear solvers

Impact of permeability varianceImpact of permeability variance

matrix order N = 106

PSPASES and BoomerAMG independent of varianceBoomerAMG faster than PSPASES with 4 processors

matrix order N = 16 106

Particle trackerParticle tracker

Impact of permeability variance and Impact of permeability variance and correlation lengthcorrelation length

number of particles injected = 1000, Peclet number = number of processors P = 64 and matrix order N = 134.22 106

Transport CPU time increases with varianceTransport CPU time slightly sensitive to correlation length

Particle trackerParticle tracker

Impact of Peclet number and correlation Impact of Peclet number and correlation lengthlength

number of particles injected = 2000, variance = 9.0,number of processors P = 64 and matrix order N = 134.22 106

Transport CPU time increases for small Peclet numbersTransport CPU time slightly sensitive to correlation length

Parallel architectureParallel architecturedistributed memorydistributed memory

2 nodes of 32 bi – processors 2 nodes of 32 bi – processors (Proc AMD Opteron 2Ghz with 2Go (Proc AMD Opteron 2Ghz with 2Go

of RAM)of RAM)

Parallel architectureParallel architecture

Parallel algorithms and Data Parallel algorithms and Data distributiondistribution

Domain decomposition into slicesGhost cells at the boundaries

Parallel matrix generation using FFTWParallel sparse solverParallel particle tracker

Parallel algorithms and Data Parallel algorithms and Data distributiondistribution

Direct and multigrid solversDirect and multigrid solvers

Parallel CPU timeParallel CPU time

variance = 9

matrix order N = 106 matrix order N = 4 106

Direct and multigrid solversDirect and multigrid solvers

Speed-upSpeed-up

matrix order N = 106 matrix order N = 4 106

Particle trackerParticle tracker

Parallel CPU timeParallel CPU time

Flow and transport computationsFlow and transport computations

SummarySummary

• PSPASES is efficient for small matrices• HYPRE-AMG and PSPASES are not sensitive to the variance • HYPRE-AMG is efficient for large matrices• HYPRE-AMG and PSPASES are scalable

• Particle tracker is sensitive to Peclet number • Particle tracker is efficient

• transport requires less CPU time than flow for large matrices