lattice boltzmann methods for nano technology, material sciences, and biomedical ... ·...

1

Lattice Boltzmann Methods forLattice Boltzmann Methods forNano Technology, Material Sciences, and Nano Technology, Material Sciences, and

Biomedical ApplicationsBiomedical Applications

Lehrstuhl für Systemsimulation

Regionales Rechenzentrum

Lehrstuhl für Technologie der Metalle

Lehrstuhl für Feststoff- und Grenzflächenverfahrenstechnik

Abt. Neuroradiologie Universitätsklinik Erlangen

Universität Erlangen-NürnbergUniversität Erlangen-Nürnberg

www10.informatik.uni-erlangen.de

Second Workshop

Perspectives of High End ComputingPerspectives of High End Computing

Erlangen, 17. März 2006

C. Feichtinger, J. Götz, K. Iglberger, C. Körner,

T. Pohl, U. Rüde, N. Thürey

2

OverviewOverview

The Lattice Boltzmann Method

LBM applicationsMaterial science and process technology: Metal Foams

Nano Particle Technology

Biomedical Technology: Simulation of Aneurysms

High Performance Computing

Conclusions

3

Part I

The Lattice Boltzmann Method The Lattice Boltzmann Method

4

The Lattice-Boltzmann MethodThe Lattice-Boltzmann Method

Real valued representation of particles

Discrete velocities and positions

Algorithm consists of two steps:Stream

Collide

5

Part II: LBM applications

Metal Foams Metal Foams

In collaboration with theIn collaboration with theInstitut für Werkstoffwissenschaften Institut für Werkstoffwissenschaften

Lehrstuhl Werkstoffkunde und Technologie der Metalle Lehrstuhl Werkstoffkunde und Technologie der Metalle WTM (R.F. Singer, C. Körner)WTM (R.F. Singer, C. Körner)

6

Towards Simulating Metal FoamsTowards Simulating Metal Foams

Bubble growth, Bubble growth, coalescence, collapse, coalescence, collapse, drainage,drainage, rheology, etc. are rheology, etc. are still poorly understoodstill poorly understood

• Simulation as a tool to Simulation as a tool to better understand, control better understand, control and optimize the processand optimize the process

7

Free surfaces with LBMFree surfaces with LBM

Metal Foams – huge gas volumes

Only simulate and track fluid motion

Compute boundary conditions at free surface

8

True Foams with Disjoining PressureTrue Foams with Disjoining Pressure

Zur Anzeige wird der QuickTime Dekompressor Cinepak

bentigt.

Zur Anzeige wird der QuickTime Dekompressor

bentigt.

9


Nano-particle Technology:Nano-particle Technology:Interacting Particles in a FluidInteracting Particles in a Fluid

Cooperation withCooperation withProf. Peukert, Dr. H.-J. SchmidProf. Peukert, Dr. H.-J. Schmid

((Particle TechnologyParticle Technology))

10

Nano-particle TechnologyNano-particle Technology

Properties of materials and products determined by structure of the nano-scale particles

Possible applications of the LBM:

Simulate the behavior of particles and particle agglomerates in solutions (e.g. breaking up or further agglomeration)

On a larger scale simulate segregation / sedimentation processes

11


Moving, curved boundary treatment

Fluid-structure interaction

Coupling to a rigid body physics engine for arbitrarily shaped nano-particles

Moving nano-particles

Extensions of the LBM:

Studienarbeit C. Feichtinger, Master thesis K. Iglberger

12


Extension of the LBM model by an electrostatic potential

Simulating charged nano-particles AND ions in the fluid

Solving the potential distribution with ParExPDE

Diplomarbeit C. Feichtinger

13

Nano-particle TechnoogyNano-particle Technoogy

Zur Anzeige wird der QuickTime Dekompressor YUV420 codec

bentigt.

Moving particle agglomerate in the flow

14


Biomedical Technology: Biomedical Technology:Simulation of AneurysmsSimulation of Aneurysms

In collaboration with theIn collaboration with theAbt. Neuroradiologie Universitätsklinik ErlangenAbt. Neuroradiologie Universitätsklinik Erlangen

Prof. Dr. med. Arnd DörflerProf. Dr. med. Arnd Dörfler

15

Aim of the project:

haemodynamical evaluation of stents at bifurcations

physically correct blood flow simulation

(near) real time simulation

Premliminary work for the numerical haemodynamic simulation:

first results for the flow simulation in blood vessels (current master thesis J.Götz)

resolution DAS and simulation of

256 x 256 x 400

simulation time of approx. 80 minutes

Modell-Datensatz

Strömungsvisualisierung

Stromlinien

Farbe = Geschwindigkeit

Simulation of AneurysmsSimulation of Aneurysms

16

Simulation of AneurysmsSimulation of AneurysmsPulsating blood flow at aneurysm (CE master thesis: J. Götz)


bentigt.

Blood flow through a blood vessel bottleneck (aneurysm)

17

Pulsating blood flow at aneurysm (CE master thesis: J. Götz)


bentigt.


Fluid-Structure Interaction: blood pressure against the blood vessel

18



bentigt.

Fluid-Structure Interaction: blood pressure against the blood vessel

19

Part III

High Performance ComputingHigh Performance Computing

20

Standard LBM Code: Scalability

Largest Simulation: 1,08*109 cells 370 GByte memory

Communication Cost because of large data volume (64 MByte)

Efficiency ~ 75%

Dissertation T. Pohl (2006)

Parallelization of Standard LBM CodeParallelization of Standard LBM Code

21

Speed-up performance on theSpeed-up performance on theLSS Opteron clusterLSS Opteron cluster

22

Nano-particle TechnologyNano-particle TechnologyFinal goal: complete sedimentation process

Assumption: diameter of one spherical body = 8 lattice cells

Assumption: diameter of one nano-particle ≈ 80 lattice cells

803 = 5.12 x 105 lattice cells

Sedimentation of 1000 nano-particles 5.12 x 108 lattice cells

Memory requirement of 77.8 Gbyte (only for the nano-particles and

using memory reduction techniques)

Super computer is obligatory

23

Parallelization strategy:Domain decomposition of the DAS data

Estimation of the relevant data regions, which contain fluid

Memory allocation only for a „fluid region“

Connecting the regions

Implicit approach towards parallelization

Blood Flow Blood Flow ParallelisationParallelisation

24

Blood Flow ParallelisationBlood Flow Parallelisation

Simulation of real blood flow with blood corpuscles:Assumptions: blood vessel of 0.01 cm2 x 1 cm 10-4 liter

Fact: 3.2 - 5.9 x 1012 red blood corpuscles per liter

Assumption: human with 4 x 1012 red blood corpuscles per liter

Simulation of 4 x 108 red blood corpuscles in a blood vessel

Diameter of red blood corpuscles: ~ 8 µm

Assumption: Simulation diameter = 8 lattice cells 83 = 512 lattice cells only for one corpuscle

2 x 1011 lattice cells (only for the corpuscles) 31 Tbyte

Direct combination with the nano-particle simulationsFluid structure interaction

25

ConclusionConclusion

The LBM is very interesting for high performance computing

Challenging applications from different sciences

Real scenarios make a super computer obligatory

26

AcknowledgementsAcknowledgementsCollaborators

In Erlangen: WTM, LSE, LSTM, LGDV, RRZE, etc.Especially for foams: C. Körner (WTM)International: Utah, Technion, Constanta, Ghent, Boulder, ...

Dissertationen ProjectsU. Fabricius (AMG-Verfahren and SW-Engineering for parallelization)C. Freundl (Parelle Expression Templates for PDE-solver)J. Härtlein (Expression Templates for FE-Applications)N. Thürey (LBM, free surfaces)T. Pohl (Parallel LBM)... and 6 more

16 Diplom- /Master- ThesisStudien- /Bachelor- Thesis

Especially for Performance-Analysis/ Optimization for LBM• J. Wilke, K. Iglberger, S. Donath

... and 21 more

KONWIHR, DFG, NATO, BMBFKONWIHR, DFG, NATO, BMBFElitenetzwerk BayernElitenetzwerk Bayern

Bavarian Graduate School in Computational EngineeringBavarian Graduate School in Computational Engineering (with TUM, since 2004) (with TUM, since 2004)

Special International PhD program: Special International PhD program: Identifikation, Optimierung und Steuerung für technische Identifikation, Optimierung und Steuerung für technische AnwendungenAnwendungen (with Bayreuth and Würzburg) to start Jan. 2006. (with Bayreuth and Würzburg) to start Jan. 2006.

27

Thanks for your attention Zur Anzeige wird der QuickTime

Dekompressor bentigt.

28

The Stream StepThe Stream Step

Move particle distribution functions along corresponding velocity vector

Normalized time step, cell size and particle speed

29

The Collide StepThe Collide Step

Amounts for collisions of particles during movement

Weigh equilibrium velocities and velocities from streaming depending on fluid viscosity

30

GlassCeramics

MetalsPolymers

Structural Properties stiffness

energy absorption damping

Functional Properties burner, shock absorber,

heat exchanger, batteries

large, dynamic surface expansion

Examples of FoamsExamples of Foams

31

Rising BubblesRising Bubbles


bentigt.

32

More Rising BubblesMore Rising Bubbles


bentigt.

33

Free surface flow: Breaking DamFree surface flow: Breaking Dam


bentigt.

34

Simulation VerificationSimulation Verificationby Experimentby Experiment

Simulation and Experiment: Simulation and Experiment: Diplomarbeit Diplomarbeit N. ThüreyN. Thürey


bentigt.

35

Boundary ConditionsBoundary Conditions

Gas

Liquid

Problem: Missing distribution functions at interface cells after streaming!

Reconstruction such that macroscopic boundary conditions are satisfied.

Körner et al. Lattice Boltzmann Model for Free Surface Flow, to be published in Journal of Computational Physics

36

Curvature calculation (version I)Curvature calculation (version I)

Alternative approaches:

Integrate normals over surface (weighted triangles)

Level set methods (track surface as implicit function)

37

20

23

25

28

30

33

35

38

40

43

45

48

50

53

55

58

60

63

65

68

70

73

75

78

80

0,000

0,001

0,002

0,003

0,004

0,005

0,006

0,007

Spalte F

Distance in l.u.

Velo

cit

y

Stokes´ Law: Climbing rate of a bubble exposed to gravity

Climb rate

Ideal bubble No boundaries Equilibrium state

R = 8, τ = 0.74, g = 10-4, σ = 2*10-2100 x 100 x 140 cellsExample:

Rel. error: 2 %

Error = function of the system size

Verification for bubble dynamicsVerification for bubble dynamics(C. Körner)(C. Körner)

Zur Anzeige wird der QuickTime Dekompressor Cinepak

bentigt.

38

VisualizationVisualization

Ray-tracingRefractionReflectionCausticsAbout 15 Min per frame

= 1 day for 4 secsAbout same compute time as flow simulation

39

Nano TechnologyNano Technology

Curved Boundaries:Particles approximated with spheresImprove accuracy of LBM simulations by using curved boundary conditions

Standard No-SlipReflect DFs at cell boundary

More accurate:Take distance to boundary surface into account, then interpolate DFs accordingly

40

Nanotechnology ApplicationsNanotechnology ApplicationsMoving particle agglomerate in the flow

K. Iglberger - Master Thesis Project


bentigt.


bentigt.

41

Pulsating Blood Flow at Aneurysm

CE Master Thesis: Jan Götz

Data Set


bentigt.

42


Based on cellular automataIntroduced by von Neumann around 1940

Famous: Conway’s Game of Life

Complex system with simple rulesRegular grid

Local rules specifying time evolution

Intrinsically parallel for model & simulation, similar to elliptic PDE solvers

43


Weakly compressible approximation of the Navier-Stokes equations

Easy implementation

Applicable for small Mach numbers (< 0.1)

Easy to adapt, e.g. forComplicated or time-varying geometries

Free surfaces

Additional physical and chemical effects

44

LBM DemonstrationLBM Demonstration(Java applet)

file:///Users/ruede/doc/lehr/vorles/ws03/hppt/lbm/jlb-comp/start.html

45

Free surface implementationFree surface implementation

Before stream step, compute mass exchange across cell boundaries for interface cells

Calculate bubble volumes and pressure

Surface curvature for surface tension

Change topology if interface cells become full or empty – keep layer of interface cells closed

46

Surface Tension (Vers. 2)Surface Tension (Vers. 2)

Vδ

ΑΑΑ −=δ

Α

Α

1ν_3n

_

2n_

Marching-cube surface triangulationCompute a curvature for each triangle

Associate with each LBM cell the average curvature of its triangles

Complicated Beats level sets for our applications (mass conservation).

k= 12dAdV

•

47

V10.000.000.000

1.000.000.000

100.000.000

10.000.000

1.000.000

100.000

1.000

10.000

100

4004

80368

Pentium

Merced

1K

64K

256K

1M4M

64M

1G

4G

1970 1975 1980 1985 1990 1995 2000 2005

Year

Tra

ns

isto

rs/D

ie

Microprocessor(Intel)

DRAM

Growth:42% per year

Growth:52% per year

Moore's Law in Semiconductor Technology(F. Hossfeld)

80468

Pentium Pro

48

1021

1018

1015

1012

109

1024

103

106

1

1950 1960 1970 1980 1990 2000 2010 2020

Year

Ato

ms/

Bit

Information Density & Energy Dissipation(adapted by F. Hossfeld from C. P. Williams et al., 1998)

10 -9

10 -6

10 -3

1

10 3

10 6

10 9

1012

1015

Ene

rgy/

log

ic O

per

atio

n [

pic

o-J

oule

s]

kT

Semiconductor Technology

≈ 2017

49

ParallelisationParallelisation

Free surface LBM-Code

Standard LBM Free surface LBM

1 sweep through grid 5 sweeps through grid

Cell type changes, Closed boundary for bubbles, Initialization of modified cells, Mass balance correction

50

Parallelization of Standard LBM CodeParallelization of Standard LBM CodeStandard LBM-Code in C (1-D Partitioning):

- excellent performance on single SR8000 node- almost linear speed-up- large partitions favorable

Performance on SR8000

Ca. 30% of Peak Performance

51

Free-Surface ParallelisationFree-Surface Parallelisation

Standard LBM Free surface LBM

1 sweep through grid 5 sweeps through grid

1 row of ghost nodes 4 rows of ghost nodes

52

PerformancePerformanceFree surface LBM-Code Free surface LBM-Code

Standard LBM-CodeStandard LBM-Code

Performance lousy on a single node! Conditionals: 2,9 SLBM 51 free surface LBMPentium 4: almost no degradation ~ 10%SR 8000: enormous degradation (pseudo-vector, predictable jumps)

53

Part VI

Outlook: Other applicationsOutlook: Other applications

3D-AnimationComputational SteeringReal-Time Simulation

54

Near-Real-Time Free-Surface LBM Near-Real-Time Free-Surface LBM (N. Thürey)(N. Thürey)


bentigt.

55

Free-Surface LBM with Adaptive Refinement Free-Surface LBM with Adaptive Refinement (N. Thürey)(N. Thürey)


bentigt.

Hochaufgelöste AnimationenAdaptive Verfeinerung/ VergröberungVisualisierung mit RaytracerFluid-Simulation in Blender 2.4 (22.12.05)Blender: 3D-ModellierungsprogrammFrei verfügbar:http://www.blender3d.org/

56

Aus dem Bericht an die National Science Foundation der USA

Simulation Based Engineering Science(http://www.ices.utexas.edu/events/SBES_Final_Report.pdf)

Meaningful advances in SBES will require dramatic changes in science and engineering education. Interdisciplinary education in computational science and computing technology must be greatly improved. Interdisciplinary programs in computational science must be encouraged, and the traditional boundaries between disciplines in higher education must be made pervious to the exchange of information between discipline scientists working

within multidisciplinary research teams.

The Erlangen CE program does exactly this since 1997!

lattice boltzmann methods for nano technology, material sciences, and biomedical ... ·...

Documents