linc: masterclass “scientific software engineering” · linc: masterclass “scientific software...

LINC: Masterclass “Scientific software engineering”

“Mastering the complexity of numerical simulation software”

Presenters: ir. J.C.M. Dijkzeul and dr.ir. A.J. VrielingLocation: LINC First School, MallorcaDate: September 11-12, 2012

Profile:• Office in Delft, NL

• Founded in 1996, now with 16 employees and growing

• Highly educated employees

• Profound knowledge of mathematics AND software development

• Technical/scientific software for simulation

September, 2012LINC: Masterclass “Scientific software engineering” 2

Introduction VORtech

VORtech

3

About us

Product and services• mathematical modeling• software development

• software maintenance• masterclass

LINC First School, Mallorca September 2012

Expertise:• complex algorithms

• model coupling• data-assimilation• HPC: speed-up/parallellisation• domain knowledge• software engineering

Simulation software

4

Example

• SIMONA: since ca. 1980 for Dutch ministry of transport and public works• operational prediction system for water levels, flows, stormsurges• mathematical physics, parallel processing, data-assimilation• more than 1 Mloc, in Fortran, C, Perl, Java, Matlab, …

LINC First School, Mallorca September 2012

Simulation software

5

More examples

• CONTACT: contact mechanics (Fortran, Java)• OpenDA: data-assimilation and model calibration (Java, C)• DaVinci: analysis of trafic data (Visual Basic, Matlab)• FEWS: prediction/warning system water/weather (Java)

• IMAGE: global change model PBL (Fortran)

LINC First School, Mallorca September 2012

Challenge for R&D companies:•How to build large computational software?•And maintain it over time?

September, 2012LINC: Masterclass “Scientific software engineering” 6

Introduction Masterclass “Scientific software engineering”

September, 2012LINC: Masterclass “Scientific software engineering” 7

Challenge for universities

Questions: • How can PhD students avoid software pitfalls?• Put more focus on their actual research?• And how do new PhD students

build upon previous works?

Solution: • Create better software

Good scientific software spans multiple disciplines• Knowledge of application area• (Mathematical) modeling• Numerical analysis, High Performance

Computing• Software development, programming

September, 2012LINC: Masterclass “Scientific software engineering” 8

Multi-disciplinarity

YES and NO!

Helpful, but not entirely fit.

September, 2012LINC: Masterclass “Scientific software engineering” 9

Standard Software Engineering Methodology?

Evolution:1. It starts with a small program or software script

2. Software is extended to fit more needs

3. Software is reused in other project(s)

4. Software is incorporated into an operational process

5. Software is consolidated, maintenance is arranged

September, 2012LINC: Masterclass “Scientific software engineering” 10

Good scientific software evolves

Fred Brooks (IBM, The mythical man-month, 1975):• Good software development is hard work

• There's no easy way out

• Provide a collection of best practices

September, 2012LINC: Masterclass “Scientific software engineering” 11

How to combat “software complexity”

Present software development techniques

that work for scientific computing software

Present software development techniques

that work for scientific computing software

September, 2012LINC: Masterclass “Scientific software engineering” 12

Aim of this masterclass

September, 2012LINC: Masterclass “Scientific software engineering” 13

Origins of this masterclass

This specialism is recognized more and more at VORtech• Contributions to software development at Shell

• Assistance of PhD students in our work at TU Delft

• Model scan and assessment of software quality for PBL

• Development of a Top 10 of best practices

• Experiences in training well-skilled employees

Parallelization = 20% algorithms + 80% “bookkeeping”

September, 2012LINC: Masterclass “Scientific software engineering” 14

It started with parallelization of software...

Bottlenecks: • Large code (>100K lines)

• many users & developers

• Worked together on large software projects

• Multiple versions, models, integration, coupling

• Dealt with essential complexity

September, 2012LINC: Masterclass “Scientific software engineering” 15

Domain (de)composition, Kalman filtering

1. Basics of software engineering“Lay the foundation before building a house”

2. What and how to automate“Everything that is worth repeating, should be automated”

3. Testing & Numerical aspects“There are so many numbers and they're never the same”

4. Importance of clarity“It took longer, because I tried to do it quickly”

5. Software development process“To prevent software decay, long-lived software must be nurtured”

6. Summary and discussion

September, 2012LINC: Masterclass “Scientific software engineering” 16

Masterclass contents