introducing madagascarreproducibility.org/wikilocal/docs/vienna.pdf · 2008. 10. 9. · introducing...

Introducing MadagascarA Computational Platform for Geophysical Data Processing and

Reproducible Numerical Experiments

Sergey Fomel1 Paul Sava2 Felix Herrmann3

1Bureau of Economic GeologyJackson School of Geosciences

University of Texas at Austin

2Colorado School of Mines3University of British Columbia

June 11, 2006

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 1 / 20

From RSF to Madagascar

RSERSF

RSG

RSF

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 2 / 20

Outline

1 New

2 Test Driven Development

3 Open Source

4 Universal File Format

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 3 / 20

Outline

1 New

2 Test Driven Development

3 Open Source

4 Universal File Format

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 3 / 20

Outline

1 New

2 Test Driven Development

3 Open Source

4 Universal File Format

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 3 / 20

Outline

1 New

2 Test Driven Development

3 Open Source

4 Universal File Format

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 3 / 20

Outline

1 New

2 Test Driven Development

3 Open Source

4 Universal File Format

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 4 / 20

History

Started in June 2003

6 main contributors

Version 0.9 released June 2006Development version

2,000 revisions300 main programs (140,000 lines of C)3,000 tests (40,000 lines of Python)30 papers (20,000 lines of LATEX)

http://rsf.sf.net/

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 5 / 20

Heritage

SEPlibRob Clayton, Jon Claerbout, Dave Hale, Stew Levin, RickOttolini, Joe Dellinger, Steve Cole, Dave Nichols, MartinKarrenbach, Biondo Biondi, Bob Clapp

SEP reproducible research systemMartin Karrenbach, Matthias Schwab, Joel Schroeder,Sergey Fomel, Bob Clapp

Seismic Unix

DDS

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 6 / 20

Outline

1 New

2 Test Driven Development

3 Open Source

4 Universal File Format

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 7 / 20

Test Driven Development

XP, agile software engineering, refactoringTests drive development

Computational geophysicsTests are numerical experiments

ScienceReproducible experiments mean progress

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 8 / 20

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 8 / 20

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 8 / 20

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 8 / 20

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 8 / 20

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 8 / 20

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 8 / 20

Inspirational Quotes

Abandoning the habit of secrecy in favor of processtransparency and peer review was the crucial step bywhich alchemy became chemistry. In the same way, itis beginning to appear that open-source developmentmay signal the long-awaited maturation of softwaredevelopment as a discipline.Eric S. Raymond, The art of UNIX programming, 2004

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 9 / 20

Inspirational Quotes

The purpose of reproducible research is to facilitatesomeone going a step further by changing something.The first step that someone will want to make is to besure that your work is reproducible before they changeand improve upon it.Jon F. Claerbout, Reproducible research

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 10 / 20

Inspirational Quotes

An article about computational science in a scientificpublication is not the scholarship itself, it is merelyadvertising of the scholarship. The actual scholarshipis the complete software development environment andthe complete set of instructions which generated thefigures.Jon B. Buckheit and David L. Donoho, WaveLab andreproducible research, 1995

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 11 / 20

Inspirational Quotes

Within the world of science, computation is now rightlyseen as a third vertex of a triangle complementingexperiment and theory. However, as it is now oftenpracticed, one can make a good case that computing isthe last refuge of the scientific scoundrel... Where elsein science can one get away with publishingobservations that are claimed to prove a theory orillustrate the success of a technique without having togive a careful description of the methods used, insufficient detail that others can attempt to repeat theexperiment?Randall J. LeVeque, Wave propagation software,computational science, and reproducible research,2006

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 12 / 20

Reproducible Research Tools

SEPGNU make rulesPerl scriptsShell scriptsLATEX macros

MadagascarSCons (Python replacement for make)

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 13 / 20

Outline

1 New

2 Test Driven Development

3 Open Source

4 Universal File Format

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 14 / 20

Open Source

Open source means freedom to the userFreedom to useFreedom to study and modifyFreedom to redistributeFreedom to improve

Open source means collaboration and peer reviewMadagascar package is 100% open-source

GPL licenseHosted by SourceforgeDeveloped with Subversion

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 15 / 20

Outline

1 New

2 Test Driven Development

3 Open Source

4 Universal File Format

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 16 / 20

Universal File Format

Borrowed from classic SEPlib

Binary data and test headers

N-dimensional hypercubes

Can represent seismic data (regular, irregular, 3-D, 9-D,etc.) as well as any other kind of data

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 17 / 20

Inspirational Quotes

To design a perfect anti-Unix, make all file formatsbinary and opaque, and require heavyweight tools toread and edit them....If you feel an urge to design a complex binary fileformat, or a complex binary application protocol, it isgenerally wise to lie down until the feeling passes.Eric S. Raymond, The art of UNIX programming, 2004

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 18 / 20

Summary

Madagascar is a software package for geophysical dataprocessing and reproducible numerical experiments.

New, test-driven, open-source, using a universal file format

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 19 / 20

Please Visit

http://rsf.sf.netSchool and workshop

Reproducible Research in Computational GeophysicsAugust 30-31, 2006Vancouver BC, Canada

S. Fomel (UT Austin) Introducing Madagascar EAGE Open-Source Software 20 / 20