debichem - packaging free software chemistry programs in ...mbanck/talks/debichem-debconf13.pdf ·...

DebichemPackaging Free Software Chemistry Programs in Debian

GNU/Linux: Past, Present and Future

Michael Banck

[email protected]

August 12th, 2013

Michael Banck ([email protected]) Debichem August 12th, 2013 1 / 26

Introduction

Chemistry is...I Between Atoms (physics team) and Solids

(nanoscale-physics team)I Blends into Biology (debian-med team)I On the TEX/Word DivideI Slow Uptake of FLOSS


Debichem

Started in 2004 by Michael BanckMostly a two-man show (Michael Banck and Daniel Leidert)Handful people upload specific packages20 project membersOver 50 packages


Mailing List Statistics

http://blends.debian.net/liststats/authorstat_debichem-devel.png



Commit Statistics




Original Mission

The Mission: Package all chemistry-related Free Software projects

In 2000, there was Chemtool, MPQC and PSI3, basicallyToday, it seems like every Ph.D. student is putting their code onSourceForge or GithubOr unmaintainable Code gets dumped onto the FLOSS communityPROSPECTIVE PACKAGES:

MOLDY, XtalOpt, MOIL-opt, DL POLY CLASSIC, AmberTools, OSRA, Coot, MDAnalysis,

OpenMM, PaDEL-Descriptor, MATCH, DynamO, DockoMatic, Pteros, Aten, CCP1GUI,

Zeobuilder, Molsketch, SCHAKAL, ccp4-suite, ezDICOM, InChI, mychem, PDB Validation Suite,

KryoMol, MayaChemTools, Frowns, OpenMCTDHB, MrPropa, MoViPAC, CMSapi, CMPTool,

BerkeleyGW, Socorro, Spglib, EPW, wannier90, Exciting, ccwatcher, MSL, Towhee, ORAC,

MESMER, MultiWell, ELPA, TAO, ERKALE, QMCPACK, QWalk, QMcBeaver


Current Mission

The Mission: Package the most appropriate chemistry-related FreeSoftware projects

Identify the best packages for a particular taskRemove no longer maintained or superseded packagesTarget: Workstation-based or small compute cluster


Fields of Interest

And corresponding tasks

Visualization of Molecular/Electronic Structure:debichem-visualization

Chemical Structure Drawing: debichem-view-edit-2dCheminformatics: debichem-cheminformaticsMolecular Modelling: debichem-modellingComputational Chemistry: debichem-abinitio,debichem-semiempirical, debichem-molmech

Mass Spectrometry: debichem-polymer


Computational ChemistryElectronic Structure (Gas Phase / Condensed Phase)

I NWCHEM (PNNL)I CP2K (ETH Zurich, University Zurich)I MPQC (Virginia Tech)I PSI (Georgia Tech, Virginia Tech)I ACES (University Florida)I QUANTUM ESPRESSO (Italy)

Semi-EmpiricalI CP2K (ETH Zurich, University Zurich)I MOPAC7 (Unmaintained)I MOLDS (Japan)

Molecular Dynamics (Classical / Ab Initio)I GROMACS (EU)I CP2KI NWCHEMI MOLDS (Japan)


Molecular Modelling

Avogadro


Molecular ModellingPyMOL


Cheminformatics

OpenBabelRDKitCDKIndigoCinfonyChemFP


Success Stories: CP2K

J. VandeVondele, U. Borstnik and J. Hutter, J. Chem. Theo. Comput., 8, 3565 (2012)


Success Stories: PSI4

E. Hohenstein, R. Parrish, D. Sherrill, J. Turney and H. F. Schaefer III, J. Chem. Phys., 135,

174107 (2011)Michael Banck ([email protected]) Debichem August 12th, 2013 14 / 26

Success Stories: PYMOL

http://pymolwiki.org/index.php/Covers


http://pymolwiki.org/index.php/Covers

Success Stories: ACESIII

V. Lotrich, J. Ponton, A. Perera, E., R. Bartlett and B. Sanders, Mol. Phys., 108, 3323 (2010)


Package Interdependencies

Quantum-Chemistry Specific LibrariesI LIBINT (MPQC2, PSI3, CP2K)I LIBINT2 (MPQC3)I LIBXC (ELK, CP2K)I LIBPSI (MPQC3)

Wavefunction UsageI QUANTUM ESPRESSO (BERKELYGW, XTALOPT,...)

Input GenerationI Avogadro (NWCHEM, MOPAC7)

Output Parsing/VisualizationI GABEDIT (MPQC)I AVOGADRO (NWCHEM, MPQC, MOPAC7, GROMACS, QUANTUM

ESPRESSO)


Packaging Principles

Subversion or Git RepositorySource Format 3.0 (quilt)Verbose Long Description

I Main FeaturesI Supported file formats (if applicable)I Supported DFT functionals (if applicable)I Parallization

Parallization via Default MPI ImplementationUpstream Metadata via debian/upstream

I CitationI Principal PublicationsI Upstream Registration SiteI Upstream Sourcecode Repository


Quality Assurance

Peer-Review of PackagingUpstream test suites

I Run a representative but limited set of test casesI Support to bypass test suite (DEB BUILD OPTIONS=nocheck)I Optionally run all test cases (DEB BUILD OPTIONS=fullcheck)I Problem: Smart parsing of tolerances - diff is not enough

WatchfilesInstall-time test suites (TODO)


Frequent Changes to Upstream

Hardcoded Datafile Locations FixesBuild-System FixesManpagesCopyright/License Clearup


Upstream Horror Stories

Downloads of GPL’d tarballs behind registration form... And registratino form forwarding to SF Download pageUsers have to register at Website to loginGit-Repositories behind semi-public password logins


Challenges

Fortran77 CodeHand-made MakefilesNever intended to be installedOnly useful on Supercomputers

I ACESIII did not run sequentially


A Look at Upstream: Computational Chemistry

Gromacs MPQC CP2K NWChem PSI QEMain Developers Various GT UZ/ETH PNNL GT/VT VariousLicense GPL2+ (L)GPL GPL2+ ECL2 GPL2+ GPL2Language C C++ F95 F77 C/C++ F77Project Hosting Redmine SF SF - SF/Trac GForgeRelease Tarballs X X X X X XStable Point Releases X - - - X -Sourcecode Repository git hg svn svn/prv git/prv svnSourcecode Hosting Local SF SF - - GForgeSnapshot Tarballs - - - X - -Bug Tracker X X X - X XDeveloper Mailing List X - - - - XUser Mailing List X - X X - XUser Web Forum - - - X - -Contact Mail Address - - - - - -


A Look at Upstream: Cheminformatics Toolkits

OpenBabel RDKit Indigo CDKMain Developers Community Community GGA CommunityLicense GPL2 BSD GPL3 LGPLLanguage C++ C++ C++ JavaProject Hosting SF SF/GC - SFRelease Tarballs X X X XStable Point Releases X - - XSourcecode Repository git git git gitSourcecode Hosting Github Github Github SFSnapshot Tarballs - - - -Bug Tracker X X X XDeveloper Mailing List X X X XUser Mailing List X X X XUser Web Forum - - - -Contact Mail Address - - - -


Future Work

Sort out TasksPackage Projects with New Functionality

I Quantum Monte-Carlo (QMCPack, QWalk, QMcBeaver)I Quantum Dynamics (MrPropa)I Complete Autodocking Suite (DockoMatic)

Package bundled parallization Libraries for wider AvailabilityI Global Arrays (NWChem, but distributed seperatedly by PNNL)I LibSC (MPQC)I LibSIP/SIAL (ACESIII)

Package bundled matrix computation Libraries for widerAvailability

I LibDBCSR (CP2K)I LibSMM (CP2K)I Various PSI4 Libraries


Future Work (Continued)

Hybrid MPI/OpenMPOpenCL Support (Free Implementation?)Support for Machine-Specific/Optimized Recompilation (cf. Atlas)

I autotune grid (CP2K)

Correctness TestingFortran90 Module FilesMultiple Compiler Support (LLVM, Intel?)Support Input Preparation/Output Visualization

I cclib (Currently only supports proprietary packages)I OpenBabelI Avogadro

Build System ImprovementsPseudo-Potential Handling


debichem - packaging free software chemistry programs in ...mbanck/talks/debichem-debconf13.pdf ·...

Documents