handling data and workflows in computational materials science: the aiida initiative

Andrea Ferretti

Handling data and workflows in computational materials science

The AiiDA initiative

Firenze, 15 Nov 2016

-  Highly accurate ab initio methods in electronic structure

-  Large computational power required (now available)

-  High-throughput screening possible

-  Reduced need for exp dat

COMPUTATIONAL MATERIALS’ SCIENCE

N. Marzari, Nature Materials, Apr 2016 PRL 105, 106601 (2010)


G. Hautier et al, Nat Comm 4, 2292 (2013)

K2Sn2O3 (rhombohedral or bcc) and Rb2Sn2O3 show the lowesteffective mass around 0.27–0.28 but a band gap on the small side(2.4 eV). Interestingly, the band gap of K2Sn2O3 can be increasedby substituting Na that would lead to lower absorption in thevisible (see Supplementary Note 1 and Supplementary Fig. S21).K2Pb2O3 also possesses a higher band gap than K2Sn2O3, withsimilar effective masses but with the drawback of Pb toxicity.On the other hand, the ZrOS and HfOS compounds have largereffective masses but with a significantly larger band gap andopportunities for full visible range transmission. Finally, bothSb4Cl2O5 and B6O have large band gaps (respectively, 3.6 and3 eV) and low effective masses (respectively, 0.37 and 0.59).

We performed our analysis using the electronic band gap, butforbidden optical transitions can make the optical band gapsignificantly larger, as is the case in In2O3 (ref. 27). Thus, wehave also computed all the optical absorption spectra (seeSupplementary Fig. S22). None of the smaller electronic gapmaterials (o3 eV) show an optical gap significantly higher thantheir electronic gap. Hence, the conclusions drawn from theelectronic band gap still hold.

p-type dopability of the most promising candidates. A low holeeffective mass and a large band gap are necessary for any high-performance p-type TCO and it is remarkable that those twosimple constraints already exclude the vast majority of knownoxides (499%). However, the possibility to generate holes in thevalence band (that is, the p-type dopability) is not guaranteed apriori for our candidates. It is indeed well reported that mostoxides have fundamental thermodynamic constraints, makingtheir p-type doping difficult28–30. More specifically, the formationof compensating intrinsic defects (hole killers) such as the oxygenvacancy when lowering the Fermi energy towards the valenceband has been identified as the main impediment to p dopingin oxides. Whereas none of our candidates have ever beentested (or even suggested) as TCOs, doping studies indicatingp-type dopability have already been reported experimentallyor computationally for several of them. B6O has beenexperimentally measured to show p-type conductivity31. It hasbeen demonstrated experimentally that PbZr0.5Ti0.5O3 can be

grown as p-type32, but some recent computations on PbTiO3seem to indicate an oxygen vacancy low in energy even inoxidizing conditions33. Finally, a recent computational study onZrOS defects demonstrated that the oxygen vacancy is not a holekiller in oxidizing conditions (to the contrary of ZrO2)34.

For the remaining chemistries of greatest interest, we performdefect computations (see Methods), focusing on all thevacancy intrinsic defects as in Trimarchi et al.35 Figure 3 shows

Sb4CI2O5

2+

Vo

1+

VCI

Vo

K2Sn2O3

K2Pb2O3

3

4

3

2

Def

ect f

orm

atio

n en

ergy

(eV

)

1

0

–1

–2

–3

–40.5 1 1.5

Fermi energy (eV)

2 2.5

2

1

0

–1

–2

–3

4

a

b

c

3

23–

1–2–

VSn

4–

1–VK

2+1–

2–

Vo

VK1–

4–

VPb

VSb

2+1+

5–

1

0

–1

–2

–3

0 1 2

Fermi energy (eV)

3 4

0.5 1 1.5

Fermi energy (eV)

2

Def

ect f

orm

atio

n en

ergy

(eV

)D

efec

t for

mat

ion

ener

gy (

eV)

Figure 3 | Vacancy formation energy versus Fermi energy. The panelsindicate results for Sb4Cl2O5 (a) K2Sn2O3 (b) and K2Pb2O3 (c). The oxygenvacancy formation energy is indicated by a blue line. The cation vacanciesare indicated by orange and purple lines. All defects are calculated inoxidizing conditions. The zero of Fermi energy is the valence bandmaximum.

1.5 3 3.5 4 4.5 5

0

0.5

1

1.5

2

2.5

3

3.5

ZnO SnO2In2O3

AlCuO2

SrCu2O2

ZnRh2O4

K2Sn2O3

Sb4Cl2O5

K2Pb2O3 PbTiO3

Ca4As2OCa4P2O

Sr4P2OSr4As2O

Hg2SO4

PbZrO3NaNbO2Tl4V2O7

Tl4O3

ZrSO HfSO

B6O

Na2Sn2O3

PbHfO3

Band gap (eV)

Effe

ctiv

e m

ass

Current p-typeTCOs

Current n-type TCOs

2 2.5

Figure 2 | Effective mass versus band gap for the p-type TCO candidates.We superposed on the band gap axis a colour spectrum corresponding tothe wavelength associated with a photon energy. The TCO candidates aremarked with red dots. A few known p-type (blue diamonds) and n-type(green square) TCOs can be compared to the new candidates. The bestTCOs should lie in the lower right corner. For clarity, we kept only onerepresentative when polymorphs existed for a given stoechiometry (forexample, PbTiO3 and K2Sn2O3) and did not plot Rb2Sn2O3, which issuperposed on K2Sn2O3.

NATURE COMMUNICATIONS | DOI: 10.1038/ncomms3292 ARTICLE

NATURE COMMUNICATIONS | 4:2292 | DOI: 10.1038/ncomms3292 | www.nature.com/naturecommunications 3

& 2013 Macmillan Publishers Limited. All rights reserved.

-  Highly accurate ab initio methods in electronic structure

-  Large computational power required (now available)

-  High-throughput screening possible

-  Reduced need for exp data

-  Data handling needed


N. Marzari, Nature Materials, Apr 2016 PRL 105, 106601 (2010)

SOME THOUGHTS ON DATA

•  In computational science, data are naturally generated, so the workflows that create properties and data from a structure are key

•  Curated data are needed (e.g. for verification or for machine learning)

•  A model of data-on-demand can be implemented (high-throughput pushes the development of robust workflows to calculate automatically).

OBJECTIVES

•  Automation: run thousands of calculations daily •  Provenance: all children and all parent data are

recorded •  Reproducibility: go back to a simulation years later,

and redo it with new parameters or codes •  Extensible/agnostic to models, codes and formats •  Workflows: dynamical, robust, complex “turnkey

solutions” that calculate desired properties on demand •  Sharing: provide the distributed environment to

disseminate workflows and data and to provide services

ADES MODEL FOR COMPUTATIONAL SCIENCE

G. Pizzi et al., Comp. Mat. Sci 111, 218-230 (2016)

Low-level pillars User-level pillars

ECOSYSTEM

Automation Data Environment Sharing Automation Database Research environment Social Remote management Provenance Scientific workflows Sharing High-throughput Storage Data analytics Standards

A factory A library A scholar A community

http://www.aiida.net (MIT BSD, jointly developed with Robert Bosch) G. Pizzi et al., Comp. Mat. Sci. 111, 218 (2016)

G. Pizzi, A.C., et al., arXiv:1504.01163

ADES

Automation in AiiDA

Remote management Coupling to data High throughput

Automation in AiiDA

1. The core of the code is the AiiDA API (Application Programming Interface), a set of Python classes that exposes the users to the key objects: Calculations, Codes, and Data.

What is AiiDA?

Automation in AiiDA

2. The AiiDA Object-Relational Mapper (ORM) maps AiiDA objects into Python Classes, so that the objects can be created/modified/queried via an agnostic high-level interface. Any interaction with Storage occurs transparently via Python calls.

Automation in AiiDA

3. A daemon manages calculation states (submission, retrieval, parsing…) without user intervention (uses Python celery+supervisor modules), through remote transports and Slurm/PBS Pro/SGE/Torque plugins.

Automation in AiiDA

4. User interactions occurs via the command line tool Verdi, the interactive shell or via Python scripts

Coupling automation with storage

•  The AiiDA-API acts as the unique interface to heterogeneous, remote HPC resources, that are abstracted away

– All work can be done on the local resources, and the user does not need to connect explicitly to remote HPC

•  Coupling automation with storage ensures: – uniformity of the input data, usage of codes and computers

(the same interface encompasses several supercomputers, different schedulers, connection protocols…

–  full reproducibility and provenance, with automatic storage of all data and links

– seamless sharing of calculations with other users


ADES

Data in AiiDA

Storage Database

Provenance

The Open Provenance Model

•  Any calculation is a function, manipulating an input to obtain an output:

out1, out2 = F(in1, in2)

•  Each functional object is a node in a graph, connected together with directional, labeled links

•  Output nodes in turn can be used as inputs of following calculations out1 out2

in1 in2

F

data

data

data

data

calc

DIRECTED ACYCLIC GRAPHS

Nodes: Calculations Codes Data

Saving the DAGs: Nodes and Links

Nodes and links: a graph structure • Each node: row in a SQL table

+ folder for files •  Links also stored in a SQL table ⇒ jobs provenance

Transitive closure (TC) table • Allows queries that traverse the graph • Automatically updated using triggers • Queries using TC in SQL faster than with

graph DB backends!

Benchmark against Neo4j •  Graph databases exist (Neo4j) •  They are still young, while SQL is very mature •  Our benchmark (with postgreSQL) vs. Neo4j on same realistic

data, ~11K graphs, ~100K nodes, >1M attributes)

AiiDA (query 1 and 2)

Neo4j (query 1)

Neo4j (query 2)

Number of results

Query time (s)

The AiiDA daemon

A daemon runs in the background

Calculation state SUBMITTING

WITHSCHEDULER

RETRIEVING PARSING

FINISHED


ADES

Environment in AiiDA

High-level workspace Scientific workflows

Data analytics

Environment in AiiDA: plugins

All functionality provided using a plugin interface

Calculation Data Parser Transport Scheduler

Generation of input files for a

given code

Quantum Espresso, Phonopy, GPAW, Yambo, NWChem,

…

Management of data objects for

input/output

files&folders, parameter sets,

remote data, structures, pseudos, ...

Parsing of code

output and generation of

new DB nodes

Quantum Espresso, Phonopy, GPAW,

Yambo, NWChem, ...

How to connect

to a cluster

local connection, ssh, ...

How to interact

with the scheduler

PBSPro, Torque, SGE, SLURM, ...

•  Full python scripting capabilities •  AiiDA manages calculation dependency •  They are modular: users can expand on the workflows of others •  A step can call nested subworkflows. •  Develop turn-key solutions for the calculation of material

properties: libraries of workflows

Environment in AiiDA: Workflows

Workflows features

•  Automatic provenance tracking, stored in DB using simple python functions inputs, outputs, function calls stored by adding simple decorator to existing functions

•  Serial and parallel execution support can launch long running tasks on separate threads and wait for result when needed

•  Control provenance granularity store level of detail relevant to the workflows

•  Seamless mixing of local and remote jobs •  Progress checkpointing

restart from arbitrary step, retry on failure

•  Easy debugging execute workflows in IDE and observe/change states of variables as it runs

•  Background execution daemon execution allows machine to be shutdown and continue from last point, essential for running long remote jobs

WORKFLOWS ENCODING CORE KNOWLEDGE

CHRONOS workflow: electronic-magnetic-atomic structure

PHONON workflow: phonon dispersions (+elastic, dielectric)

Single q calculation


Phonon initialization

Energy calculation

Input parameters

Dynamical matrices

Phonon calculation

Phonon calculation


Collect results Fourier interpolation

Phonon dispersion

q-points distribution

Loops on itself if fails (change parameters)

Restart if clean stop (max CPU time reached)

Phonon “restart” sub-workflow Testing metallic

character

Generating structures with random magnetizations

Structure

Magnetic energy relax.

Fully relaxed structure



Lowest energy configuration

Non-magnetic energy relaxation

Final energy relaxation + bands

Electronic bands

Energy calculation + bands

Finding magnetic properties

Set of tested & converged

pseudos (SSSP)

InlineCalculation (4825159)elastic_constants_inline()

ParameterData (4825161)

output_parameters

StructureData (4781156)InSe

structure

PwCalculation (4795040)relax FINISHED

structure

InlineCalculation (4781184)deformation_inline()

structure


structure


bestfit_1


bestfit_0

InlineCalculation (4781155)standardize_structure_inline()

standardized_structure

InlineCalculation (4825154)best_fit_inline()

output_parameters


output_parameters


parameters

StructureData (288318)'3D_with_2D_substructure'

InSe

structure


parameters


lagrangian_strain_8


lagrangian_strain_9

lagrangian_strain_10


lagrangian_strain_0


lagrangian_strain_1


lagrangian_strain_2


lagrangian_strain_3


lagrangian_strain_4


lagrangian_strain_5


lagrangian_strain_6


lagrangian_strain_7


parameters


lagrangian_strain_8ParameterData (4815214)

lagrangian_strain_9


lagrangian_strain_0 ParameterData (4795620)

lagrangian_strain_1


lagrangian_strain_2


lagrangian_strain_3



lagrangian_strain_5


lagrangian_strain_6


lagrangian_strain_7

PwCalculation (269210)vc-relax FINISHED

output_structure


output_parameters output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters

Code (124499)'pw-5.2-rhoxml-piz-dora_aprun'

code


code

SinglefileData (260128)

vdw_table

vdw_table vdw_tablevdw_tablevdw_table vdw_table vdw_tablevdw_tablevdw_table vdw_table vdw_table vdw_tablevdw_table vdw_table vdw_table vdw_tablevdw_table vdw_table vdw_table

vdw_table

vdw_table

vdw_table

PwCalculation (4793553)relax FAILED

vdw_table


parameters


settings

KpointsData (246769)10x10x2 (+0.0,0.0,0.0)

kpoints

kpoints kpointskpointskpoints kpoints kpointskpointskpoints kpoints kpoints kpointskpoints kpoints kpoints kpointskpoints kpoints kpoints

kpoints

kpoints

kpoints

kpoints

UpfData (81898)

pseudo_In

pseudo_In pseudo_Inpseudo_Inpseudo_In pseudo_In pseudo_Inpseudo_Inpseudo_In pseudo_In pseudo_In pseudo_Inpseudo_In pseudo_In pseudo_In pseudo_Inpseudo_In pseudo_In pseudo_In

pseudo_In

pseudo_In

pseudo_In

pseudo_In

UpfData (95553)

pseudo_Se

pseudo_Se pseudo_Sepseudo_Sepseudo_Se pseudo_Se pseudo_Sepseudo_Sepseudo_Se pseudo_Se pseudo_Se pseudo_Sepseudo_Se pseudo_Se pseudo_Se pseudo_Sepseudo_Se pseudo_Se pseudo_Se

pseudo_Se

pseudo_Se

pseudo_Se

pseudo_Se


InSe

structure

Code (4634612)'pw-5.2-rhoxml-piz-daint'

code codecodecode code codecodecode code code codecode code code codecode code code

code

code code


parameters


settings


structure


parameters


settings


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure

RemoteData (4793804)

parent_calc_folder


parameters


settings


structure

structure


parameters


settings


structure

output_structure

deformed_structure_8 deformed_structure_7deformed_structure_6 deformed_structure_4 deformed_structure_3deformed_structure_2deformed_structure_1 deformed_structure_0 deformed_structure_9 deformed_structure_9deformed_structure_8 deformed_structure_7 deformed_structure_6 deformed_structure_4deformed_structure_3 deformed_structure_2 deformed_structure_1 deformed_structure_0deformed_structure_10

remote_folder


parameters


settings


InSe

structure


parameters


parameters


parameters


settings

InlineCalculation (34977)primitive_structure_inline()

primitive_structure_spg

CifData (15308)

cif


parameters

CiffilterCalculation (37378) FINISHED

cif

Code (33048)'cif_select'

code

CifData (3743)

cif


parameters


cif

Code (24766)'cif_filter'

code

CifData (13415)

cif


parameters

InlineCalculation (4825159)elastic_constants_inline()


output_parameters


structure


structure


structure


structure


bestfit_1


bestfit_0

InlineCalculation (4781155)standardize_structure_inline()

standardized_structure


output_parameters


output_parameters


parameters


InSe

structure


parameters


lagrangian_strain_8


lagrangian_strain_9

lagrangian_strain_10


lagrangian_strain_0


lagrangian_strain_1


lagrangian_strain_2


lagrangian_strain_3


lagrangian_strain_4


lagrangian_strain_5


lagrangian_strain_6


lagrangian_strain_7


parameters


lagrangian_strain_8ParameterData (4815214)

lagrangian_strain_9



lagrangian_strain_1


lagrangian_strain_2


lagrangian_strain_3



lagrangian_strain_5


lagrangian_strain_6


lagrangian_strain_7


output_structure


output_parameters output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters


output_parameters

Code (124499)'pw-5.2-rhoxml-piz-dora_aprun'

code


code

SinglefileData (260128)

vdw_table

vdw_table vdw_tablevdw_tablevdw_table vdw_table vdw_tablevdw_tablevdw_table vdw_table vdw_table vdw_tablevdw_table vdw_table vdw_table vdw_tablevdw_table vdw_table vdw_table

vdw_table

vdw_table

vdw_table

PwCalculation (4793553)relax FAILED

vdw_table


parameters


settings

KpointsData (246769)10x10x2 (+0.0,0.0,0.0)

kpoints

kpoints kpointskpointskpoints kpoints kpointskpointskpoints kpoints kpoints kpointskpoints kpoints kpoints kpointskpoints kpoints kpoints

kpoints

kpoints

kpoints

kpoints

UpfData (81898)

pseudo_In

pseudo_In pseudo_Inpseudo_Inpseudo_In pseudo_In pseudo_Inpseudo_Inpseudo_In pseudo_In pseudo_In pseudo_Inpseudo_In pseudo_In pseudo_In pseudo_Inpseudo_In pseudo_In pseudo_In

pseudo_In

pseudo_In

pseudo_In

pseudo_In

UpfData (95553)

pseudo_Se

pseudo_Se pseudo_Sepseudo_Sepseudo_Se pseudo_Se pseudo_Sepseudo_Sepseudo_Se pseudo_Se pseudo_Se pseudo_Sepseudo_Se pseudo_Se pseudo_Se pseudo_Sepseudo_Se pseudo_Se pseudo_Se

pseudo_Se

pseudo_Se

pseudo_Se

pseudo_Se


InSe

structure

Code (4634612)'pw-5.2-rhoxml-piz-daint'

code codecodecode code codecodecode code code codecode code code codecode code code

code

code code


parameters


settings


structure


parameters


settings


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure


parameters


settings


structure

RemoteData (4793804)

parent_calc_folder


parameters


settings


structure

structure


parameters


settings


structure

output_structure

deformed_structure_8 deformed_structure_7deformed_structure_6 deformed_structure_4 deformed_structure_3deformed_structure_2deformed_structure_1 deformed_structure_0 deformed_structure_9 deformed_structure_9deformed_structure_8 deformed_structure_7 deformed_structure_6 deformed_structure_4deformed_structure_3 deformed_structure_2 deformed_structure_1 deformed_structure_0deformed_structure_10

remote_folder


parameters


settings


InSe

structure


parameters


parameters


parameters


settings

InlineCalculation (34977)primitive_structure_inline()

primitive_structure_spg

CifData (15308)

cif


parameters


cif

Code (33048)'cif_select'

code

CifData (3743)

cif


parameters


cif

Code (24766)'cif_filter'

code

CifData (13415)

cif


parameters

WHAT REALLY HAPPENS


ADES

Sharing in AiiDA

Social ecosystem Repository pipelines

Standardization

Sharing in AiiDA

Clusters

Users Databases

Private data

Public/shared data

Group 1

Group 3

Group 2

Some data shared

Some data shared

•  Sharing model in AiiDA •  Data can be pushed to the

outside world or other repositories

•  Importer of previous calculations

•  UUIDs used to uniquely identify all data/calculation objects

MATERIALS CLOUD INFRASTRUCTURE

•  server side AiiDA API •  federated data via iRODs •  client side API in AngularJS

CONCLUSIONS

l  In computational science, data are naturally

calculated, not harvested

l  ADES model

(automation – data – environment - sharing)

l  AiiDA v1.0 released by end of 2016

l  A DMP is part of (and distributed with) the AiiDA sw

l  AiiDA as a turn-key solution for Data management

Giovanni Pizzi

(EPFL)

Riccardo Sabatini

(Hum. Longevity)

Andrea Cepellotti (EPFL)

Andrius Merkys (Vilnius)

Nicolas Mounet (EPFL)

Boris Kozinsky (BOSCH)

Martin Uhrin

(EPFL)

Spyros Zoupanos

(EPFL)

Snehal Waychal (EPFL)

Nicola Varini

(EPFL)

Leonid Kahle

(EPFL)

Anton Kozhevnikov

(CSCS)

Fernando Gargiulo (EPFL)

THE AiiDA TEAM

GeorgySamsonidze,PrateekMehta,AndreaGreco@Bosch

SUPPORT MOSTLY FROM

http://nccr-marvel.ch http://www.bosch.us http://max-centre.eu http://nffa.eu http://emmc.info

handling data and workflows in computational materials science: the aiida initiative

Data & Analytics