AiiDA 1.0, a scalable computational infrastructure for automated reproducible workflows and data provenance

Abstract

The ever-growing availability of computing power and the sustained development of advanced computational methods have contributed much to recent scientific progress. These developments present new challenges driven by the sheer amount of calculations and data to manage. Next-generation exascale supercomputers will harden these challenges, such that automated and scalable solutions become crucial. In recent years, we have been developing AiiDA (http://www.aiida.net), a robust open-source high-throughput infrastructure addressing the challenges arising from the needs of automated workflow management and data provenance recording. Here, we introduce developments and capabilities required to reach sustained performance, with AiiDA supporting throughputs of tens of thousands processes/hour, while automatically preserving and storing the full data provenance in a relational database making it queryable and traversable, thus enabling high-performance data analytics. AiiDA's workflow language provides advanced automation, error handling features and a flexible plugin model to allow interfacing with any simulation software. The associated plugin registry enables seamless sharing of extensions, empowering a vibrant user community dedicated to making simulations more robust, user-friendly and reproducible.

References in corpus (20)

• Quantum ESPRESSO: a modular and open-source software project for quantum simulations of materials
• The SIESTA method for ab initio order-N materials simulation
• Efficient index handling of multidimensional periodic boundary conditions
• Two-dimensional materials from high-throughput computational exfoliation of experimentally known compounds
• Wannier90 as a community code: new features and applications
• AFLOW: An automatic framework for high-throughput materials discovery
• Precision and efficiency in solid-state pseudopotential calculations
• Band structure diagram paths based on crystallography
• AiiDA: Automated Interactive Infrastructure and Database for Computational Science
• Engineering of robust topological quantum phases in graphene nanoribbons
• Many-body perturbation theory calculations using the yambo code
• Materials Cloud, a platform for open computational science
• Parsl: Pervasive Parallel Programming in Python
• Accurate and efficient linear scaling DFT calculations with universal applicability
• Workflows in AiiDA: Engineering a high-throughput, event-based engine for robust and modular computational workflows
• Simple Data and Workflow Management with the signac Framework
• High-throughput computational screening for solid-state Li-ion conductors
• Automated high-throughput Wannierisation
• Electronic and optical properties of doped TiO2 by many-body perturbation theory
• A posteriori metadata from automated provenance tracking: Integration of AiiDA and TCOD

Cited by in corpus (86)

• Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems
• Materials Cloud, a platform for open computational science
• TB2J: a python package for computing magnetic interaction parameters
• Workflows in AiiDA: Engineering a high-throughput, event-based engine for robust and modular computational workflows
• HP -- A code for the calculation of Hubbard parameters using density-functional perturbation theory
• A Deep Dive into Machine Learning Density Functional Theory for Materials Science and Chemistry
• OPTIMADE, an API for exchanging materials data
• AiiDAlab -- an ecosystem for developing, executing, and sharing scientific workflows
• How to verify the precision of density-functional-theory implementations via reproducible and universal workflows
• doped: Python toolkit for robust and repeatable charged defect supercell calculations
• Thermodynamics and dielectric response of ${\text{BaTiO}}_3$ by data-driven modeling
• Skyrmionic Spin Structures in Layered Fe5GeTe2 Up To Room Temperature
• Predicting binding motifs of complex adsorbates using machine learning with a physics-inspired graph representation
• Accurate electronic properties and intercalation voltages of olivine-type Li-ion cathode materials from extended Hubbard functionals
• El Agente: An Autonomous Agent for Quantum Chemistry
• Developments and applications of the OPTIMADE API for materials discovery, design, and data exchange
• Machine-learning structural reconstructions for accelerated point defect calculations
• Universal materials model of deep-learning density functional theory Hamiltonian
• Common workflows for computing material properties using different quantum engines
• Fermi energy determination for advanced smearing techniques
• Multiple antiferromagnetic phases and magnetic anisotropy in exfoliated CrBr${}_3$ multilayers
• Orbital-resolved DFT+U for molecules and solids
• Artificial Intelligence in Materials Science and Engineering: Current Landscape, Key Challenges, and Future Trajectorie
• koopmans: an open-source package for accurately and efficiently predicting spectral properties with Koopmans functionals
• Abinit 2025: New Capabilities for the Predictive Modeling of Solids and Nanomaterials
• EPIq : an open-source software for the calculation of electron-phonon interaction related properties
• The Wannier Function Software Ecosystem for Materials Simulations
• Automated all-functionals infrared and Raman spectra
• Projectability disentanglement for accurate and automated electronic-structure Hamiltonians
• Remote free-carrier screening to boost the mobility of Fröhlich-limited 2D semiconductors
• Density functional Bogoliubov-de Gennes analysis of superconducting Nb and Nb(110) surfaces
• Understanding the role of Hubbard corrections in the rhombohedral phase of BaTiO${}_3$
• Magnetism and stability of all primitive stacking patterns in bilayer chromium trihalides
• Machine-learning accelerated identification of exfoliable two-dimensional materials
• Magnetic domain walls of the van der Waals material Fe${}_3$GeTe${}_2$
• Microscopic picture of paraelectric perovskites from structural prototypes
• Towards high-throughput many-body perturbation theory: efficient algorithms and automated workflows
• A Terminology for Scientific Workflow Systems
• Automated mixing of maximally localized Wannier functions into target manifolds
• Enhancing ReaxFF for Molecular Dynamics Simulations of Lithium-Ion Batteries: An interactive reparameterization protocol
• Exploring the Cs-Te phase space via high-throughput density-functional theory calculations beyond the generalized-gradient approximation
• A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations
• Screening Mixed-Metal Sn${}_2$M(III)Ch${}_2$X${}_3$ Chalcohalides for Photovoltaic Applications
• TurboGenius: Python suite for high-throughput calculations of ab initio quantum Monte Carlo methods
• Atomic diffusion-induced polarization and superconductivity in topological insulator-based heterostructures
• Multitask methods for predicting molecular properties from heterogeneous data
• High-throughput screening of Weyl semimetals
• Training-free hyperparameter optimization of neural networks for electronic structures in matter
• Charting the landscape of Bardeen-Cooper-Schrieffer superconductors in experimentally known compounds
• High-throughput calculations of spin Hall conductivity in non-magnetic 2D materials
• First-principles Hubbard parameters with automated and reproducible workflows
• A robust and efficient line search for self-consistent field iterations
• Inter-orbital Cooper pairing at finite energies in Rashba surface states
• A Fully Quantum-Mechanical Treatment for Kaolinite
• The type-I antiferromagnetic Weyl semimetal InMnTi${}_2$
• Interoperable Workflows by Exchanging Grid-Based Data between Quantum-Chemical Program Packages
• A complementary screening for quantum spin Hall insulators in 2D exfoliable materials
• A Python workflow definition for computational materials design
• Magnons from time-dependent density-functional perturbation theory and nonempirical Hubbard functionals
• Low-energy modeling of three-dimensional topological insulator nanostructures
• Change Point Detection of Events in Molecular Simulations using dupin
• mkite: A distributed computing platform for high-throughput materials simulations
• Automated computational workflows for muon spin spectroscopy
• The AiiDA-Spirit plugin for automated spin-dynamics simulations and multi-scale modelling based on first-principles calculations
• MC3D: The Materials Cloud computational database of experimentally known stoichiometric inorganics
• CateCom: a practical data-centric approach to categorization of computational models
• Numerical simulation projects in micromagnetics with Jupyter
• Enhanced Spin Hall Ratio in Two-Dimensional Semiconductors
• Spectral Operator Representations
• Metadata practices for simulation workflows
• Robust Wannierization including magnetization and spin-orbit coupling via projectability disentanglement
• Comparative study of magnetic exchange parameters and magnon dispersions in NiO and MnO from first principles
• Accelerating point defect simulations using data-driven and machine learning approaches
• mmodel: A workflow framework to accelerate the development of experimental simulations
• Strategies to search for two-dimensional materials with long spin qubit coherence time
• Exploratory Data Science on Supercomputers for Quantum Mechanical Calculations
• Making atomistic materials calculations accessible with the AiiDAlab Quantum ESPRESSO app
• AMaRaNTA: Automated First-Principles Exchange Parameters In 2D Magnets
• Towards knowledge-based workflows: a semantic approach to atomistic simulations for mechanical and thermodynamic properties
• Automated Prediction of Thermodynamic Properties via Bayesian Free-Energy Reconstruction from Molecular Dynamics
• Ab initio functional-independent calculations of the clamped Pockels tensor of tetragonal barium titanate
• M-CODE: Materials Categorization via Ontology, Dimensionality and Evolution
• Elastic Constants and Bending Rigidities from Long-Wavelength Perturbation Expansions
• Uncoupling System and Environment Simulation Cells for Fast-Scaling Modeling of Complex Continuum Embeddings
• Score-based diffusion models for accurate crystal-structure inpainting and reconstruction of hydrogen positions
• Density-functional description of materials for topological qubits and superconducting spintronics