Extensive deep neural networks for transferring small scale learning to large scale systems

Abstract

We present a physically-motivated topology of a deep neural network that can efficiently infer extensive parameters (such as energy, entropy, or number of particles) of arbitrarily large systems, doing so with O(N) scaling. We use a form of domain decomposition for training and inference, where each sub-domain (tile) is comprised of a non-overlapping focus region surrounded by an overlapping context region. The size of these regions is motivated by the physical interaction length scales of the problem. We demonstrate the application of EDNNs to three physical systems: the Ising model and two hexagonal/graphene-like datasets. In the latter, an EDNN was able to make total energy predictions of a 60 atoms system, with comparable accuracy to density functional theory (DFT), in 57 milliseconds. Additionally EDNNs are well suited for massively parallel evaluation, as no communication is necessary during neural network evaluation. We demonstrate that EDNNs can be used to make an energy prediction of a two-dimensional 35.2 million atom system, over 1 square micrometer of material, at an accuracy comparable to DFT, in under 25 minutes. Such a system exists on a length scale visible with optical microscopy and larger than some living organisms.

arXiv commentEDNN training code and tutorials available at http://kylemills.net/ednn, porous graphene dataset available for download at https://doi.org/10.4224/c8sc04578j.data

References in corpus (27)

• Automatic chemical design using a data-driven continuous representation of molecules
• Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
• On representing chemical environments
• SchNet - a deep learning architecture for molecules and materials
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Machine learning phases of matter
• Learning phase transitions by confusion
• By-passing the Kohn-Sham equations with machine learning
• Machine-learning based interatomic potential for amorphous carbon
• Neural networks for the prediction organic chemistry reactions
• Insightful classification of crystal structures using deep learning
• A Convolutional Neural Network Neutrino Event Classifier
• Quantum Entanglement in Deep Learning Architectures
• Deep learning and the Schrödinger equation
• Non-covalent interactions across organic and biological subsets of chemical space: Physics-based potentials parametrized from machine learning
• Machine learning for many-body physics: The case of the Anderson impurity model
• Convolutional Neural Networks Applied to Neutrino Events in a Liquid Argon Time Projection Chamber
• Solving frustrated quantum many-particle models with convolutional neural networks
• Smallest Neural Network to Learn the Ising Criticality
• Unsupervised machine learning for detection of phase transitions in off-lattice systems I. Foundations
• Convolutional neural networks for atomistic systems
• Unsupervised machine learning for detection of phase transitions in off-lattice systems II. Applications
• Nearsightedness of Electronic Matter in One Dimension
• Deep neural networks for direct, featureless learning through observation: the case of 2d spin models
• Smart energy models for atomistic simulations using a DFT-driven multifidelity approach

Cited by in corpus (23)

• Machine learning and the physical sciences
• Physics-inspired structural representations for molecules and materials
• Deep-Learning Density Functional Theory Hamiltonian for Efficient ab initio Electronic-Structure Calculation
• Deep Learning and Density Functional Theory
• Predicting electronic structures at any length scale with machine learning
• Drawing Phase Diagrams of Random Quantum Systems by Deep Learning the Wave Functions
• Scalable neural networks for the efficient learning of disordered quantum systems
• Machine Learning Diffusion Monte Carlo Energies
• Deep learning and high harmonic generation
• Supervised learning of few dirty bosons with variable particle number
• Supervised learning of random quantum circuits via scalable neural networks
• Convolutional network learning of self-consistent electron density via grid-projected atomic fingerprints
• Machine learning for structure-property relationships: Scalability and limitations
• Watch and learn -- a generalized approach for transferrable learning in deep neural networks via physical principles
• Machine learning the electronic structure of matter across temperatures
• Deep learning density functionals for gradient descent optimization
• A learning algorithm with emergent scaling behavior for classifying phase transitions
• Deep learning nonlocal and scalable energy functionals for quantum Ising models
• Synergy between noisy quantum computers and scalable classical deep learning
• Machine-learning semi-local density functional theory for many-body lattice models at zero and finite temperature
• Challenges and opportunities in the supervised learning of quantum circuit outputs
• DDU-Net: A Domain Decomposition-Based CNN for High-Resolution Image Segmentation on Multiple GPUs
• Weakly-supervised multi-class object localization using only object counts as labels