Regression-clustering for Improved Accuracy and Training Cost with Molecular-Orbital-Based Machine Learning

Abstract

Machine learning (ML) in the representation of molecular-orbital-based (MOB) features has been shown to be an accurate and transferable approach to the prediction of post-Hartree-Fock correlation energies. Previous applications of MOB-ML employed Gaussian Process Regression (GPR), which provides good prediction accuracy with small training sets; however, the cost of GPR training scales cubically with the amount of data and becomes a computational bottleneck for large training sets. In the current work, we address this problem by introducing a clustering/regression/classification implementation of MOB-ML. In a first step, regression clustering (RC) is used to partition the training data to best fit an ensemble of linear regression (LR) models; in a second step, each cluster is regressed independently, using either LR or GPR; and in a third step, a random forest classifier (RFC) is trained for the prediction of cluster assignments based on MOB feature values. Upon inspection, RC is found to recapitulate chemically intuitive groupings of the frontier molecular orbitals, and the combined RC/LR/RFC and RC/GPR/RFC implementations of MOB-ML are found to provide good prediction accuracy with greatly reduced wall-clock training times. For a dataset of thermalized geometries of 7211 organic molecules of up to seven heavy atoms, both implementations reach chemical accuracy (1 kcal/mol error) with only 300 training molecules, while providing 35000-fold and 4500-fold reductions in the wall-clock training time, respectively, compared to MOB-ML without clustering. The resulting models are also demonstrated to retain transferability for the prediction of large-molecule energies with only small-molecule training data. Finally, it is shown that capping the number of training datapoints per cluster leads to further improvements in prediction accuracy with negligible increases in wall-clock training time.

arXiv comment31 pages, 10 figures, with an SI

References in corpus (24)

• Scikit-learn: Machine Learning in Python
• Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics
• ANI-1: An extensible neural network potential with DFT accuracy at force field computational cost
• Learning to Plan Chemical Syntheses
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• Quantum-Chemical Insights from Deep Tensor Neural Networks
• Deep Reinforcement Learning for De-Novo Drug Design
• Big Data meets Quantum Chemistry Approximations: The $\Delta$-Machine Learning Approach
• By-passing the Kohn-Sham equations with machine learning
• Machine Learning of Molecular Electronic Properties in Chemical Compound Space
• Machine Learning Molecular Dynamics for the Simulation of Infrared Spectra
• Neural networks for the prediction organic chemistry reactions
• FCHL revisited: faster and more accurate quantum machine learning
• Alchemical and structural distribution based representation for improved QML
• Hierarchical modeling of molecular energies using a deep neural network
• Electronic Spectra from TDDFT and Machine Learning in Chemical Space
• A Transferable Machine-Learning Model of the Electron Density
• Transferability in Machine Learning for Electronic Structure via the Molecular Orbital Basis
• Comparison of permutationally invariant polynomials, neural networks, and Gaussian approximation potentials in representing water interactions through many-body expansions
• Operators in Machine Learning: Response Properties in Chemical Space
• A Universal Density Matrix Functional from Molecular Orbital-Based Machine Learning: Transferability across Organic Molecules
• Gaussian approximation potential modeling of lithium intercalation in carbon nanostructures

Cited by in corpus (14)

• OrbNet: Deep Learning for Quantum Chemistry Using Symmetry-Adapted Atomic-Orbital Features
• DeePKS: a comprehensive data-driven approach towards chemically accurate density functional theory
• Ground state energy functional with Hartree-Fock efficiency and chemical accuracy
• Improved accuracy and transferability of molecular-orbital-based machine learning: Organics, transition-metal complexes, non-covalent interactions, and transition states
• Accurate Molecular-Orbital-Based Machine Learning Energies via Unsupervised Clustering of Chemical Space
• An orbital-based representation for accurate Quantum Machine Learning
• Analytical Gradients for Molecular-Orbital-Based Machine Learning
• Mean-Field Density Matrix Decompositions
• Data-driven Refinement of Electronic Energies from Two-Electron Reduced-Density-Matrix Theory
• Molecular-orbital-based Machine Learning for Open-shell and Multi-reference Systems with Kernel Addition Gaussian Process Regression
• Low-data deep quantum chemical learning for accurate MP2 and coupled-cluster correlations
• Molecular Dipole Moment Learning via Rotationally Equivariant Gaussian Process Regression with Derivatives in Molecular-orbital-based Machine Learning
• Physical machine learning outperforms "human learning" in Quantum Chemistry
• Assessing zero-shot generalisation behaviour in graph-neural-network interatomic potentials