Building Robust Machine Learning Models for Small Chemical Science Data: The Case of Shear Viscosity

Abstract

Shear viscosity, though being a fundamental property of all liquids, is computationally expensive to estimate from equilibrium molecular dynamics simulations. Recently, Machine Learning (ML) methods have been used to augment molecular simulations in many contexts, thus showing promise to estimate viscosity too in a relatively inexpensive manner. However, ML methods face significant challenges like overfitting when the size of the data set is small, as is the case with viscosity. In this work, we train several ML models to predict the shear viscosity of a Lennard-Jones (LJ) fluid, with particular emphasis on addressing issues arising from a small data set. Specifically, the issues related to model selection, performance estimation and uncertainty quantification were investigated. First, we show that the widely used performance estimation procedure of using a single unseen data set shows a wide variability on small data sets. In this context, the common practice of using Cross validation (CV) to select the hyperparameters (model selection) can be adapted to estimate the generalization error (performance estimation) as well. We compare two simple CV procedures for their ability to do both model selection and performance estimation, and find that k-fold CV based procedure shows a lower variance of error estimates. We discuss the role of performance metrics in training and evaluation. Finally, Gaussian Process Regression (GPR) and ensemble methods were used to estimate the uncertainty on individual predictions. The uncertainty estimates from GPR were also used to construct an applicability domain using which the ML models provided more reliable predictions on another small data set generated in this work. Overall, the procedures prescribed in this work, together, lead to robust ML models for small data sets.

arXiv commentmain: 17 pages, 11 figures ; SI: 55 pages, 29 figures ; to be submitted to Journal of Chemical Physics

References in corpus (21)

• Scikit-learn: Machine Learning in Python
• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Array Programming with NumPy
• A survey of cross-validation procedures for model selection
• Machine learning for molecular simulation
• TorchMD: A deep learning framework for molecular simulations
• The first principles calculation of transport coefficients
• Leakage and the Reproducibility Crisis in ML-based Science
• Identifying time scales for violation/preservation of Stokes-Einstein relation in supercooled water
• Uncertainty estimation for molecular dynamics and sampling
• Probing the link between residual entropy and viscosity of molecular fluids and model potentials
• Strategies for the Construction of Machine-Learning Potentials for Accurate and Efficient Atomic-Scale Simulations
• Uncertainty Prediction for Machine Learning Models of Material Properties
• Diffusion coefficient and shear viscosity of rigid water models
• Validating the Validation: Reanalyzing a large-scale comparison of Deep Learning and Machine Learning models for bioactivity prediction
• Metrics for Benchmarking and Uncertainty Quantification: Quality, Applicability, and a Path to Best Practices for Machine Learning in Chemistry
• Viscosity in water from first-principles and deep-neural-network simulations
• Nature of Intrinsic Uncertainties in Equilibrium Molecular Dynamics Estimation of Shear Viscosity for Simple and Complex Fluids
• Microscopic origins of the viscosity of a Lennard-Jones liquid
• Density-Dependent Finite System-Size Effects in Equilibrium Molecular Dynamics Estimation of Shear Viscosity: Hydrodynamic and Configurational Study
• Super-resolution in Molecular Dynamics Trajectory Reconstruction with Bi-Directional Neural Networks