A comprehensive study on the prediction reliability of graph neural networks for virtual screening
arXiv:2003.07611
Abstract
Prediction models based on deep neural networks are increasingly gaining attention for fast and accurate virtual screening systems. For decision makings in virtual screening, researchers find it useful to interpret an output of classification system as probability, since such interpretation allows them to filter out more desirable compounds. However, probabilistic interpretation cannot be correct for models that hold over-parameterization problems or inappropriate regularizations, leading to unreliable prediction and decision making. In this regard, we concern the reliability of neural prediction models on molecular properties, especially when models are trained with sparse data points and imbalanced distributions. This work aims to propose guidelines for training reliable models, we thus provide methodological details and ablation studies on the following train principles. We investigate the effects of model architectures, regularization methods, and loss functions on the prediction performance and reliability of classification results. Moreover, we evaluate prediction reliability of models on virtual screening scenario. Our result highlights that correct choice of regularization and inference methods is evidently important to achieve high success rate, especially in data imbalanced situation. All experiments were performed under a single unified model implementation to alleviate external randomness in model training and to enable precise comparison of results.
References in corpus (21)
- Adam: A Method for Stochastic Optimization
- Decoupled Weight Decay Regularization
- Semi-Supervised Classification with Graph Convolutional Networks
- Representation Learning with Contrastive Predictive Coding
- mixup: Beyond Empirical Risk Minimization
- Neural Message Passing for Quantum Chemistry
- Learning to Plan Chemical Syntheses
- On Calibration of Modern Neural Networks
- Molecular Graph Convolutions: Moving Beyond Fingerprints
- Understanding deep learning requires rethinking generalization
- Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- MolGAN: An implicit generative model for small molecular graphs
- Using Pre-Training Can Improve Model Robustness and Uncertainty
- Self-Attention Graph Pooling
- How Powerful are Graph Neural Networks?
- Bayesian Deep Learning and a Probabilistic Perspective of Generalization
- A Simple Baseline for Bayesian Uncertainty in Deep Learning
- Bayesian semi-supervised learning for uncertainty-calibrated prediction of molecular properties and active learning
- Practical Deep Learning with Bayesian Principles
- Retrosynthesis Prediction with Conditional Graph Logic Network