Twin Neural Network Regression

Abstract

We introduce twin neural network (TNN) regression. This method predicts differences between the target values of two different data points rather than the targets themselves. The solution of a traditional regression problem is then obtained by averaging over an ensemble of all predicted differences between the targets of an unseen data point and all training data points. Whereas ensembles are normally costly to produce, TNN regression intrinsically creates an ensemble of predictions of twice the size of the training set while only training a single neural network. Since ensembles have been shown to be more accurate than single models this property naturally transfers to TNN regression. We show that TNNs are able to compete or yield more accurate predictions for different data sets, compared to other state-of-the-art methods. Furthermore, TNN regression is constrained by self-consistency conditions. We find that the violation of these conditions provides an estimate for the prediction uncertainty.

References in corpus (20)

• Scikit-learn: Machine Learning in Python
• Gaussian Approximation Potentials: the accuracy of quantum mechanics, without the electrons
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulation
• Learning with Pseudo-Ensembles
• Learning to Order Things
• Fast and Accurate Uncertainty Estimation in Chemical Machine Learning
• Deep Learning and Density Functional Theory
• Theory and Applications of Proper Scoring Rules
• Deep Confidence: A Computationally Efficient Framework for Calculating Reliable Errors for Deep Neural Networks
• Error-Controlled Exploration of Chemical Reaction Networks with Gaussian Processes
• Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning
• Gaussian Process-Based Refinement of Dispersion Corrections
• Discovering Symmetry Invariants and Conserved Quantities by Interpreting Siamese Neural Networks
• Minimum energy path calculations with Gaussian process regression
• Siamese Regression Networks with Efficient mid-level Feature Extraction for 3D Object Pose Estimation
• Hydra: Preserving Ensemble Diversity for Model Distillation
• Twin Neural Network Regression is a Semi-Supervised Regression Algorithm

Cited by in corpus (4)

• Twin Neural Network Regression is a Semi-Supervised Regression Algorithm
• Uncertainty Quantification in Pairwise Difference Learning for Classification
• Applying the Case Difference Heuristic to Learn Adaptations from Deep Network Features
• Efficient Pairwise Difference Learning: A Comparative Study of Encoding Strategies and Training Pair Selection