Learning to Pivot with Adversarial Networks

Abstract

Several techniques for domain adaptation have been proposed to account for differences in the distribution of the data used for training and testing. The majority of this work focuses on a binary domain label. Similar problems occur in a scientific context where there may be a continuous family of plausible data generation processes associated to the presence of systematic uncertainties. Robust inference is possible if it is based on a pivot -- a quantity whose distribution does not depend on the unknown values of the nuisance parameters that parametrize this family of data generation processes. In this work, we introduce and derive theoretical results for a training procedure based on adversarial networks for enforcing the pivotal property (or, equivalently, fairness with respect to continuous attributes) on a predictive model. The method includes a hyperparameter to control the trade-off between accuracy and robustness. We demonstrate the effectiveness of this approach with a toy example and examples from particle physics.

v1: Original submission. v2: Fixed references. v3: version submitted to NIPS'2017. Code available at https://github.com/glouppe/paper-learning-to-pivot

Cited by in corpus (45)

• A review of domain adaptation without target labels
• Fairness in Machine Learning: A Survey
• Invariant Risk Minimization
• Fader Networks: Manipulating Images by Sliding Attributes
• Identification of heavy, energetic, hadronically decaying particles using machine-learning techniques
• Adversarially-trained autoencoders for robust unsupervised new physics searches
• Think Locally, Act Globally: Federated Learning with Local and Global Representations
• Dijet resonance search with weak supervision using $\sqrt{s}=13$ TeV $pp$ collisions in the ATLAS detector
• Tag N' Train: A Technique to Train Improved Classifiers on Unlabeled Data
• Fast and accurate simulation of particle detectors using generative adversarial networks
• Simulation-Assisted Decorrelation for Resonant Anomaly Detection
• Modern Machine Learning and Particle Physics
• Interaction networks for the identification of boosted $H\to b\overline{b}$ decays
• A search for the standard model Higgs boson decaying to charm quarks
• Interpretable Deep Learning for Two-Prong Jet Classification with Jet Spectra
• INFERNO: Inference-Aware Neural Optimisation
• Online-compatible Unsupervised Non-resonant Anomaly Detection
• Measurements of Higgs Bosons Decaying to Bottom Quarks from Vector Boson Fusion Production with the ATLAS Experiment at $\sqrt{s}=$ 13 TeV
• Anomalous Jet Identification via Sequence Modeling
• A Cautionary Tale of Decorrelating Theory Uncertainties
• LHC analysis-specific datasets with Generative Adversarial Networks
• Opening the black box of neural nets: case studies in stop/top discrimination
• A survey on Adversarial Recommender Systems: from Attack/Defense strategies to Generative Adversarial Networks
• Achieving Equalized Odds by Resampling Sensitive Attributes
• Overcoming Language Priors in Visual Question Answering with Adversarial Regularization
• Matrix Element Regression with Deep Neural Networks -- breaking the CPU barrier
• Charm-quark Yukawa Coupling in $h\to c\bar{c}\gamma$ at LHC
• Partially Encrypted Machine Learning using Functional Encryption
• Invariant Representations from Adversarially Censored Autoencoders
• Fairness-Aware Neural Réyni Minimization for Continuous Features
• Dealing with Nuisance Parameters using Machine Learning in High Energy Physics: a Review
• The Fairness-Accuracy Pareto Front
• GanDef: A GAN based Adversarial Training Defense for Neural Network Classifier
• NIPS - Not Even Wrong? A Systematic Review of Empirically Complete Demonstrations of Algorithmic Effectiveness in the Machine Learning and Artificial Intelligence Literature
• Strategic Plan for a Scientific Software Innovation Institute (S2I2) for High Energy Physics
• Adjusting for Confounding in Unsupervised Latent Representations of Images
• The use of adversaries for optimal neural network training
• Towards Reducing Bias in Gender Classification
• Towards Learning Fine-Grained Disentangled Representations from Speech
• Fairness-aware Personalized Ranking Recommendation via Adversarial Learning
• ZK-GanDef: A GAN based Zero Knowledge Adversarial Training Defense for Neural Networks
• Image-Based Jet Analysis
• Scrutinizing and De-Biasing Intuitive Physics with Neural Stethoscopes
• Towards Auditability for Fairness in Deep Learning
• Learning to Collaborate for User-Controlled Privacy