Satisfying Real-world Goals with Dataset Constraints

Abstract

The goal of minimizing misclassification error on a training set is often just one of several real-world goals that might be defined on different datasets. For example, one may require a classifier to also make positive predictions at some specified rate for some subpopulation (fairness), or to achieve a specified empirical recall. Other real-world goals include reducing churn with respect to a previously deployed model, or stabilizing online training. In this paper we propose handling multiple goals on multiple datasets by training with dataset constraints, using the ramp penalty to accurately quantify costs, and present an efficient algorithm to approximately optimize the resulting non-convex constrained optimization problem. Experiments on both benchmark and real-world industry datasets demonstrate the effectiveness of our approach.

References in corpus (1)

• Equality of Opportunity in Supervised Learning

Cited by in corpus (39)

• Fairness in Machine Learning: A Survey
• Fairness in Machine Learning
• 50 Years of Test (Un)fairness: Lessons for Machine Learning
• User-oriented Fairness in Recommendation
• Automated Directed Fairness Testing
• Connecting User and Item Perspectives in Popularity Debiasing for Collaborative Recommendation
• Identifying and Correcting Label Bias in Machine Learning
• The Promise and Peril of Human Evaluation for Model Interpretability
• Hierarchically Fair Federated Learning
• Robust Optimization for Fairness with Noisy Protected Groups
• Mitigating Bias in Federated Learning
• Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
• Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems
• Improved Adversarial Learning for Fair Classification
• Transfer of Machine Learning Fairness across Domains
• Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements
• Review of Mathematical frameworks for Fairness in Machine Learning
• Equalizing Recourse across Groups
• On the Reproducibility of Neural Network Predictions
• Tuning Fairness by Balancing Target Labels
• Fairness-aware Classification: Criterion, Convexity, and Bounds
• Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints
• Privacy for All: Demystify Vulnerability Disparity of Differential Privacy against Membership Inference Attack
• Two-Player Games for Efficient Non-Convex Constrained Optimization
• Deontological Ethics By Monotonicity Shape Constraints
• Fair Classification with Noisy Protected Attributes: A Framework with Provable Guarantees
• Toward a better trade-off between performance and fairness with kernel-based distribution matching
• Understanding the Downstream Instability of Word Embeddings
• Pareto Efficient Fairness in Supervised Learning: From Extraction to Tracing
• Pairwise Fairness for Ranking and Regression
• Optimizing Generalized Rate Metrics through Game Equilibrium
• Automatic Fairness Testing of Neural Classifiers through Adversarial Sampling
• Technologies for Trustworthy Machine Learning: A Survey in a Socio-Technical Context
• Locally Adaptive Label Smoothing for Predictive Churn
• Churn Reduction via Distillation
• Regularization Strategies for Quantile Regression
• Fair Classification with Adversarial Perturbations
• Group-based Fair Learning Leads to Counter-intuitive Predictions
• Training Over-parameterized Models with Non-decomposable Objectives