Regression Under Human Assistance

Abstract

Decisions are increasingly taken by both humans and machine learning models. However, machine learning models are currently trained for full automation -- they are not aware that some of the decisions may still be taken by humans. In this paper, we take a first step towards the development of machine learning models that are optimized to operate under different automation levels. More specifically, we first introduce the problem of ridge regression under human assistance and show that it is NP-hard. Then, we derive an alternative representation of the corresponding objective function as a difference of nondecreasing submodular functions. Building on this representation, we further show that the objective is nondecreasing and satisfies $\alpha$-submodularity, a recently introduced notion of approximate submodularity. These properties allow a simple and efficient greedy algorithm to enjoy approximation guarantees at solving the problem. Experiments on synthetic and real-world data from two important applications -- medical diagnosis and content moderation-demonstrate that our algorithm outsources to humans those samples in which the prediction error of the ridge regression model would have been the highest if it had to make a prediction, it outperforms several competitive baselines, and its performance is robust with respect to several design choices and hyperparameters used in the experiments.

arXiv commentExtended version of AAAI 2020 paper

References in corpus (18)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Cooperative Inverse Reinforcement Learning
• Combating Label Noise in Deep Learning Using Abstention
• Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications
• Learning Policy Representations in Multiagent Systems
• The Algorithmic Automation Problem: Prediction, Triage, and Human Effort
• Direct Uncertainty Prediction for Medical Second Opinions
• Submodular Maximization Beyond Non-negativity: Guarantees, Fast Algorithms, and Applications
• Teaching Inverse Reinforcement Learners via Features and Demonstrations
• Robust Maximization of Non-Submodular Objectives
• Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraints
• Active Regression via Linear-Sample Sparsification
• Mathematical Models of Adaptation in Human-Robot Collaboration
• Learning Not to Learn in the Presence of Noisy Labels
• Adaptive Hard Thresholding for Near-optimal Consistent Robust Regression
• Active Regression by Stratification
• Towards Deployment of Robust AI Agents for Human-Machine Partnerships
• Non-submodular Function Maximization subject to a Matroid Constraint, with Applications