A method of supervised learning from conflicting data with hidden contexts

Abstract

Conventional supervised learning assumes a stable input-output relationship. However, this assumption fails in open-ended training settings where the input-output relationship depends on hidden contexts. In this work, we formulate a more general supervised learning problem in which training data is drawn from multiple unobservable domains, each potentially exhibiting distinct input-output maps. This inherent conflict in data renders standard empirical risk minimization training ineffective. To address this challenge, we propose a method LEAF that introduces an allocation function, which learns to assign conflicting data to different predictive models. We establish a connection between LEAF and a variant of the Expectation-Maximization algorithm, allowing us to derive an analytical expression for the allocation function. Finally, we provide a theoretical analysis of LEAF and empirically validate its effectiveness on both synthetic and real-world tasks involving conflicting data.

35 pages, 9 figures

References in corpus (14)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• A Convergence Theory for Deep Learning via Over-Parameterization
• Large-scale Multi-label Learning with Missing Labels
• Towards a Critical Race Methodology in Algorithmic Fairness
• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
• A PAC-Bayesian bound for Lifelong Learning
• Online Meta-Learning
• Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory
• Are we done with ImageNet?
• AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence
• Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL
• Visual Concept-Metaconcept Learning