Learning from Label Proportions with Consistency Regularization
arXiv:1910.13188
Abstract
The problem of learning from label proportions (LLP) involves training classifiers with weak labels on bags of instances, rather than strong labels on individual instances. The weak labels only contain the label proportion of each bag. The LLP problem is important for many practical applications that only allow label proportions to be collected because of data privacy or annotation cost, and has recently received lots of research attention. Most existing works focus on extending supervised learning models to solve the LLP problem, but the weak learning nature makes it hard to further improve LLP performance with a supervised angle. In this paper, we take a different angle from semi-supervised learning. In particular, we propose a novel model inspired by consistency regularization, a popular concept in semi-supervised learning that encourages the model to produce a decision boundary that better describes the data manifold. With the introduction of consistency regularization, we further extend our study to non-uniform bag-generation and validation-based parameter-selection procedures that better match practical needs. Experiments not only justify that LLP with consistency regularization achieves superior performance, but also demonstrate the practical usability of the proposed procedures.
References in corpus (10)
- Adam: A Method for Stochastic Optimization
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- Temporal Ensembling for Semi-Supervised Learning
- Interpolation Consistency Training for Semi-Supervised Learning
- MixMatch: A Holistic Approach to Semi-Supervised Learning
- Co-training for Demographic Classification Using Deep Learning from Label Proportions
- On Learning from Label Proportions
- Learning about individuals from group statistics
- SVM for learning with label proportions
- Deep multi-class learning from label proportions