Improvability Through Semi-Supervised Learning: A Survey of Theoretical Results

Abstract

Semi-supervised learning is a setting in which one has labeled and unlabeled data available. In this survey we explore different types of theoretical results when one uses unlabeled data in classification and regression tasks. Most methods that use unlabeled data rely on certain assumptions about the data distribution. When those assumptions are not met in reality, including unlabeled data may actually decrease performance. Studying such methods, it therefore is particularly important to have an understanding of the underlying theory. In this review we gather results about the possible gains one can achieve when using semi-supervised learning as well as results about the limits of such methods. More precisely, this review collects the answers to the following questions: What are, in terms of improving supervised methods, the limits of semi-supervised learning? What are the assumptions of different methods? What can we achieve if the assumptions are true? Finally, we also discuss the biggest bottleneck of semi-supervised learning, namely the assumptions they make.

References in corpus (6)

• On Causal and Anticausal Learning
• Local polynomial regression on unknown manifolds
• Generalization error bounds in semi-supervised classification under the cluster assumption
• Explicit Learning Curves for Transduction and Application to Clustering and Compression Algorithms
• A Distribution Dependent and Independent Complexity Analysis of Manifold Regularization
• The information-theoretic value of unlabeled data in semi-supervised learning

Cited by in corpus (4)

• A survey on Semi-, Self- and Unsupervised Learning for Image Classification
• Semi-Supervised Learning, Causality and the Conditional Cluster Assumption
• A Distribution Dependent and Independent Complexity Analysis of Manifold Regularization
• Metric learning by Similarity Network for Deep Semi-Supervised Learning