Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning

Abstract

As annotations of data can be scarce in large-scale practical problems, leveraging unlabelled examples is one of the most important aspects of machine learning. This is the aim of semi-supervised learning. To benefit from the access to unlabelled data, it is natural to diffuse smoothly knowledge of labelled data to unlabelled one. This induces to the use of Laplacian regularization. Yet, current implementations of Laplacian regularization suffer from several drawbacks, notably the well-known curse of dimensionality. In this paper, we provide a statistical analysis to overcome those issues, and unveil a large body of spectral filtering methods that exhibit desirable behaviors. They are implemented through (reproducing) kernel methods, for which we provide realistic computational guidelines in order to make our method usable with large amounts of data.

arXiv comment38 pages, 6 figures

References in corpus (11)

• HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
• Interpolation Consistency Training for Semi-Supervised Learning
• Large-scale Multi-label Learning with Missing Labels
• Generalization error bounds in semi-supervised classification under the cluster assumption
• Nonparametric sparsity and regularization
• Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spaces
• Scaling Gaussian Process Regression with Derivatives
• Asymptotic behavior of ${\ell }_p$-based Laplacian regularization in semi-supervised learning
• Structured Learning from Partial Annotations
• Statistical Estimation of the Poincar{é} constant and Application to Sampling Multimodal Distributions
• Disambiguation of weak supervision with exponential convergence rates