Investigating the Effect of Intraclass Variability in Temporal Ensembling
arXiv:2008.08956
Abstract
Temporal Ensembling is a semi-supervised approach that allows training deep neural network models with a small number of labeled images. In this paper, we present our preliminary study on the effect of intraclass variability on temporal ensembling, with a focus on seed size and seed type, respectively. Through our experiments we find that (a) there is a significant drop in accuracy with datasets that offer high intraclass variability, (b) more seed images offer consistently higher accuracy across the datasets, and (c) seed type indeed has an impact on the overall efficiency, where it produces a spectrum of accuracy both lower and higher. Additionally, based on our experiments, we also find KMNIST to be a competitive baseline for temporal ensembling.
arXiv commentPreliminary Results; More Experiments to be added
References in corpus (7)
- Auto-Encoding Variational Bayes
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
- Semi-Supervised Learning with Deep Generative Models
- Semi-Supervised Learning with Ladder Networks
- Bayesian Semi-supervised Learning with Graph Gaussian Processes
- Bayesian Semisupervised Learning with Deep Generative Models