Generative Models For Deep Learning with Very Scarce Data

Abstract

The goal of this paper is to deal with a data scarcity scenario where deep learning techniques use to fail. We compare the use of two well established techniques, Restricted Boltzmann Machines and Variational Auto-encoders, as generative models in order to increase the training set in a classification framework. Essentially, we rely on Markov Chain Monte Carlo (MCMC) algorithms for generating new samples. We show that generalization can be improved comparing this methodology to other state-of-the-art techniques, e.g. semi-supervised learning with ladder networks. Furthermore, we show that RBM is better than VAE generating new samples for training a classifier with good generalization capabilities.

References in corpus (6)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Sequence to Sequence Learning with Neural Networks
• Conditional Image Generation with PixelCNN Decoders
• Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference
• Semi-Supervised Learning with Ladder Networks
• A Bayesian Data Augmentation Approach for Learning Deep Models

Cited by in corpus (1)

• Model Patching: Closing the Subgroup Performance Gap with Data Augmentation