Online Semi-Supervised Learning with Deep Hybrid Boltzmann Machines and Denoising Autoencoders

Abstract

Two novel deep hybrid architectures, the Deep Hybrid Boltzmann Machine and the Deep Hybrid Denoising Auto-encoder, are proposed for handling semi-supervised learning problems. The models combine experts that model relevant distributions at different levels of abstraction to improve overall predictive performance on discriminative tasks. Theoretical motivations and algorithms for joint learning for each are presented. We apply the new models to the domain of data-streams in work towards life-long learning. The proposed architectures show improved performance compared to a pseudo-labeled, drop-out rectifier network.

References in corpus (4)

• Improving neural networks by preventing co-adaptation of feature detectors
• Deeply-Supervised Nets
• Sum-Product Networks: A New Deep Architecture
• Layer-wise learning of deep generative models

Cited by in corpus (8)

• Conducting Credit Assignment by Aligning Local Representations
• Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data
• Speaker Diarization as a Fully Online Learning Problem in MiniVox
• Learning to Adapt by Minimizing Discrepancy
• Spiking Neural Predictive Coding for Continual Learning from Data Streams
• Contextual Bandit with Missing Rewards
• Contextual Bandit with Adaptive Feature Extraction
• Online Semi-Supervised Learning in Contextual Bandits with Episodic Reward