Semi-Supervised Learning with Deep Generative Models

Abstract

The ever-increasing size of modern data sets combined with the difficulty of obtaining label information has made semi-supervised learning one of the problems of significant practical importance in modern data analysis. We revisit the approach to semi-supervised learning with generative models and develop new models that allow for effective generalisation from small labelled data sets to large unlabelled ones. Generative approaches have thus far been either inflexible, inefficient or non-scalable. We show that deep generative models and approximate Bayesian inference exploiting recent advances in variational methods can be used to provide significant improvements, making generative approaches highly competitive for semi-supervised learning.

To appear in the proceedings of Neural Information Processing Systems (NIPS) 2014

References in corpus (1)

• Auto-Encoding Variational Bayes

Cited by in corpus (335)

• Bootstrap your own latent: A new approach to self-supervised Learning
• An Introduction to Variational Autoencoders
• Unsupervised Data Augmentation for Consistency Training
• Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
• Tutorial on Variational Autoencoders
• Variational Inference with Normalizing Flows
• Recent advances and clinical applications of deep learning in medical image analysis
• Generative Modeling by Estimating Gradients of the Data Distribution
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
• Disentangling by Factorising
• A Survey on Dialogue Systems: Recent Advances and New Frontiers
• Semi-Supervised Learning with Ladder Networks
• Semi-supervised Deep Reinforcement Learning in Support of IoT and Smart City Services
• A note on the evaluation of generative models
• Enabling Cognitive Smart Cities Using Big Data and Machine Learning: Approaches and Challenges
• Variational Autoencoder for Deep Learning of Images, Labels and Captions
• Mixed supervision for surface-defect detection: from weakly to fully supervised learning
• Adversarial Autoencoders
• Improved Precision and Recall Metric for Assessing Generative Models
• BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
• Towards Biologically Plausible Deep Learning
• Learning a Probabilistic Model for Diffeomorphic Registration
• Ladder Variational Autoencoders
• An Overview of Deep Semi-Supervised Learning
• Self-training with Noisy Student improves ImageNet classification
• Conditional molecular design with deep generative models
• AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
• Winner-Take-All Autoencoders
• Unsupervised Learning via Meta-Learning
• A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark
• Invertible Residual Networks
• Deep Learning and Model Predictive Control for Self-Tuning Mode-Locked Lasers
• A Probabilistic U-Net for Segmentation of Ambiguous Images
• Federated Adversarial Domain Adaptation
• RSGAN: Face Swapping and Editing using Face and Hair Representation in Latent Spaces
• Revisiting Self-Training for Neural Sequence Generation
• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data
• Deep Learning and Knowledge-Based Methods for Computer Aided Molecular Design -- Toward a Unified Approach: State-of-the-Art and Future Directions
• Self-Training: A Survey
• Generative Image Modeling Using Spatial LSTMs
• Information Constraints on Auto-Encoding Variational Bayes
• ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling
• Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data
• Audio-visual Speech Enhancement Using Conditional Variational Auto-Encoders
• Efficient Training for Positive Unlabeled Learning
• Semi-supervised Deep Generative Modelling of Incomplete Multi-Modality Emotional Data
• In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning
• Weakly Supervised Disentanglement with Guarantees
• Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder
• DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
• Deconstructing the Ladder Network Architecture
• Lifelong Teacher-Student Network Learning
• Improving Variational Autoencoder with Deep Feature Consistent and Generative Adversarial Training
• Classification Aware Neural Topic Model and its Application on a New COVID-19 Disinformation Corpus
• A Survey of Deep Learning for Scientific Discovery
• Deep Neural Generative Model of Functional MRI Images for Psychiatric Disorder Diagnosis
• Topic-Guided Variational Autoencoders for Text Generation
• Low-Quality Training Data Only? A Robust Framework for Detecting Encrypted Malicious Network Traffic
• Constrained Bayesian Optimization for Automatic Chemical Design
• Disentangling Disentanglement in Variational Autoencoders
• Disentangling Factors of Variation Using Few Labels
• ACVAE-VC: Non-parallel many-to-many voice conversion with auxiliary classifier variational autoencoder
• Yes, we GAN: Applying Adversarial Techniques for Autonomous Driving
• Distribution Matching in Variational Inference
• Learning Causal Semantic Representation for Out-of-Distribution Prediction
• IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks
• From the digital data revolution to digital health and digital economy toward a digital society: Pervasiveness of Artificial Intelligence
• Data Generation as Sequential Decision Making
• Uncertainty-aware Self-training for Text Classification with Few Labels
• Semi-Automatic Data Annotation guided by Feature Space Projection
• A Flexible Generative Framework for Graph-based Semi-supervised Learning
• Image Captioning with Very Scarce Supervised Data: Adversarial Semi-Supervised Learning Approach
• Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition
• DVAE++: Discrete Variational Autoencoders with Overlapping Transformations
• Recovering compressed images for automatic crack segmentation using generative models
• Unsupervised Domain Adaptation for Object Detection via Cross-Domain Semi-Supervised Learning
• SEALion: a Framework for Neural Network Inference on Encrypted Data
• Graphite: Iterative Generative Modeling of Graphs
• Grale: Designing Networks for Graph Learning
• Generalizing Skills with Semi-Supervised Reinforcement Learning
• Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks
• Neural Network Attributions: A Causal Perspective
• Adversarial Autoencoders with Constant-Curvature Latent Manifolds
• Semi-Supervised Learning of Bearing Anomaly Detection via Deep Variational Autoencoders
• Deep Semi-Supervised Anomaly Detection
• Focal Frequency Loss for Image Reconstruction and Synthesis
• Semi-supervised source localization with deep generative modeling
• Meta-Consolidation for Continual Learning
• Continual Learning with Adaptive Weights (CLAW)
• A probabilistic generative model for semi-supervised training of coarse-grained surrogates and enforcing physical constraints through virtual observables
• A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities
• GAR: An efficient and scalable Graph-based Activity Regularization for semi-supervised learning
• Learning Generative Models across Incomparable Spaces
• 3DMolNet: A Generative Network for Molecular Structures
• Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations
• RLAD: Time Series Anomaly Detection through Reinforcement Learning and Active Learning
• Max-margin Deep Generative Models
• Decision-Making with Auto-Encoding Variational Bayes
• Learning to Generate Chairs, Tables and Cars with Convolutional Networks
• Learning Kernel for Conditional Moment-Matching Discrepancy-based Image Classification
• Semi-Conditional Normalizing Flows for Semi-Supervised Learning
• Relaxing Bijectivity Constraints with Continuously Indexed Normalising Flows
• Hebbian Semi-Supervised Learning in a Sample Efficiency Setting
• Label Efficient Semi-Supervised Learning via Graph Filtering
• Separating physically distinct mechanisms in complex infrared plasmonic nanostructures via machine learning enhanced electron energy loss spectroscopy
• Entanglement-based quantum deep learning
• Reducing Data Complexity using Autoencoders with Class-informed Loss Functions
• Fast Second-Order Stochastic Backpropagation for Variational Inference
• Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradients
• Gaussian Process Prior Variational Autoencoders
• Learning Disentangled Representations with Reference-Based Variational Autoencoders
• Tree Annotations in LiDAR Data Using Point Densities and Convolutional Neural Networks
• Semi-supervised Text Regression with Conditional Generative Adversarial Networks
• Guided Variational Autoencoder for Speech Enhancement With a Supervised Classifier
• Omni-sourced Webly-supervised Learning for Video Recognition
• Improving novelty detection with generative adversarial networks on hand gesture data
• Learning to Interpret Satellite Images in Global Scale Using Wikipedia
• Deep Learning in Protein Structural Modeling and Design
• Robust Learning Under Label Noise With Iterative Noise-Filtering
• Semi-Supervised Generative Modeling for Controllable Speech Synthesis
• A Probabilistic Framework for Deep Learning
• Image Generation From Small Datasets via Batch Statistics Adaptation
• Learning Interpretable Deep Disentangled Neural Networks for Hyperspectral Unmixing
• Dual Swap Disentangling
• Semi-Supervised StyleGAN for Disentanglement Learning
• Learning to Generate with Memory
• Disentanglement Learning for Variational Autoencoders Applied to Audio-Visual Speech Enhancement
• Hidden Talents of the Variational Autoencoder
• Multi-Task Attention-Based Semi-Supervised Learning for Medical Image Segmentation
• Effect of The Latent Structure on Clustering with GANs
• Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling
• Rethinking Assumptions in Deep Anomaly Detection
• Study of Deep Generative Models for Inorganic Chemical Compositions
• MAE: Mutual Posterior-Divergence Regularization for Variational AutoEncoders
• Multi-Modal Anomaly Detection for Unstructured and Uncertain Environments
• Deep Metric Transfer for Label Propagation with Limited Annotated Data
• An Identifiable Double VAE For Disentangled Representations
• Unsupervised Dialog Structure Learning
• Deep Partition Aggregation: Provable Defense against General Poisoning Attacks
• Score-based Generative Modeling in Latent Space
• Aggregated Multi-GANs for Controlled 3D Human Motion Prediction
• Unsupervised Category Discovery via Looped Deep Pseudo-Task Optimization Using a Large Scale Radiology Image Database
• Variational AutoEncoder For Regression: Application to Brain Aging Analysis
• Deep Co-Space: Sample Mining Across Feature Transformation for Semi-Supervised Learning
• From Ansätze to Z-gates: a NASA View of Quantum Computing
• Unsupervised Discovery of 3D Physical Objects from Video
• Sentiment-oriented Transformer-based Variational Autoencoder Network for Live Video Commenting
• Attribute Guided Unpaired Image-to-Image Translation with Semi-supervised Learning
• Bayesian Optimization in Variational Latent Spaces with Dynamic Compression
• A Tutorial on Sparse Gaussian Processes and Variational Inference
• Efficient Marginalization of Discrete and Structured Latent Variables via Sparsity
• Generalization Guarantees for Imitation Learning
• vONTSS: vMF based semi-supervised neural topic modeling with optimal transport
• Asynchronous Distributed Learning from Constraints
• Identifying Invariant Texture Violation for Robust Deepfake Detection
• Adversarial Approximate Inference for Speech to Electroglottograph Conversion
• Information Maximizing Visual Question Generation
• A Contrastive Learning Approach for Training Variational Autoencoder Priors
• EVA: Generating Longitudinal Electronic Health Records Using Conditional Variational Autoencoders
• ChartPointFlow for Topology-Aware 3D Point Cloud Generation
• EC-GAN: Low-Sample Classification using Semi-Supervised Algorithms and GANs
• Synthetic Data -- Anonymisation Groundhog Day
• Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations
• ESAD: End-to-end Deep Semi-supervised Anomaly Detection
• An Attention-Based System for Damage Assessment Using Satellite Imagery
• Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning
• Context-aware learning for generative models
• Disentangling and Learning Robust Representations with Natural Clustering
• Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning
• Self-supervised Semi-supervised Learning for Data Labeling and Quality Evaluation
• A Three-Stage Self-Training Framework for Semi-Supervised Semantic Segmentation
• Real-time Adaptation for Condition Monitoring Signal Prediction using Label-aware Neural Processes
• VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
• Probabilistic Programming with Programmable Variational Inference
• Generative Model with Coordinate Metric Learning for Object Recognition Based on 3D Models
• Label-Noise Robust Generative Adversarial Networks
• Factorized Discriminant Analysis for Genetic Signatures of Neuronal Phenotypes
• Improving Positive Unlabeled Learning: Practical AUL Estimation and New Training Method for Extremely Imbalanced Data Sets
• Semi-supervised Learning for Quantification of Pulmonary Edema in Chest X-Ray Images
• Unsupervised Joint Mining of Deep Features and Image Labels for Large-scale Radiology Image Categorization and Scene Recognition
• A Bayesian Perspective of Convolutional Neural Networks through a Deconvolutional Generative Model
• Semi-Supervised Learning with Normalizing Flows
• Memory-Efficient Semi-Supervised Continual Learning: The World is its Own Replay Buffer
• The Future of Data Analysis in the Neurosciences
• A Variational Autoencoder for Probabilistic Non-Negative Matrix Factorisation
• Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learning
• TSIT: A Simple and Versatile Framework for Image-to-Image Translation
• Multi-objects Generation with Amortized Structural Regularization
• Boundary-Refined Prototype Generation: A General End-to-End Paradigm for Semi-Supervised Semantic Segmentation
• Unpaired Pose Guided Human Image Generation
• CoDeGAN: Contrastive Disentanglement for Generative Adversarial Network
• Adversarial Self-Supervised Learning for Semi-Supervised 3D Action Recognition
• Intact-VAE: Estimating Treatment Effects under Unobserved Confounding
• Multimodal Variational Autoencoders for Semi-Supervised Learning: In Defense of Product-of-Experts
• Automatic Relevance Determination For Deep Generative Models
• Robust Imitation Learning from Noisy Demonstrations
• Capturing Label Characteristics in VAEs
• Continuous Transfer Learning with Label-informed Distribution Alignment
• Protecting Classifiers From Attacks
• Training Normalizing Flows with the Information Bottleneck for Competitive Generative Classification
• How Does GAN-based Semi-supervised Learning Work?
• Lung CT Imaging Sign Classification through Deep Learning on Small Data
• Probabilistic Video Generation using Holistic Attribute Control
• Zero-Shot Learning with Knowledge Enhanced Visual Semantic Embeddings
• Scalable Deep Unsupervised Clustering with Concrete GMVAEs
• Learning Disentangled Representations of Timbre and Pitch for Musical Instrument Sounds Using Gaussian Mixture Variational Autoencoders
• Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets
• DynamicVAE: Decoupling Reconstruction Error and Disentangled Representation Learning
• Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder
• Semi-Supervised Empirical Risk Minimization: Using unlabeled data to improve prediction
• A Wasserstein GAN model with the total variational regularization
• Unsupervised Data Imputation via Variational Inference of Deep Subspaces
• Modality Conversion of Handwritten Patterns by Cross Variational Autoencoders
• Do sequence-to-sequence VAEs learn global features of sentences?
• SHOT-VAE: Semi-supervised Deep Generative Models With Label-aware ELBO Approximations
• Independence Promoted Graph Disentangled Networks
• An Improved Semi-Supervised VAE for Learning Disentangled Representations
• Blinder: End-to-end Privacy Protection in Sensing Systems via Personalized Federated Learning
• Towards Disentangled Representations for Human Retargeting by Multi-view Learning
• Target-Embedding Autoencoders for Supervised Representation Learning
• Streaming Adaptive Nonparametric Variational Autoencoder
• Latent feature disentanglement for 3D meshes
• dMelodies: A Music Dataset for Disentanglement Learning
• Improving Disentangled Representation Learning with the Beta Bernoulli Process
• Affine Variational Autoencoders: An Efficient Approach for Improving Generalization and Robustness to Distribution Shift
• Task-agnostic Continual Learning with Hybrid Probabilistic Models
• Energy-Based Models with Applications to Speech and Language Processing
• Manifold Interpolation for Large-Scale Multi-Objective Optimization via Generative Adversarial Networks
• Universal Design Methodology for Printable Microstructural Materials via a New Deep Generative Learning Model: Application to a Piezocomposite
• Multi-modal data generation with a deep metric variational autoencoder
• Unsupervised dynamic modeling of medical image transformation
• Bi-level Score Matching for Learning Energy-based Latent Variable Models
• GRADE: Graph Dynamic Embedding
• AMENet: Attentive Maps Encoder Network for Trajectory Prediction
• Semi-supervised Disentanglement with Independent Vector Variational Autoencoders
• Few Is Enough: Task-Augmented Active Meta-Learning for Brain Cell Classification
• An interpretable semi-supervised classifier using two different strategies for amended self-labeling
• Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoders
• Semi-Supervised Variational Autoencoder for Survival Prediction
• Truncated Gaussian-Mixture Variational AutoEncoder
• Adversarial Disentanglement with Grouped Observations
• Class-Distinct and Class-Mutual Image Generation with GANs
• Mode matching in GANs through latent space learning and inversion
• Cross-Domain Adversarial Auto-Encoder
• Crossmodal Voice Conversion
• Dueling Decoders: Regularizing Variational Autoencoder Latent Spaces
• Semi-Supervised Learning by Disentangling and Self-Ensembling Over Stochastic Latent Space
• Generative-Discriminative Complementary Learning
• Cascading Denoising Auto-Encoder as a Deep Directed Generative Model
• Speak Like a Dog: Human to Non-human creature Voice Conversion
• Understanding and Improving Virtual Adversarial Training
• Exploring Semi-supervised Variational Autoencoders for Biomedical Relation Extraction
• Simple, Scalable, and Stable Variational Deep Clustering
• DmifNet:3D Shape Reconstruction Based on Dynamic Multi-Branch Information Fusion
• High-dimensional Assisted Generative Model for Color Image Restoration
• On the Transferability of VAE Embeddings using Relational Knowledge with Semi-Supervision
• Likelihood Contribution based Multi-scale Architecture for Generative Flows
• A Meta-Bayesian Model of Intentional Visual Search
• Semi-supervised Sequential Generative Models
• Condition-Transforming Variational AutoEncoder for Conversation Response Generation
• Label Prediction Framework for Semi-Supervised Cross-Modal Retrieval
• Dispersed Exponential Family Mixture VAEs for Interpretable Text Generation
• WeLa-VAE: Learning Alternative Disentangled Representations Using Weak Labels
• Deep Unsupervised Clustering with Clustered Generator Model
• Memory and attention in deep learning
• Representation Learning for Words and Entities
• On The Distribution of Penultimate Activations of Classification Networks
• Knowledge Efficient Deep Learning for Natural Language Processing
• Combining Probabilistic Logic and Deep Learning for Self-Supervised Learning
• You Never Cluster Alone
• Learning Invariances for Interpretability using Supervised VAE
• Semi-supervised learning via Feedforward-Designed Convolutional Neural Networks
• Text Modeling with Syntax-Aware Variational Autoencoders
• Learning Signal-Agnostic Manifolds of Neural Fields
• Uncertainty-Aware Semantic Augmentation for Neural Machine Translation
• Target Speech Extraction Based on Blind Source Separation and X-vector-based Speaker Selection Trained with Data Augmentation
• Learning Conditional Invariance through Cycle Consistency
• Capturing Actionable Dynamics with Structured Latent Ordinary Differential Equations
• Cauchy-Schwarz Regularized Autoencoder
• Semi-supervised Learning of Galaxy Morphology using Equivariant Transformer Variational Autoencoders
• Learning Energy-Based Models as Generative ConvNets via Multi-grid Modeling and Sampling
• Painting Many Pasts: Synthesizing Time Lapse Videos of Paintings
• MCENET: Multi-Context Encoder Network for Homogeneous Agent Trajectory Prediction in Mixed Traffic
• Disentanglement Learning via Topology
• CRL: Class Representative Learning for Image Classification
• Learning from Demonstration with Weakly Supervised Disentanglement
• Semi-Supervised Dialogue Policy Learning via Stochastic Reward Estimation
• A New Distribution on the Simplex with Auto-Encoding Applications
• Attention-guided Quality Assessment for Automated Cryo-EM Grid Screening
• Investigating the Effect of Intraclass Variability in Temporal Ensembling
• Towards Verified Stochastic Variational Inference for Probabilistic Programs
• Manifold Regularized Discriminative Neural Networks
• $\rho$-VAE: Autoregressive parametrization of the VAE encoder
• Tackling the Problem of Limited Data and Annotations in Semantic Segmentation
• Semi-supervised voice conversion with amortized variational inference
• Density Fixing: Simple yet Effective Regularization Method based on the Class Prior
• Deep Goal-Oriented Clustering
• Contributions to Large Scale Bayesian Inference and Adversarial Machine Learning
• Semi-Supervised Deep Learning Using Improved Unsupervised Discriminant Projection
• Learning Improved Representations by Transferring Incomplete Evidence Across Heterogeneous Tasks
• Semi-supervised learning objectives as log-likelihoods in a generative model of data curation
• Coarse Grained Exponential Variational Autoencoders
• Unsupervised Diffeomorphic Surface Registration and Non-Linear Modelling
• From abstract items to latent spaces to observed data and back: Compositional Variational Auto-Encoder
• Fast MVAE: Joint separation and classification of mixed sources based on multichannel variational autoencoder with auxiliary classifier
• MirrorNet: A Deep Bayesian Approach to Reflective 2D Pose Estimation from Human Images
• Disassembling Object Representations without Labels
• Exploiting Vulnerability of Pooling in Convolutional Neural Networks by Strict Layer-Output Manipulation for Adversarial Attacks
• Geo-Spatiotemporal Features and Shape-Based Prior Knowledge for Fine-grained Imbalanced Data Classification
• EXoN: EXplainable encoder Network
• Conditioned Text Generation with Transfer for Closed-Domain Dialogue Systems
• Pseudo Siamese Network for Few-shot Intent Generation
• Estimating Disentangled Belief about Hidden State and Hidden Task for Meta-RL
• Group-based Learning of Disentangled Representations with Generalizability for Novel Contents
• Iterative multi-path tracking for video and volume segmentation with sparse point supervision
• Cause-Effect Deep Information Bottleneck For Systematically Missing Covariates
• MisMatch: Calibrated Segmentation via Consistency on Differential Morphological Feature Perturbations with Limited Labels
• $\beta$-Intact-VAE: Identifying and Estimating Causal Effects under Limited Overlap
• Variational Auto-Encoder Architectures that Excel at Causal Inference
• Inducing Functions through Reinforcement Learning without Task Specification
• Semi-supervised Acoustic Event Detection based on tri-training
• Variational latent discrete representation for time series modelling
• A Two-Step Disentanglement Method
• SepVAE: a contrastive VAE to separate pathological patterns from healthy ones
• Learning Sparsity of Representations with Discrete Latent Variables
• Action2video: Generating Videos of Human 3D Actions
• Semi-supervised Stochastic Multi-Domain Learning using Variational Inference
• Labeled Data Generation with Inexact Supervision
• Semi-supervised lane detection with Deep Hough Transform
• Preliminary study on using vector quantization latent spaces for TTS/VC systems with consistent performance
• Self-supervised self-supervision by combining deep learning and probabilistic logic
• Semi-Supervised Self-Growing Generative Adversarial Networks for Image Recognition