Multi-Task Variational Information Bottleneck

Abstract

Multi-task learning (MTL) is an important subject in machine learning and artificial intelligence. Its applications to computer vision, signal processing, and speech recognition are ubiquitous. Although this subject has attracted considerable attention recently, the performance and robustness of the existing models to different tasks have not been well balanced. This article proposes an MTL model based on the architecture of the variational information bottleneck (VIB), which can provide a more effective latent representation of the input features for the downstream tasks. Extensive observations on three public data sets under adversarial attacks show that the proposed model is competitive to the state-of-the-art algorithms concerning the prediction accuracy. Experimental results suggest that combining the VIB and the task-dependent uncertainties is a very effective way to abstract valid information from the input features for accomplishing multiple tasks.

arXiv comment10 pages

References in corpus (23)

• Auto-Encoding Variational Bayes
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• An Overview of Multi-Task Learning in Deep Neural Networks
• The information bottleneck method
• Tutorial on Variational Autoencoders
• Weight Uncertainty in Neural Networks
• A Survey on Multi-Task Learning
• Variational Dropout and the Local Reparameterization Trick
• Recent Advances in Autoencoder-Based Representation Learning
• Variational Dropout Sparsifies Deep Neural Networks
• Multi-Task Learning as Multi-Objective Optimization
• Understanding disentangling in $\beta$-VAE
• Auxiliary Deep Generative Models
• Which Tasks Should Be Learned Together in Multi-task Learning?
• Learning Multiple Tasks with Multilinear Relationship Networks
• Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
• Compressing Neural Networks using the Variational Information Bottleneck
• Learning Robust Representations via Multi-View Information Bottleneck
• Pareto Multi-Task Learning
• Phase Transitions for the Information Bottleneck in Representation Learning
• The Deep Weight Prior
• The Variational Bandwidth Bottleneck: Stochastic Evaluation on an Information Budget
• Multi-objects Generation with Amortized Structural Regularization

Cited by in corpus (2)

• Compressed Hierarchical Representations for Multi-Task Learning and Task Clustering
• Multi-Task Neural Processes