Infomax Neural Joint Source-Channel Coding via Adversarial Bit Flip
arXiv:2004.01454
Abstract
Although Shannon theory states that it is asymptotically optimal to separate the source and channel coding as two independent processes, in many practical communication scenarios this decomposition is limited by the finite bit-length and computational power for decoding. Recently, neural joint source-channel coding (NECST) is proposed to sidestep this problem. While it leverages the advancements of amortized inference and deep learning to improve the encoding and decoding process, it still cannot always achieve compelling results in terms of compression and error correction performance due to the limited robustness of its learned coding networks. In this paper, motivated by the inherent connections between neural joint source-channel coding and discrete representation learning, we propose a novel regularization method called Infomax Adversarial-Bit-Flip (IABF) to improve the stability and robustness of the neural joint source-channel coding scheme. More specifically, on the encoder side, we propose to explicitly maximize the mutual information between the codeword and data; while on the decoder side, the amortized reconstruction is regularized within an adversarial framework. Extensive experiments conducted on various real-world datasets evidence that our IABF can achieve state-of-the-art performances on both compression and error correction benchmarks and outperform the baselines by a significant margin.
arXiv commentAAAI2020
References in corpus (15)
- Neural Discrete Representation Learning
- Variational image compression with a scale hyperprior
- Disentangling by Factorising
- Learning with Pseudo-Ensembles
- InfoVAE: Information Maximizing Variational Autoencoders
- Distributional Smoothing with Virtual Adversarial Training
- Learning Discrete Representations via Information Maximizing Self-Augmented Training
- Lossy Image Compression with Compressive Autoencoders
- Learning in Implicit Generative Models
- Variational inference for Monte Carlo objectives
- On distinguishability criteria for estimating generative models
- Amortized Inference Regularization
- Discrete Variational Autoencoders
- The Information Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Models
- Uncertainty Autoencoders: Learning Compressed Representations via Variational Information Maximization