A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck

James Henderson, Fabio Fehr

We propose a VAE for Transformers by developing a variational information bottleneck regulariser for Transformer embeddings. We formalise the embedding space of Transformer encoder…