activity
20222026
most citedHyperMixer: An MLP-based Low Cost Alternative to Transformers

2 citations · 2 across the 5 of their papers we have counts for

collaborators
Showing 2022Show all

2 papers · 1 filter

cs.LG2022

A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck

James Henderson, Fabio Fehr

We propose a VAE for Transformers by developing a variational information bottleneck regulariser for Transformer embeddings. We formalise the embedding space of Transformer encoder…

cs.CL2022★ 2 cited

HyperMixer: An MLP-based Low Cost Alternative to Transformers

Florian Mai, Arnaud Pannatier, Fabio Fehr +4

Transformer-based architectures are the model of choice for natural language understanding, but they come at a significant cost, as they have quadratic complexity in the input leng…