activity
20202022
most citedDeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders

51 citations · 133 across the 7 of their papers we have counts for

collaborators
Showing 2022Show all

1 paper · 1 filter

cs.LG2022★ 11 cited

Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers

Rui Liu, Young Jin Kim, Alexandre Muzio +1

Sparsely activated transformers, such as Mixture of Experts (MoE), have received great interest due to their outrageous scaling capability which enables dramatical increases in mod…