2 citations · 4 across the 7 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.CL2023
Sparse Universal Transformer
Shawn Tan, Yikang Shen, Zhenfang Chen +2
The Universal Transformer (UT) is a variant of the Transformer that shares parameters across its layers. Empirical evidence shows that UTs have better compositional generalization…
cs.CL2023★ 2 cited
ModuleFormer: Modularity Emerges from Mixture-of-Experts
Yikang Shen, Zheyu Zhang, Tianyou Cao +3
Large Language Models (LLMs) have achieved remarkable results. However, existing models are expensive to train and deploy, and it is also difficult to expand their knowledge beyond…