7 citations · 15 across the 3 of their papers we have counts for
3 papers
cs.CL2023★ 7 cited
PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing
Xiaozhe Ren, Pingyi Zhou, Xinfan Meng +14
The scaling of large language models has greatly improved natural language understanding, generation, and reasoning. In this work, we develop a system that trained a trillion-param…
cs.CL2022★ 6 cited
Sparse Structure Search for Parameter-Efficient Tuning
Shengding Hu, Zhen Zhang, Ning Ding +4
Adapting large pre-trained models (PTMs) through fine-tuning imposes prohibitive computational and storage burdens. Recent studies of parameter-efficient tuning (PET) find that onl…
cs.CL2022★ 2 cited
HyperPELT: Unified Parameter-Efficient Language Model Tuning for Both Language and Vision-and-Language Tasks
Zhengkun Zhang, Wenya Guo, Xiaojun Meng +5
The workflow of pretraining and fine-tuning has emerged as a popular paradigm for solving various NLP and V&L (Vision-and-Language) downstream tasks. With the capacity of pretraine…