1.4k citations · 1.7k across the 32 of their papers we have counts for
8 papers · 1 filter
Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules
Chaojun Xiao, Yuqi Luo, Wenbin Zhang +8
Pre-trained language models (PLMs) have achieved remarkable results on NLP tasks but at the expense of huge parameter sizes and the consequent computational costs. In this paper, w…
CA-LoRA: Adapting Existing LoRA for Compressed LLMs to Enable Efficient Multi-Tasking on Personal Devices
Weilin Zhao, Yuxiang Huang, Xu Han +6
Recently, there has been a demand to deploy Large Language Models (LLMs) on personal devices such as laptops and smartphones. These LLMs have different model variants when handling…
Plug-and-Play Document Modules for Pre-trained Models
Chaojun Xiao, Zhengyan Zhang, Xu Han +7
Large-scale pre-trained models (PTMs) have been widely used in document-oriented NLP tasks, such as question answering. However, the encoding-task coupling requirement results in t…
UNTER: A Unified Knowledge Interface for Enhancing Pre-trained Language Models
Deming Ye, Yankai Lin, Zhengyan Zhang +1
Recent research demonstrates that external knowledge injection can advance pre-trained language models (PLMs) in a variety of downstream NLP tasks. However, existing knowledge inje…
Emergent Modularity in Pre-trained Transformers
Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin +7
This work examines the presence of modularity in pre-trained Transformers, a feature commonly found in human brains and thought to be vital for general intelligence. In analogy to…
Plug-and-Play Knowledge Injection for Pre-trained Language Models
Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin +8
Injecting external knowledge can improve the performance of pre-trained language models (PLMs) on various downstream NLP tasks. However, massive retraining is required to deploy ne…
Bridging the Language Gap: Knowledge Injected Multilingual Question Answering
Zhichao Duan, Xiuxing Li, Zhengyan Zhang +3
Question Answering (QA) is the task of automatically answering questions posed by humans in natural languages. There are different settings to answer a question, such as abstractiv…
READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises
Chenglei Si, Zhengyan Zhang, Yingfa Chen +3
For many real-world applications, the user-generated inputs usually contain various noises due to speech recognition errors caused by linguistic variations1 or typographical errors…