1.4k citations · 1.7k across the 34 of their papers we have counts for
5 papers · 1 filter
COPEN: Probing Conceptual Knowledge in Pre-trained Language Models
Hao Peng, Xiaozhi Wang, Shengding Hu +5
Conceptual knowledge is fundamental to human cognition and knowledge bases. However, existing knowledge probing works only focus on evaluating factual knowledge of pre-trained lang…
Sparse Structure Search for Parameter-Efficient Tuning
Shengding Hu, Zhen Zhang, Ning Ding +4
Adapting large pre-trained models (PTMs) through fine-tuning imposes prohibitive computational and storage burdens. Recent studies of parameter-efficient tuning (PET) find that onl…
A Roadmap for Big Model
Sha Yuan, Hanyu Zhao, Shuai Zhao +97
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm. Researchers have achieved various outcomes in the c…
Prototypical Verbalizer for Prompt-based Few-shot Tuning
Ganqu Cui, Shengding Hu, Ning Ding +2
Prompt-based tuning for pre-trained language models (PLMs) has shown its effectiveness in few-shot learning. Typically, prompt-based tuning wraps the input text into a cloze questi…
Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models
Ning Ding, Yujia Qin, Guang Yang +17
Despite the success, the process of fine-tuning large-scale PLMs brings prohibitive adaptation costs. In fact, fine-tuning all the parameters of a colossal model and retaining sepa…