311 citations · 815 across the 62 of their papers we have counts for
21 papers · 1 filter
Model Uncertainty-Aware Knowledge Amalgamation for Pre-Trained Language Models
Lei Li, Yankai Lin, Xuancheng Ren +4
As many fine-tuned pre-trained language models~(PLMs) with promising performance are generously released, investigating better ways to reuse these models is vital as it can greatly…
On Transferability of Prompt Tuning for Natural Language Processing
Yusheng Su, Xiaozhi Wang, Yujia Qin +10
Prompt tuning (PT) is a promising parameter-efficient method to utilize extremely large pre-trained language models (PLMs), which can achieve comparable performance to full-paramet…
Exploring Universal Intrinsic Task Subspace via Prompt Tuning
Yujia Qin, Xiaozhi Wang, Yusheng Su +10
Why can pre-trained language models (PLMs) learn universal representations and effectively adapt to broad NLP tasks differing a lot superficially? In this work, we empirically find…
MoEfication: Transformer Feed-forward Layers are Mixtures of Experts
Zhengyan Zhang, Yankai Lin, Zhiyuan Liu +3
Recent work has shown that feed-forward networks (FFNs) in pre-trained Transformers are a key component, storing various linguistic and factual knowledge. However, the computationa…
Feature Correlation Aggregation: on the Path to Better Graph Neural Networks
Jieming Zhou, Tong Zhang, Pengfei Fang +2
Prior to the introduction of Graph Neural Networks (GNNs), modeling and analyzing irregular data, particularly graphs, was thought to be the Achilles' heel of deep learning. The co…
Reasoning Graph Networks for Kinship Verification: from Star-shaped to Hierarchical
Wanhua Li, Jiwen Lu, Abudukelimu Wuerkaixi +2
In this paper, we investigate the problem of facial kinship verification by learning hierarchical reasoning graph networks. Conventional methods usually focus on learning discrimin…