1.2k citations · 2.6k across the 49 of their papers we have counts for
8 papers · 1 filter
Gradient-based Intra-attention Pruning on Pre-trained Language Models
Ziqing Yang, Yiming Cui, Xin Yao +1
Pre-trained language models achieve superior performance but are computationally expensive. Techniques such as pruning and knowledge distillation have been developed to reduce thei…
LERT: A Linguistically-motivated Pre-trained Language Model
Yiming Cui, Wanxiang Che, Shijin Wang +1
Pre-trained Language Model (PLM) has become a representative foundation model in the natural language processing field. Most PLMs are trained with linguistic-agnostic pre-training…
HIT at SemEval-2022 Task 2: Pre-trained Language Model for Idioms Detection
Zheng Chu, Ziqing Yang, Yiming Cui +2
The same multi-word expressions may have different meanings in different sentences. They can be mainly divided into two categories, which are literal meaning and idiomatic meaning.…
HFL at SemEval-2022 Task 8: A Linguistics-inspired Regression Model with Data Augmentation for Multilingual News Similarity
Zihang Xu, Ziqing Yang, Yiming Cui +1
This paper describes our system designed for SemEval-2022 Task 8: Multilingual News Article Similarity. We proposed a linguistics-inspired model trained with a few task-specific st…
TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models
Ziqing Yang, Yiming Cui, Zhigang Chen
Pre-trained language models have been prevailed in natural language processing and become the backbones of many NLP tasks, but the demands for computational resources have limited…
PERT: Pre-training BERT with Permuted Language Model
Yiming Cui, Ziqing Yang, Ting Liu
Pre-trained Language Models (PLMs) have been widely used in various natural language processing (NLP) tasks, owing to their powerful text representations trained on large-scale cor…