565 citations · 1.3k across the 39 of their papers we have counts for
6 papers · 1 filter
Gradient-based Intra-attention Pruning on Pre-trained Language Models
Ziqing Yang, Yiming Cui, Xin Yao +1
Pre-trained language models achieve superior performance but are computationally expensive. Techniques such as pruning and knowledge distillation have been developed to reduce thei…
LERT: A Linguistically-motivated Pre-trained Language Model
Yiming Cui, Wanxiang Che, Shijin Wang +1
Pre-trained Language Model (PLM) has become a representative foundation model in the natural language processing field. Most PLMs are trained with linguistic-agnostic pre-training…
Overview of CTC 2021: Chinese Text Correction for Native Speakers
Honghong Zhao, Baoxin Wang, Dayong Wu +3
In this paper, we present an overview of the CTC 2021, a Chinese text correction task for native speakers. We give detailed descriptions of the task definition and the data for tra…
JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem Understanding
Wayne Xin Zhao, Kun Zhou, Zheng Gong +7
This paper aims to advance the mathematical intelligence of machines by presenting the first Chinese mathematical pre-trained language model~(PLM) for effectively understanding and…
Cross-Lingual Text Classification with Multilingual Distillation and Zero-Shot-Aware Training
Ziqing Yang, Yiming Cui, Zhigang Chen +1
Multilingual pre-trained language models (MPLMs) not only can handle tasks in different languages but also exhibit surprising zero-shot cross-lingual transferability. However, MPLM…
InterHT: Knowledge Graph Embeddings by Interaction between Head and Tail Entities
Baoxin Wang, Qingye Meng, Ziyue Wang +6
Knowledge graph embedding (KGE) models learn the representation of entities and relations in knowledge graphs. Distance-based methods show promising performance on link prediction…