14 citations · 29 across the 11 of their papers we have counts for
9 papers · 1 filter
mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learning
Ying Mo, Jian Yang, Jiahao Liu +4
Cross-lingual named entity recognition (CrossNER) faces challenges stemming from uneven performance due to the scarcity of multilingual corpora, especially for non-English data. Wh…
GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model
Shicheng Tan, Weng Lam Tam, Yuanchun Wang +9
Currently, the reduction in the parameter scale of large-scale pre-trained language models (PLMs) through knowledge distillation has greatly facilitated their widespread deployment…
PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models
Zhuocheng Gong, Jiahao Liu, Qifan Wang +6
While transformer-based pre-trained language models (PLMs) have dominated a number of NLP applications, these models are heavy to deploy and expensive to use. Therefore, effectivel…
RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank
Jiduan Liu, Jiahao Liu, Qifan Wang +6
Unsupervised sentence representation learning is one of the fundamental problems in natural language processing with various downstream applications. Recently, contrastive learning…
Lifting the Curse of Capacity Gap in Distilling Language Models
Chen Zhang, Yang Yang, Jiahao Liu +4
Pretrained language models (LMs) have shown compelling performance on various downstream tasks, but unfortunately they require a tremendous amount of inference compute. Knowledge d…
Multi-task Transformer with Relation-attention and Type-attention for Named Entity Recognition
Ying Mo, Hongyin Tang, Jiahao Liu +5
Named entity recognition (NER) is an important research problem in natural language processing. There are three types of NER tasks, including flat, nested and discontinuous entity…
MiniDisc: Minimal Distillation Schedule for Language Model Compression
Chen Zhang, Yang Yang, Qifan Wang +4
Recent studies have uncovered that language model distillation is less effective when facing a large capacity gap between the teacher and the student, and introduced teacher assist…
VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation
Fuli Luo, Wei Wang, Jiahao Liu +5
Existing work in multilingual pretraining has demonstrated the potential of cross-lingual transferability by training a unified Transformer encoder for multiple languages. However,…
A Planning based Framework for Essay Generation
Bing Qin, Duyu Tang, Xinwei Geng +3
Generating an article automatically with computer program is a challenging task in artificial intelligence and natural language processing. In this paper, we target at essay genera…