115 citations · 613 across the 46 of their papers we have counts for
35 papers · 1 filter
Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning
Zhen-Ru Zhang, Chuanqi Tan, Haiyang Xu +3
Fine-tuning large pre-trained language models on various downstream tasks with whole parameters is prohibitively expensive. Hence, Parameter-efficient fine-tuning has attracted att…
VECO 2.0: Cross-lingual Language Model Pre-training with Multi-granularity Contrastive Learning
Zhen-Ru Zhang, Chuanqi Tan, Songfang Huang +1
Recent studies have demonstrated the potential of cross-lingual transferability by training a unified Transformer encoder for multiple languages. In addition to involving the maske…
RRHF: Rank Responses to Align Language Models with Human Feedback without tears
Zheng Yuan, Hongyi Yuan, Chuanqi Tan +3
Reinforcement Learning from Human Feedback (RLHF) facilitates the alignment of large language models with human preferences, significantly enhancing the quality of interactions bet…
Revisiting Automatic Question Summarization Evaluation in the Biomedical Domain
Hongyi Yuan, Yaoyun Zhang, Fei Huang +1
Automatic evaluation metrics have been facilitating the rapid development of automatic summarization methods by providing instant and fair assessments of the quality of summaries.…
How well do Large Language Models perform in Arithmetic tasks?
Zheng Yuan, Hongyi Yuan, Chuanqi Tan +2
Large language models have emerged abilities including chain-of-thought to answer math word problems step by step. Solving math word problems not only requires abilities to disasse…
SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers
Hongyi Yuan, Zheng Yuan, Chuanqi Tan +2
Diffusion model, a new generative modelling paradigm, has achieved great success in image, audio, and video generation. However, considering the discrete categorical nature of text…
HyPe: Better Pre-trained Language Model Fine-tuning with Hidden Representation Perturbation
Hongyi Yuan, Zheng Yuan, Chuanqi Tan +2
Language models with the Transformers structure have shown great performance in natural language processing. However, there still poses problems when fine-tuning pre-trained langua…
SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition
Jianing Wang, Chengcheng Han, Chengyu Wang +5
Few-shot Named Entity Recognition (NER) aims to identify named entities with very little annotated data. Previous methods solve this problem based on token-wise classification, whi…
A Survey on Open Information Extraction from Rule-based Model to Large Language Model
Pai Liu, Wenyang Gao, Wenjie Dong +7
Open Information Extraction (OpenIE) represents a crucial NLP task aimed at deriving structured information from unstructured text, unrestricted by relation type or domain. This su…
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections
Chenliang Li, Haiyang Xu, Junfeng Tian +12
Large-scale pretrained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstrea…
Towards Unified Prompt Tuning for Few-shot Text Classification
Jianing Wang, Chengyu Wang, Fuli Luo +6
Prompt-based fine-tuning has boosted the performance of Pre-trained Language Models (PLMs) on few-shot text classification by employing task-specific prompts. Yet, PLMs are unfamil…
Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiency
Yanyang Li, Fuli Luo, Runxin Xu +3
Structured pruning has been extensively studied on monolingual pre-trained language models and is yet to be fully evaluated on their multilingual counterparts. This work investigat…
Contrastive Demonstration Tuning for Pre-trained Language Models
Xiaozhuan Liang, Ningyu Zhang, Siyuan Cheng +3
Pretrained language models can be effectively stimulated by textual prompts or demonstrations, especially in low-data scenarios. Recent works have focused on automatically searchin…
Making Pre-trained Language Models End-to-end Few-shot Learners with Contrastive Prompt Tuning
Ziyun Xu, Chengyu Wang, Minghui Qiu +4
Pre-trained Language Models (PLMs) have achieved remarkable performance for various language understanding tasks in IR systems, which require the fine-tuning process based on label…
Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Coding
Zheng Yuan, Chuanqi Tan, Songfang Huang
Automatic ICD coding is defined as assigning disease codes to electronic medical records (EMRs). Existing methods usually apply label attention with code representations to match r…
From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression
Runxin Xu, Fuli Luo, Chengyu Wang +4
Pre-trained Language Models (PLMs) have achieved great success in various Natural Language Processing (NLP) tasks under the pre-training and fine-tuning paradigm. With large quanti…
LOGEN: Few-shot Logical Knowledge-Conditioned Text Generation with Self-training
Shumin Deng, Jiacheng Yang, Hongbin Ye +6
Natural language generation from structured data mainly focuses on surface-level descriptions, suffering from uncontrollable content selection and low fidelity. Previous works leve…
Achieving Human Parity on Visual Question Answering
Ming Yan, Haiyang Xu, Chenliang Li +14
The Visual Question Answering (VQA) task utilizes both visual image and language analysis to answer a textual question with respect to an image. It has been a popular research topi…
A Chinese Multi-type Complex Questions Answering Dataset over Wikidata
Jianyun Zou, Min Yang, Lichao Zhang +8
Complex Knowledge Base Question Answering is a popular area of research in the past decade. Recent public datasets have led to encouraging results in this field, but are mostly lim…
Fusing Heterogeneous Factors with Triaffine Mechanism for Nested Named Entity Recognition
Zheng Yuan, Chuanqi Tan, Songfang Huang +1
Nested entities are observed in many domains due to their compositionality, which cannot be easily recognized by the widely-used sequence labeling framework. A natural solution is…
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning
Runxin Xu, Fuli Luo, Zhiyuan Zhang +4
Recent pretrained language models extend from millions to billions of parameters. Thus the need to fine-tune an extremely large pretrained model with a limited training corpus aris…
Lattice-BERT: Leveraging Multi-Granularity Representations in Chinese Pre-trained Language Models
Yuxuan Lai, Yijia Liu, Yansong Feng +2
Chinese pre-trained language models usually process text as a sequence of characters, while ignoring more coarse granularity, e.g., words. In this work, we propose a novel pre-trai…
StructuralLM: Structural Pre-training for Form Understanding
Chenliang Li, Bin Bi, Ming Yan +4
Large pre-trained language models achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, they almost exclusively focus on text-only representation, whil…
Improving Biomedical Pretrained Language Models with Knowledge
Zheng Yuan, Yijia Liu, Chuanqi Tan +2
Pretrained language models have shown success in many natural language processing tasks. Many works explore incorporating knowledge into language models. In the biomedical domain,…