123 citations · 495 across the 30 of their papers we have counts for
30 papers
Investigating the Learning Behaviour of In-context Learning: A Comparison with Supervised Learning
Xindi Wang, Yufei Wang, Can Xu +6
Large language models (LLMs) have shown remarkable capacity for in-context learning (ICL), where learning a new task from just a few training examples is done without being explici…
WizardCoder: Empowering Code Large Language Models with Evol-Instruct
Ziyang Luo, Can Xu, Pu Zhao +7
Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. However, most existing models are solely pre-trained on…
Augmented Large Language Models with Parametric Knowledge Guiding
Ziyang Luo, Can Xu, Pu Zhao +5
Large Language Models (LLMs) have significantly advanced natural language processing (NLP) with their impressive language understanding and generation capabilities. However, their…
Synergistic Interplay between Search and Large Language Models for Information Retrieval
Jiazhan Feng, Chongyang Tao, Xiubo Geng +5
Information retrieval (IR) plays a crucial role in locating relevant resources from vast amounts of data, and its applications have evolved from traditional knowledge bases to mode…
Inference with Reference: Lossless Acceleration of Large Language Models
Nan Yang, Tao Ge, Liang Wang +5
We propose LLMA, an LLM accelerator to losslessly speed up Large Language Model (LLM) inference with references. LLMA is motivated by the observation that there are abundant identi…
Large Language Models are Strong Zero-Shot Retriever
Tao Shen, Guodong Long, Xiubo Geng +3
In this work, we propose a simple method that applies a large language model (LLM) to large-scale retrieval in zero-shot scenarios. Our method, the Language language model as Retri…
WizardLM: Empowering large pre-trained language models to follow complex instructions
Can Xu, Qingfeng Sun, Kai Zheng +6
Training large language models (LLMs) with open-domain instruction following data brings colossal success. However, manually creating such instruction data is very time-consuming a…
LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Retrieval
Ziyang luo, Pu Zhao, Can Xu +6
Image-text retrieval (ITR) is a task to retrieve the relevant images/texts, given the query from another modality. The conventional dense retrieval paradigm relies on encoding imag…
Fine-Grained Distillation for Long Document Retrieval
Yucheng Zhou, Tao Shen, Xiubo Geng +4
Long document retrieval aims to fetch query-relevant documents from a large-scale collection, where knowledge distillation has become de facto to improve a retriever by mimicking a…
Adam: Dense Retrieval Distillation with Adaptive Dark Examples
Chongyang Tao, Chang Liu, Tao Shen +4
To improve the performance of the dual-encoder retriever, one effective approach is knowledge distillation from the cross-encoder ranker. Existing works construct the candidate pas…
Text Embeddings by Weakly-Supervised Contrastive Pre-training
Liang Wang, Nan Yang, Xiaolong Huang +5
This paper presents E5, a family of state-of-the-art text embeddings that transfer well to a wide range of tasks. The model is trained in a contrastive manner with weak supervision…
VATLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learning
Qiushi Zhu, Long Zhou, Ziqiang Zhang +7
Although speech is a simple and effective way for humans to communicate with the outside world, a more realistic speech interaction contains multimodal information, e.g., vision, t…
LexMAE: Lexicon-Bottlenecked Pretraining for Large-Scale Retrieval
Tao Shen, Xiubo Geng, Chongyang Tao +5
In large-scale retrieval, the lexicon-weighting paradigm, learning weighted sparse representations in vocabulary space, has shown promising results with high quality and low latenc…
LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval
Kai Zhang, Chongyang Tao, Tao Shen +4
Retrieval models based on dense representations in semantic space have become an indispensable branch for first-stage retrieval. These retrievers benefit from surging advances in r…
Towards Robust Ranker for Text Retrieval
Yucheng Zhou, Tao Shen, Xiubo Geng +5
A ranker plays an indispensable role in the de facto 'retrieval & rerank' pipeline, but its training still lags behind -- learning from moderate negatives or/and serving as an auxi…
Task-Specific Expert Pruning for Sparse Mixture-of-Experts
Tianyu Chen, Shaohan Huang, Yuan Xie +5
The sparse Mixture-of-Experts (MoE) model is powerful for large-scale pre-training and has achieved promising results due to its model capacity. However, with trillions of paramete…
KnowDA: All-in-One Knowledge Mixture Model for Data Augmentation in Low-Resource NLP
Yufei Wang, Jiayi Zheng, Can Xu +4
This paper focuses on the data augmentation for low-resource NLP tasks where the training set is limited. The existing solutions either leverage task-independent heuristic rules (e…
UnifieR: A Unified Retriever for Large-Scale Retrieval
Tao Shen, Xiubo Geng, Chongyang Tao +4
Large-scale retrieval is to recall relevant documents from a huge collection given a query. It relies on representation learning to embed documents and queries into a common semant…
Stylized Knowledge-Grounded Dialogue Generation via Disentangled Template Rewriting
Qingfeng Sun, Can Xu, Huang Hu +6
Current Knowledge-Grounded Dialogue Generation (KDG) models specialize in producing rational and factual responses. However, to establish long-term relationships with users, the KD…
FORCE: A Framework of Rule-Based Conversational Recommender System
Jun Quan, Ze Wei, Qiang Gan +11
The conversational recommender systems (CRSs) have received extensive attention in recent years. However, most of the existing works focus on various deep learning models, which ar…
PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks
Yufei Wang, Can Xu, Qingfeng Sun +4
This paper focuses on the Data Augmentation for low-resource Natural Language Understanding (NLU) tasks. We propose Prompt-based D}ata Augmentation model (PromDA) which only trains…
TegTok: Augmenting Text Generation via Task-specific and Open-world Knowledge
Chao-Hong Tan, Jia-Chen Gu, Chongyang Tao +5
Generating natural and informative texts has been a long-standing problem in NLP. Much effort has been dedicated into incorporating pre-trained language models (PLMs) with various…
HeterMPC: A Heterogeneous Graph Neural Network for Response Generation in Multi-Party Conversations
Jia-Chen Gu, Chao-Hong Tan, Chongyang Tao +4
Recently, various response generation models for two-party conversations have achieved impressive improvements, but less effort has been paid to multi-party conversations (MPCs) wh…
ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification
Yucheng Zhou, Tao Shen, Xiubo Geng +2
Generating new events given context with correlated ones plays a crucial role in many event-centric reasoning tasks. Existing works either limit their scope to specific scenarios o…