163 citations · 753 across the 42 of their papers we have counts for
36 papers · 1 filter
Arena Learning: Build Data Flywheel for LLMs Post-training via Simulated Chatbot Arena
Haipeng Luo, Qingfeng Sun, Can Xu +6
Assessing the effectiveness of large language models (LLMs) presents substantial challenges. The method of conducting human-annotated battles in an online Chatbot Arena is a highly…
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Marah Abdin, Jyoti Aneja, Hany Awadalla +126
We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal test…
Re-Reading Improves Reasoning in Large Language Models
Xiaohan Xu, Chongyang Tao, Tao Shen +5
To enhance the reasoning capabilities of off-the-shelf Large Language Models (LLMs), we introduce a simple, yet general and effective prompting method, Re2, i.e., \textbf{Re}-\text…
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct
Haipeng Luo, Qingfeng Sun, Can Xu +8
Large language models (LLMs), such as GPT-4, have shown remarkable performance in natural language processing (NLP) tasks, including challenging mathematical reasoning. However, mo…
Investigating the Learning Behaviour of In-context Learning: A Comparison with Supervised Learning
Xindi Wang, Yufei Wang, Can Xu +6
Large language models (LLMs) have shown remarkable capacity for in-context learning (ICL), where learning a new task from just a few training examples is done without being explici…
WizardCoder: Empowering Code Large Language Models with Evol-Instruct
Ziyang Luo, Can Xu, Pu Zhao +7
Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. However, most existing models are solely pre-trained on…
Augmented Large Language Models with Parametric Knowledge Guiding
Ziyang Luo, Can Xu, Pu Zhao +5
Large Language Models (LLMs) have significantly advanced natural language processing (NLP) with their impressive language understanding and generation capabilities. However, their…
Synergistic Interplay between Search and Large Language Models for Information Retrieval
Jiazhan Feng, Chongyang Tao, Xiubo Geng +5
Information retrieval (IR) plays a crucial role in locating relevant resources from vast amounts of data, and its applications have evolved from traditional knowledge bases to mode…
WizardLM: Empowering large pre-trained language models to follow complex instructions
Can Xu, Qingfeng Sun, Kai Zheng +6
Training large language models (LLMs) with open-domain instruction following data brings colossal success. However, manually creating such instruction data is very time-consuming a…
MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation
Jiazhan Feng, Qingfeng Sun, Can Xu +5
Responding with multi-modal content has been recognized as an essential capability for an intelligent conversational agent. In this paper, we introduce the MMDialog dataset to bett…
Adam: Dense Retrieval Distillation with Adaptive Dark Examples
Chongyang Tao, Chang Liu, Tao Shen +4
To improve the performance of the dual-encoder retriever, one effective approach is knowledge distillation from the cross-encoder ranker. Existing works construct the candidate pas…
LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval
Kai Zhang, Chongyang Tao, Tao Shen +4
Retrieval models based on dense representations in semantic space have become an indispensable branch for first-stage retrieval. These retrievers benefit from surging advances in r…
Stylized Knowledge-Grounded Dialogue Generation via Disentangled Template Rewriting
Qingfeng Sun, Can Xu, Huang Hu +6
Current Knowledge-Grounded Dialogue Generation (KDG) models specialize in producing rational and factual responses. However, to establish long-term relationships with users, the KD…
PromDA: Prompt-based Data Augmentation for Low-Resource NLU Tasks
Yufei Wang, Can Xu, Qingfeng Sun +4
This paper focuses on the Data Augmentation for low-resource Natural Language Understanding (NLU) tasks. We propose Prompt-based D}ata Augmentation model (PromDA) which only trains…
TegTok: Augmenting Text Generation via Task-specific and Open-world Knowledge
Chao-Hong Tan, Jia-Chen Gu, Chongyang Tao +5
Generating natural and informative texts has been a long-standing problem in NLP. Much effort has been dedicated into incorporating pre-trained language models (PLMs) with various…
PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddings
Qiyu Wu, Chongyang Tao, Tao Shen +3
Learning sentence embeddings in an unsupervised manner is fundamental in natural language processing. Recent common practice is to couple pre-trained language models with unsupervi…
Small Changes Make Big Differences: Improving Multi-turn Response Selection in Dialogue Systems via Fine-Grained Contrastive Learning
Yuntao Li, Can Xu, Huang Hu +3
Retrieve-based dialogue response selection aims to find a proper response from a candidate set given a multi-turn context. Pre-trained language models (PLMs) based methods have yie…
Multimodal Dialogue Response Generation
Qingfeng Sun, Yujing Wang, Can Xu +7
Responsing with image has been recognized as an important capability for an intelligent conversational agent. Yet existing works only focus on exploring the multimodal dialogue mod…
Neural Rule-Execution Tracking Machine For Transformer-Based Text Generation
Yufei Wang, Can Xu, Huang Hu +5
Sequence-to-Sequence (S2S) neural text generation models, especially the pre-trained ones (e.g., BART and T5), have exhibited compelling performance on various natural language gen…
ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation
Weizhen Qi, Yeyun Gong, Yu Yan +9
Now, the pre-training technique is ubiquitous in natural language processing field. ProphetNet is a pre-training based natural language generation method which shows powerful perfo…
Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues?
Yufan Zhao, Wei Wu, Can Xu
We study knowledge-grounded dialogue generation with pre-trained language models. Instead of pursuing new state-of-the-art on benchmarks, we try to understand if the knowledge stor…
Knowledge-Grounded Dialogue Generation with Pre-trained Language Models
Xueliang Zhao, Wei Wu, Can Xu +3
We study knowledge-grounded dialogue generation with pre-trained language models. To leverage the redundant external knowledge under capacity constraint, we propose equipping respo…
StyleDGPT: Stylized Response Generation with Pre-trained Language Models
Ze Yang, Wei Wu, Can Xu +5
Generating responses following a desired style has great potentials to extend applications of open-domain dialogue systems, yet is refrained by lacking of parallel data for trainin…
Zero-Resource Knowledge-Grounded Dialogue Generation
Linxiao Li, Can Xu, Wei Wu +3
While neural conversation models have shown great potentials towards generating informative and engaging responses via introducing external knowledge, learning such a model often r…