699 citations · 3.7k across the 65 of their papers we have counts for
18 papers · 1 filter
Towards Building the Federated GPT: Federated Instruction Tuning
Jianyi Zhang, Saeed Vahidian, Martin Kuo +6
While "instruction-tuned" generative large language models (LLMs) have demonstrated an impressive ability to generalize to new tasks, the training phases heavily rely on large amou…
Instruction Tuning with GPT-4
Baolin Peng, Chunyuan Li, Pengcheng He +2
Prior work has shown that finetuning large language models (LLMs) using machine-generated instruction-following data enables such models to achieve remarkable zero-shot capabilitie…
STT: Soft Template Tuning for Few-Shot Adaptation
Ping Yu, Wei Wang, Chunyuan Li +3
Prompt tuning has been an extremely effective tool to adapt a pre-trained model to downstream tasks. However, standard prompt-based methods mainly consider the case of sufficient d…
SYNERGY: Building Task Bots at Scale Using Symbolic Knowledge and Machine Teaching
Baolin Peng, Chunyuan Li, Zhu Zhang +3
In this paper we explore the use of symbolic knowledge and machine teaching to reduce human data labeling efforts in building neural task bots. We propose SYNERGY, a hybrid learnin…
SDA: Improving Text Generation with Self Data Augmentation
Ping Yu, Ruiyi Zhang, Yang Zhao +3
Data augmentation has been widely used to improve deep neural networks in many research fields, such as computer vision. However, less work has been done in the context of text, pa…
Few-Shot Named Entity Recognition: A Comprehensive Study
Jiaxin Huang, Chunyuan Li, Krishan Subudhi +6
This paper presents a comprehensive study to efficiently build named entity recognition (NER) systems when a small number of in-domain labeled data is available. Based upon recent…
Improving Text Generation with Student-Forcing Optimal Transport
Guoyin Wang, Chunyuan Li, Jianqiao Li +10
Neural language models are often trained with maximum likelihood estimation (MLE), where the next word is generated conditioned on the ground-truth word tokens. During testing, how…
SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching
Baolin Peng, Chunyuan Li, Jinchao Li +3
We present a new method SOLOIST that uses transfer learning and machine teaching to build task bots at scale. We parameterize classical modular task-oriented dialog systems using a…
POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training
Yizhe Zhang, Guoyin Wang, Chunyuan Li +3
Large-scale pre-trained language models, such as BERT and GPT-2, have achieved excellent performance in language representation learning and free-form text generation. However, the…
Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space
Chunyuan Li, Xiang Gao, Yuan Li +4
When trained effectively, the Variational Autoencoder (VAE) can be both a powerful generative model and an effective representation learning framework for natural language. In this…
Multi-View Learning for Vision-and-Language Navigation
Qiaolin Xia, Xiujun Li, Chunyuan Li +5
Learning to navigate in a visual environment following natural language instructions is a challenging task because natural language instructions are highly variable, ambiguous, and…
Few-shot Natural Language Generation for Task-Oriented Dialog
Baolin Peng, Chenguang Zhu, Chunyuan Li +4
As a crucial component in task-oriented dialog systems, the Natural Language Generation (NLG) module converts a dialog act represented in a semantic form into a response in natural…
Robust Navigation with Language Pretraining and Stochastic Sampling
Xiujun Li, Chunyuan Li, Qiaolin Xia +5
Core to the vision-and-language navigation (VLN) challenge is building robust instruction representations and action decoding schemes, which can generalize well to previously unsee…
DoubleTransfer at MEDIQA 2019: Multi-Source Transfer Learning for Natural Language Understanding in the Medical Domain
Yichong Xu, Xiaodong Liu, Chunyuan Li +2
This paper describes our competing system to enter the MEDIQA-2019 competition. We use a multi-source transfer learning approach to transfer the knowledge from MT-DNN and SciBERT t…
Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms
Dinghan Shen, Guoyin Wang, Wenlin Wang +6
Many deep learning architectures have been proposed to model the compositionality in text sequences, requiring a substantial number of parameters and expensive computations. Howeve…
Joint Embedding of Words and Labels for Text Classification
Guoyin Wang, Chunyuan Li, Wenlin Wang +5
Word embeddings are effective intermediate representations for capturing semantic regularities between words, when learning the representations of text sequences. We propose to vie…
Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling
Zhe Gan, Chunyuan Li, Changyou Chen +3
Recurrent neural networks (RNNs) have shown promising performance for language modeling. However, traditional training of RNNs using back-propagation through time often suffers fro…
Learning Generic Sentence Representations Using Convolutional Neural Networks
Zhe Gan, Yunchen Pu, Ricardo Henao +3
We propose a new encoder-decoder approach to learn distributed sentence representations that are applicable to multiple purposes. The model is learned by using a convolutional neur…