399 citations · 4k across the 100 of their papers we have counts for
61 papers · 1 filter
Augmenting Language Models with Long-Term Memory
Weizhi Wang, Li Dong, Hao Cheng +4
Existing large language models (LLMs) can only afford fix-sized inputs due to the input length limit, preventing them from utilizing rich long-context information from past inputs.…
Is Self-Repair a Silver Bullet for Code Generation?
Theo X. Olausson, Jeevana Priya Inala, Chenglong Wang +2
Large language models have shown remarkable aptitude in code generation, but still struggle to perform complex tasks. Self-repair -- in which the model debugs and repairs its own c…
Differentiable Tree Operations Promote Compositional Generalization
Paul Soulos, Edward Hu, Kate McCurdy +4
In the context of structure-to-structure transformation tasks, learning sequences of discrete symbolic operations poses significant challenges due to their non-differentiability. T…
Self-Verification Improves Few-Shot Clinical Information Extraction
Zelalem Gero, Chandan Singh, Hao Cheng +4
Extracting patient information from unstructured text is a critical task in health decision-support and clinical research. Large language models (LLMs) have shown the potential to…
Chain-of-Skills: A Configurable Model for Open-domain Question Answering
Kaixin Ma, Hao Cheng, Yu Zhang +3
The retrieval model is an indispensable component for real-world knowledge-intensive tasks, e.g., open-domain question answering (ODQA). As separate retrieval skills are annotated…
Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers
Linyuan Gong, Chenyan Xiong, Xiaodong Liu +5
This paper explores the effectiveness of model-generated signals in improving zero-shot generalization of text-to-text Transformers such as T5. We study various designs to pretrain…
Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding
Yu Zhang, Hao Cheng, Zhihong Shen +3
Scientific literature understanding tasks have gained significant attention due to their potential to accelerate scientific discovery. Pre-trained language models (LMs) have shown…
Instruction Tuning with GPT-4
Baolin Peng, Chunyuan Li, Pengcheng He +2
Prior work has shown that finetuning large language models (LLMs) using machine-generated instruction-following data enables such models to achieve remarkable zero-shot capabilitie…
Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models
Pan Lu, Baolin Peng, Hao Cheng +5
Large language models (LLMs) have achieved remarkable progress in solving various natural language processing tasks due to emergent reasoning abilities. However, LLMs have inherent…
Pre-training Transformers for Knowledge Graph Completion
Sanxing Chen, Hao Cheng, Xiaodong Liu +3
Learning transferable representation of knowledge graphs (KGs) is challenging due to the heterogeneous, multi-relational nature of graph structures. Inspired by Transformer-based p…
Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback
Baolin Peng, Michel Galley, Pengcheng He +8
Large language models (LLMs), such as ChatGPT, are able to generate human-like, fluent responses for many downstream tasks, e.g., task-oriented dialog and question answering. Howev…
Interactive Text Generation
Felix Faltings, Michel Galley, Baolin Peng +5
Users interact with text, image, code, or other editors on a daily basis. However, machine learning models are rarely trained in the settings that reflect the interactivity between…
Guiding Large Language Models via Directional Stimulus Prompting
Zekun Li, Baolin Peng, Pengcheng He +3
We introduce Directional Stimulus Prompting, a novel framework for guiding black-box large language models (LLMs) toward specific desired outputs. Instead of directly adjusting LLM…
Efficient Long Sequence Modeling via State Space Augmented Transformer
Simiao Zuo, Xiaodong Liu, Jian Jiao +4
Transformer models have achieved superior performance in various natural language processing tasks. However, the quadratic computational cost of the attention mechanism limits its…
Grounded Keys-to-Text Generation: Towards Factual Open-Ended Generation
Faeze Brahman, Baolin Peng, Michel Galley +4
Large pre-trained language models have recently enabled open-ended generation frameworks (e.g., prompt-to-text NLG) to tackle a variety of tasks going beyond the traditional data-t…
Language Models as Inductive Reasoners
Zonglin Yang, Li Dong, Xinya Du +5
Inductive reasoning is a core component of human intelligence. In the past research of inductive reasoning within computer science, formal language is used as representations of kn…
DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization
Yu Li, Baolin Peng, Pengcheng He +3
Dialogue summarization has recently garnered significant attention due to its wide range of applications. However, existing methods for summarizing dialogues have limitations becau…
Enhancing Task Bot Engagement with Synthesized Open-Domain Dialog
Miaoran Li, Baolin Peng, Michel Galley +2
Many efforts have been made to construct dialog systems for different types of conversations, such as task-oriented dialog (TOD) and open-domain dialog (ODD). To better mimic human…
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
Yaqing Wang, Sahaj Agarwal, Subhabrata Mukherjee +4
Standard fine-tuning of large pre-trained language models (PLMs) for downstream tasks requires updating hundreds of millions to billions of parameters, and storing a large copy of…
ConvLab-3: A Flexible Dialogue System Toolkit Based on a Unified Data Format
Qi Zhu, Christian Geishauser, Hsien-chin Lin +10
Task-oriented dialogue (TOD) systems function as digital assistants, guiding users through various tasks such as booking flights or finding restaurants. Existing toolkits for build…
Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge
Kaixin Ma, Hao Cheng, Xiaodong Liu +2
We propose a novel open-domain question answering (ODQA) framework for answering single/multi-hop questions across heterogeneous knowledge sources. The key novelty of our method is…
AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation
Ganesh Jawahar, Subhabrata Mukherjee, Xiaodong Liu +6
Mixture-of-Expert (MoE) models have obtained state-of-the-art performance in Neural Machine Translation (NMT) tasks. Existing works in MoE mostly consider a homogeneous design wher…
Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering
Hao Cheng, Hao Fang, Xiaodong Liu +1
Given its effectiveness on knowledge-intensive natural language processing tasks, dense retrieval models have become increasingly popular. Specifically, the de-facto architecture f…
Structural Biases for Improving Transformers on Translation into Morphologically Rich Languages
Paul Soulos, Sudha Rao, Caitlin Smith +9
Machine translation has seen rapid progress with the advent of Transformer-based models. These models have no explicit linguistic structure built into them, yet they may still impl…