49 citations · 127 across the 12 of their papers we have counts for
6 papers · 1 filter
DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training
Dacheng Li, Rulin Shao, Anze Xie +5
FlashAttention (Dao, 2023) effectively reduces the quadratic peak memory usage to linear in training transformer-based large language models (LLMs) on a single GPU. In this paper,…
MIDDAG: Where Does Our News Go? Investigating Information Diffusion via Community-Level Information Pathways
Mingyu Derek Ma, Alexander K. Taylor, Nuan Wen +9
We present MIDDAG, an intuitive, interactive system that visualizes the information propagation paths on social media triggered by COVID-19-related news articles accompanied by com…
RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generation
Shuai Liu, Hyundong J. Cho, Marjorie Freedman +2
Endowing chatbots with a consistent persona is essential to an engaging conversation, yet it remains an unresolved challenge. In this work, we propose a new retrieval-enhanced appr…
LIMA: Less Is More for Alignment
Chunting Zhou, Pengfei Liu, Puxin Xu +12
Large language models are trained in two stages: (1) unsupervised pretraining from raw text, to learn general-purpose representations, and (2) large scale instruction tuning and re…
Challenges in Context-Aware Neural Machine Translation
Linghao Jin, Jacqueline He, Jonathan May +1
Context-aware neural machine translation involves leveraging information beyond sentence-level context to resolve inter-sentential discourse dependencies and improve document-level…
Look-back Decoding for Open-Ended Text Generation
Nan Xu, Chunting Zhou, Asli Celikyilmaz +1
Given a prefix (context), open-ended generation aims to decode texts that are coherent, which do not abruptly drift from previous topics, and informative, which do not suffer from…