397 citations · 1k across the 73 of their papers we have counts for
10 papers · 1 filter
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BigScience Workshop, :, Teven Le Scao +391
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to wi…
Evaluating Parameter Efficient Learning for Generation
Peng Xu, Mostofa Patwary, Shrimai Prabhumoye +6
Parameter efficient learning methods (PERMs) have recently gained significant attention as they provide an efficient way for pre-trained language models (PLMs) to adapt to a downst…
Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models
David Wingate, Mohammad Shoeybi, Taylor Sorensen
We explore the idea of compressing the prompts used to condition language models, and show that compressed prompts can retain a substantive amount of information about the original…
Context Generation Improves Open Domain Question Answering
Dan Su, Mostofa Patwary, Shrimai Prabhumoye +6
Closed-book question answering (QA) requires a model to directly answer an open-domain question without access to any external knowledge. Prior work on closed-book QA either direct…
FP8 Formats for Deep Learning
Paulius Micikevicius, Dusan Stosic, Neil Burgess +12
FP8 is a natural progression for accelerating deep learning training inference beyond the 16-bit formats common in modern processors. In this paper we propose an 8-bit floating poi…
Factuality Enhanced Language Models for Open-Ended Text Generation
Nayeon Lee, Wei Ping, Peng Xu +4
Pretrained language models (LMs) are susceptible to generate text with nonfactual information. In this work, we measure and improve the factual accuracy of large-scale LMs for open…