2.2k citations · 7.1k across the 72 of their papers we have counts for
6 papers · 2 filters
Evaluating Parameter Efficient Learning for Generation
Peng Xu, Mostofa Patwary, Shrimai Prabhumoye +6
Parameter efficient learning methods (PERMs) have recently gained significant attention as they provide an efficient way for pre-trained language models (PLMs) to adapt to a downst…
Context Generation Improves Open Domain Question Answering
Dan Su, Mostofa Patwary, Shrimai Prabhumoye +6
Closed-book question answering (QA) requires a model to directly answer an open-domain question without access to any external knowledge. Prior work on closed-book QA either direct…
Factuality Enhanced Language Models for Open-Ended Text Generation
Nayeon Lee, Wei Ping, Peng Xu +4
Pretrained language models (LMs) are susceptible to generate text with nonfactual information. In this work, we measure and improve the factual accuracy of large-scale LMs for open…
Multi-Stage Prompting for Knowledgeable Dialogue Generation
Zihan Liu, Mostofa Patwary, Ryan Prenger +4
Existing knowledge-grounded dialogue systems typically use finetuned versions of a pretrained language model (LM) and large-scale knowledge bases. These models typically fail to ge…
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Shaden Smith, Mostofa Patwary, Brandon Norick +17
Pretrained general-purpose language models can achieve state-of-the-art accuracies in various natural language processing domains by adapting to downstream tasks via zero-shot, few…
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models
Boxin Wang, Wei Ping, Chaowei Xiao +6
Pre-trained language models (LMs) are shown to easily generate toxic language. In this work, we systematically explore domain-adaptive training to reduce the toxicity of language m…