299 citations · 628 across the 30 of their papers we have counts for
36 papers
Shieldstral
Antonia Calvi, Avinash Sooriyarachchi, Giada Pistilli +273
We introduce Shieldstral, a 3B-parameter policy-adaptive multimodal safety classifier that matches or outperforms models nearly 7 its size on text safety benchmarks and set…
Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions
Angana Borah, Rada Mihalcea
As Large Language Models (LLMs) continue to evolve, they are increasingly being employed in numerous studies to simulate societies and execute diverse social tasks. However, LLMs a…
MIND: Math Informed syNthetic Dialogues for Pretraining LLMs
Syeda Nahida Akter, Shrimai Prabhumoye, John Kamalu +5
The utility of synthetic data to enhance pretraining data quality and hence to improve downstream task accuracy has been widely explored in recent large language models (LLMs). Yet…
Data, Data Everywhere: A Guide for Pretraining Dataset Construction
Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings +6
The impressive capabilities of recent language models can be largely attributed to the multi-trillion token pretraining datasets that they are trained on. However, model developers…
Nemotron-4 340B Technical Report
Nvidia, :, Bo Adler +80
We release the Nemotron-4 340B model family, including Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward. Our models are open access under the NVIDIA Open…
AgentKit: Structured LLM Reasoning with Dynamic Graphs
Yue Wu, Yewen Fan, So Yeon Min +6
We propose an intuitive LLM prompting framework (AgentKit) for multifunctional agents. AgentKit offers a unified framework for explicitly constructing a complex "thought process" f…
Nemotron-4 15B Technical Report
Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings +24
We introduce Nemotron-4 15B, a 15-billion-parameter large multilingual language model trained on 8 trillion text tokens. Nemotron-4 15B demonstrates strong performance when assesse…
Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents
Yue Wu, So Yeon Min, Yonatan Bisk +5
Pre-trained large language models (LLMs) capture procedural knowledge about the world. Recent work has leveraged LLM's ability to generate abstract plans to simplify challenging co…
SPRING: Studying the Paper and Reasoning to Play Games
Yue Wu, Shrimai Prabhumoye, So Yeon Min +5
Open-world survival games pose significant challenges for AI algorithms due to their multi-tasking, deep exploration, and goal prioritization requirements. Despite reinforcement le…
Self-Refine: Iterative Refinement with Self-Feedback
Aman Madaan, Niket Tandon, Prakhar Gupta +13
Like humans, large language models (LLMs) do not always generate the best output on their first try. Motivated by how humans refine their written text, we introduce Self-Refine, an…
Adding Instructions during Pretraining: Effective Way of Controlling Toxicity in Language Models
Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi +1
Pretrained large language models have become indispensable for solving various natural language processing (NLP) tasks. However, safely deploying them in real world applications is…
BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Models
Rafal Kocielnik, Shrimai Prabhumoye, Vivian Zhang +3
Pretrained Language Models (PLMs) harbor inherent social biases that can result in harmful real-world implications. Such social biases are measured through the probability values t…
Can You Label Less by Using Out-of-Domain Data? Active & Transfer Learning with Few-shot Instructions
Rafal Kocielnik, Sara Kangaslahti, Shrimai Prabhumoye +3
Labeling social-media data for custom dimensions of toxicity and social bias is challenging and labor-intensive. Existing transfer and active learning approaches meant to reduce an…
Evaluating Parameter Efficient Learning for Generation
Peng Xu, Mostofa Patwary, Shrimai Prabhumoye +6
Parameter efficient learning methods (PERMs) have recently gained significant attention as they provide an efficient way for pre-trained language models (PLMs) to adapt to a downst…
Context Generation Improves Open Domain Question Answering
Dan Su, Mostofa Patwary, Shrimai Prabhumoye +6
Closed-book question answering (QA) requires a model to directly answer an open-domain question without access to any external knowledge. Prior work on closed-book QA either direct…
Multi-Stage Prompting for Knowledgeable Dialogue Generation
Zihan Liu, Mostofa Patwary, Ryan Prenger +4
Existing knowledge-grounded dialogue systems typically use finetuned versions of a pretrained language model (LM) and large-scale knowledge bases. These models typically fail to ge…
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Shaden Smith, Mostofa Patwary, Brandon Norick +17
Pretrained general-purpose language models can achieve state-of-the-art accuracies in various natural language processing domains by adapting to downstream tasks via zero-shot, few…
Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases
Shrimai Prabhumoye, Rafal Kocielnik, Mohammad Shoeybi +2
Detecting social bias in text is challenging due to nuance, subjectivity, and difficulty in obtaining good quality labeled datasets at scale, especially given the evolving nature o…
Focused Attention Improves Document-Grounded Generation
Shrimai Prabhumoye, Kazuma Hashimoto, Yingbo Zhou +2
Document grounded generation is the task of using the information provided in a document to improve text generation. This work focuses on two different document grounded generation…
CURIE: An Iterative Querying Approach for Reasoning About Situations
Dheeraj Rajagopal, Aman Madaan, Niket Tandon +5
Recently, models have been shown to predict the effects of unexpected situations, e.g., would cloudy skies help or hinder plant growth? Given a context, the goal of such situationa…
EIGEN: Event Influence GENeration using Pre-trained Language Models
Aman Madaan, Dheeraj Rajagopal, Yiming Yang +3
Reasoning about events and tracking their influences is fundamental to understanding processes. In this paper, we present EIGEN - a method to leverage pre-trained language models t…
Case Study: Deontological Ethics in NLP
Shrimai Prabhumoye, Brendon Boldt, Ruslan Salakhutdinov +1
Recent work in natural language processing (NLP) has focused on ethical challenges such as understanding and mitigating bias in data and algorithms; identifying objectionable conte…
Exploring Controllable Text Generation Techniques
Shrimai Prabhumoye, Alan W Black, Ruslan Salakhutdinov
Neural controllable text generation is an important area gaining attention due to its plethora of applications. Although there is a large body of prior work in controllable text ge…
Topological Sort for Sentence Ordering
Shrimai Prabhumoye, Ruslan Salakhutdinov, Alan W Black
Sentence ordering is the task of arranging the sentences of a given text in the correct order. Recent work using deep neural networks for this task has framed it as a sequence pred…