9 citations · 30 across the 15 of their papers we have counts for
17 papers
PolyAlign: Conditional Human-Distribution Alignment
L. D. M. S. Sai Teja, Ufaq Khan, Sathira Silva +2
Post-training methods such as supervised fine-tuning (SFT) and preference optimization typically align language models toward a single global assistant behavior. While effective fo…
HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages
Zhilin Wang, Jiaqi Zeng, Olivier Delalleau +6
Preference datasets are essential for training general-domain, instruction-following language models with Reinforcement Learning from Human Feedback (RLHF). Each subsequent data re…
HelpSteer3: Human-Annotated Feedback and Edit Data to Empower Inference-Time Scaling in Open-Ended General-Domain Tasks
Zhilin Wang, Jiaqi Zeng, Olivier Delalleau +6
Inference-Time Scaling has been critical to the success of recent models such as OpenAI o1 and DeepSeek R1. However, many techniques used to train models for inference-time scaling…
Diverging Preferences: When do Annotators Disagree and do Models Know?
Michael JQ Zhang, Zhilin Wang, Jena D. Hwang +6
We examine diverging preferences in human-labeled preference datasets. We develop a taxonomy of disagreement sources spanning ten categories across four high-level classes and find…
HelpSteer2-Preference: Complementing Ratings with Preferences
Zhilin Wang, Alexander Bukharin, Olivier Delalleau +5
Reward models are critical for aligning models to follow instructions, and are typically trained following one of two popular paradigms: Bradley-Terry style or Regression style. Ho…
Data, Data Everywhere: A Guide for Pretraining Dataset Construction
Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings +6
The impressive capabilities of recent language models can be largely attributed to the multi-trillion token pretraining datasets that they are trained on. However, model developers…
HelpSteer2: Open-source dataset for training top-performing reward models
Zhilin Wang, Yi Dong, Olivier Delalleau +6
High-quality preference datasets are essential for training reward models that can effectively guide large language models (LLMs) in generating high-quality responses aligned with…
Nemotron-4 340B Technical Report
Nvidia, :, Bo Adler +80
We release the Nemotron-4 340B model family, including Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward. Our models are open access under the NVIDIA Open…
NeMo-Aligner: Scalable Toolkit for Efficient Model Alignment
Gerald Shen, Zhilin Wang, Olivier Delalleau +10
Aligning Large Language Models (LLMs) with human values and preferences is essential for making them helpful and safe. However, building efficient tools to perform alignment can be…
HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM
Zhilin Wang, Yi Dong, Jiaqi Zeng +8
Existing open-source helpfulness preference datasets do not specify what makes some responses more helpful and others less so. Models trained on these datasets can incidentally lea…
Humanoid Agents: Platform for Simulating Human-like Generative Agents
Zhilin Wang, Yu Ying Chiu, Yu Cheung Chiu
Just as computational simulations of atoms, molecules and cells have shaped the way we study the sciences, true-to-life simulations of human-like agents can be valuable tools for s…
SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF
Yi Dong, Zhilin Wang, Makesh Narsimhan Sreedhar +2
Model alignment with human preferences is an essential step in making Large Language Models (LLMs) helpful and consistent with human values. It typically consists of supervised fin…
FVQA 2.0: Introducing Adversarial Samples into Fact-based Visual Question Answering
Weizhe Lin, Zhilin Wang, Bill Byrne
The widely used Fact-based Visual Question Answering (FVQA) dataset contains visually-grounded questions that require information retrieval using common sense knowledge graphs to a…
Transformer-Empowered Content-Aware Collaborative Filtering
Weizhe Lin, Linjun Shou, Ming Gong +4
Knowledge graph (KG) based Collaborative Filtering is an effective approach to personalizing recommendation systems for relatively static domains such as movies and books, by lever…
Extracting and Inferring Personal Attributes from Dialogue
Zhilin Wang, Xuhui Zhou, Rik Koncel-Kedziorski +2
Personal attributes represent structured information about a person, such as their hobbies, pets, family, likes and dislikes. We introduce the tasks of extracting and inferring per…
Learning Similarity between Movie Characters and Its Potential Implications on Understanding Human Experiences
Zhilin Wang, Weizhe Lin, Xiaodong Wu
While many different aspects of human experiences have been studied by the NLP community, none has captured its full richness. We propose a new task to capture this richness based…
Author2Vec: A Framework for Generating User Embedding
Xiaodong Wu, Weizhe Lin, Zhilin Wang +1
Online forums and social media platforms provide noisy but valuable data every day. In this paper, we propose a novel end-to-end neural network-based user embedding system, Author2…