Publications (80)
Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning
Yanda Chen, Chandan Singh, Xiaodong Liu +4
Large language models (LLMs) often generate convincing, fluent explanations. However, different from humans, they often generate inconsistent explanations on different inputs. For…
Parallel Structures in Pre-training Data Yield In-Context Learning
Yanda Chen, Chen Zhao, Zhou Yu +2
Pre-trained language models (LMs) are capable of in-context learning (ICL): they can adapt to a task with only a few examples given in the prompt without any parameter update. Howe…
Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples
Abulhair Saparov, Richard Yuanzhe Pang, Vishakh Padmakumar +4
Given the intractably large size of the space of proofs, any model that is capable of general deductive reasoning must generalize to proofs of greater complexity. Recent studies ha…
Jailbreak Transferability Emerges from Shared Representations
Rico Angell, Jannik Brinkmann, He He
Jailbreak transferability is the surprising phenomenon when an adversarial attack compromising one model also elicits harmful responses from other models. Despite widespread demons…
Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization
Faisal Ladhak, Esin Durmus, He He +2
Despite recent progress in abstractive summarization, systems still suffer from faithfulness errors. While prior work has proposed models that improve faithfulness, it is unclear w…
Pragmatic Radiology Report Generation
Dang Nguyen, Chacha Chen, He He +1
When pneumonia is not found on a chest X-ray, should the report describe this negative observation or omit it? We argue that this question cannot be answered from the X-ray alone a…
Unsupervised Extractive Summarization using Pointwise Mutual Information
Vishakh Padmakumar, He He
Unsupervised approaches to extractive summarization usually rely on a notion of sentence importance defined by the semantic similarity between a sentence and the document. We propo…
Is Agent Code Less Maintainable Than Human Code?
Shaswat Patel, Betty Li Hou, Arun Purohit +4
Maintainability is a core dimension of software engineering, shaping how code is written, reviewed, and developed over time. While coding agents have demonstrated strong performanc…
Different Set Domain Adaptation for Brain-Computer Interfaces: A Label Alignment Approach
He He, Dongrui Wu
A brain-computer interface (BCI) system usually needs a long calibration session for each new subject/task to adjust its parameters, which impedes its transition from the laborator…
Personas as a Way to Model Truthfulness in Language Models
Nitish Joshi, Javier Rando, Abulhair Saparov +2
Large language models (LLMs) are trained on vast amounts of text from the internet, which contains both factual and misleading information about the world. While unintuitive from a…
QuALITY: Question Answering with Long Input Texts, Yes!
Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi +8
To enable building and testing models on long-document comprehension, we introduce QuALITY, a multiple-choice QA dataset with context passages in English that have an average lengt…
On the Relation between Sensitivity and Accuracy in In-context Learning
Yanda Chen, Chen Zhao, Zhou Yu +2
In-context learning (ICL) suffers from oversensitivity to the prompt, making it unreliable in real-world scenarios. We study the sensitivity of ICL with respect to multiple perturb…
Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing
Tuhin Chakrabarty, Vishakh Padmakumar, He He
Recent work in training large language models (LLMs) to follow natural language instructions has opened up exciting opportunities for natural language interface design. Building on…
Reward Gaming in Conditional Text Generation
Richard Yuanzhe Pang, Vishakh Padmakumar, Thibault Sellam +2
To align conditional text generation model outputs with desired behaviors, there has been an increasing focus on training the model using reinforcement learning (RL) with reward fu…
Language Models Learn to Mislead Humans via RLHF
Jiaxin Wen, Ruiqi Zhong, Akbir Khan +6
Language models (LMs) can produce errors that are hard to detect for humans, especially when the task is complex. RLHF, the most popular post-training method, may exacerbate this p…
Leveraging Implicit Feedback from Deployment Data in Dialogue
Richard Yuanzhe Pang, Stephen Roller, Kyunghyun Cho +2
We study improving social conversational agents by learning from natural dialogue between users and a deployed model, without extra annotations. To implicitly measure the quality o…
Opponent Modeling in Deep Reinforcement Learning
He He, Jordan Boyd-Graber, Kevin Kwok +1
Opponent modeling is necessary in multi-agent settings where secondary agents with competing goals also adapt their strategies, yet it remains challenging because strategies intera…
A Credit Assignment Compiler for Joint Prediction
Kai-Wei Chang, He He, Hal Daumé +2
Many machine learning applications involve jointly predicting multiple mutually dependent output variables. Learning to search is a family of methods where the complex decision pro…
Transfer Learning Enhanced Common Spatial Pattern Filtering for Brain Computer Interfaces (BCIs): Overview and a New Approach
He He, Dongrui Wu
The electroencephalogram (EEG) is the most widely used input for brain computer interfaces (BCIs), and common spatial pattern (CSP) is frequently used to spatially filter it to inc…
Spatial Filtering for Brain Computer Interfaces: A Comparison between the Common Spatial Pattern and Its Variant
He He, Dongrui Wu
The electroencephalogram (EEG) is the most popular form of input for brain computer interfaces (BCIs). However, it can be easily contaminated by various artifacts and noise, e.g.,…
LLMs Are Prone to Fallacies in Causal Inference
Nitish Joshi, Abulhair Saparov, Yixin Wang +1
Recent work shows that causal facts can be effectively extracted from LLMs through prompting, facilitating the creation of causal graphs for causal inference tasks. However, it is…
Transfer Learning for Brain-Computer Interfaces: A Euclidean Space Data Alignment Approach
He He, Dongrui Wu
Objective: This paper targets a major challenge in developing practical EEG-based brain-computer interfaces (BCIs): how to cope with individual differences so that better learning…
Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens
Nitish Joshi, Xiang Pan, He He
The term `spurious correlations' has been used in NLP to informally denote any undesirable feature-label correlations. However, a correlation can be undesirable because (i) the fea…
PaintBench: Deterministic Evaluation of Precise Visual Editing
Kai Xu, Ellis Brown, Shrikar Madhu +3
While current multimodal models are proficient at open-ended visual editing, executing precise single-answer edits remains an important obstacle. To probe this challenge, we introd…
Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats
Jiaxin Wen, Vivek Hebbar, Caleb Larson +9
As large language models (LLMs) become increasingly capable, it is prudent to assess whether safety measures remain effective even if LLMs intentionally try to bypass them. Previou…
Estimating Tail Risks in Language Model Output Distributions
Rico Angell, Raghav Singhal, Zachary Horvitz +4
Language models are increasingly capable and are being rapidly deployed on a population-level scale. As a result, the safety of these models is increasingly high-stakes. Fortunatel…
Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context
Urvashi Khandelwal, He He, Peng Qi +1
We know very little about how neural language models (LM) use prior linguistic context. In this paper, we investigate the role of context in an LSTM LM, through ablation studies. S…
How do decoding algorithms distribute information in dialogue responses?
Saranya Venkatraman, He He, David Reitter
Humans tend to follow the Uniform Information Density (UID) principle by distributing information evenly in utterances. We study if decoding algorithms implicitly follow this UID p…
Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer
Juncen Li, Robin Jia, He He +1
We consider the task of text attribute transfer: transforming a sentence to alter a specific attribute (e.g., sentiment) while preserving its attribute-independent content (e.g., c…
Show Your Work with Confidence: Confidence Bands for Tuning Curves
Nicholas Lourie, Kyunghyun Cho, He He
The choice of hyperparameters greatly impacts performance in natural language processing. Often, it is hard to tell if a method is better than another or just better tuned. Tuning…
Is It Thinking or Cheating? Detecting Implicit Reward Hacking by Measuring Reasoning Effort
Xinpeng Wang, Nitish Joshi, Barbara Plank +2
Reward hacking, where a reasoning model exploits loopholes in a reward function to achieve high rewards without solving the intended task, poses a significant threat. This behavior…
Factuality on Demand: Controlling the Factuality-Informativeness Trade-off in Text Generation
Ziwei Gong, Yanda Chen, Julia Hirschberg +4
Large language models (LLMs) encode knowledge with varying degrees of confidence. When responding to queries, models face an inherent trade-off: they can generate responses that ar…
Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations
Chenglei Si, Dan Friedman, Nitish Joshi +3
In-context learning (ICL) is an important paradigm for adapting large language models (LLMs) to new tasks, but the generalization behavior of ICL remains poorly understood. We inve…
Efficient Shapley Values Estimation by Amortization for Text Classification
Chenghao Yang, Fan Yin, He He +3
Despite the popularity of Shapley Values in explaining neural text classification models, computing them is prohibitive for large pretrained models due to a large number of model e…
An Investigation of the (In)effectiveness of Counterfactually Augmented Data
Nitish Joshi, He He
While pretrained language models achieve excellent performance on natural language understanding benchmarks, they tend to rely on spurious correlations and generalize poorly to out…
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Usman Anwar, Abulhair Saparov, Javier Rando +39
This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs). These challenges are organized into three different categories…
Transformers Struggle to Learn to Search
Abulhair Saparov, Srushti Pawar, Shreyas Pimpalgaonkar +6
Search is an ability foundational in many important tasks, and recent studies have shown that large language models (LLMs) struggle to perform search robustly. It is unknown whethe…
Monitoring Decomposition Attacks in LLMs with Lightweight Sequential Monitors
Chen Yueh-Han, Nitish Joshi, Yulin Chen +3
Current LLM safety defenses fail under decomposition attacks, where a malicious goal is decomposed into benign subtasks that circumvent refusals. The challenge lies in the existing…
Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations
Yanda Chen, Ruiqi Zhong, Narutatsu Ri +5
Large language models (LLMs) are trained to imitate humans to explain human decisions. However, do LLMs explain themselves? Can they help humans build mental models of how LLMs pro…
QuAC : Question Answering in Context
Eunsol Choi, He He, Mohit Iyyer +5
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total). The dialogs involve two crowd workers: (1)…
The Unlearnability Phenomenon in RLVR for Language Models
Yulin Chen, He He, Chen Zhao
Reinforcement Learning with Verifiable Reward (RLVR) has proven effective in improving Large Language Model's (LLM) reasoning ability. However, the learning dynamics of RLVR remain…
Active Information Acquisition
He He, Paul Mineiro, Nikos Karampatziakis
We propose a general framework for sequential and dynamic acquisition of useful information in order to solve a particular task. While our goal could in principle be tackled by gen…
Beyond the Binary: Capturing Diverse Preferences With Reward Regularization
Vishakh Padmakumar, Chuanyang Jin, Hannah Rose Kirk +1
Large language models (LLMs) are increasingly deployed via public-facing interfaces to interact with millions of users, each with diverse preferences. Despite this, preference tuni…
Using Sloppy Models for Constrained Emittance Minimization at the Cornell Electron Storage Ring (CESR)
William F. Bergan, Adam C. Bartnik, Ivan V. Bazarov +3
In order to minimize the emittance at the Cornell Electron Storage Ring (CESR), we measure and correct the orbit, dispersion, and transverse coupling of the beam. However, this met…
Decoupling Strategy and Generation in Negotiation Dialogues
He He, Derek Chen, Anusha Balakrishnan +1
We consider negotiation settings in which two agents use natural language to bargain on goods. Agents need to decide on both high-level strategy (e.g., proposing $50) and the exec…
Robustification of Multilingual Language Models to Real-world Noise in Crosslingual Zero-shot Settings with Robust Contrastive Pretraining
Asa Cooper Stickland, Sailik Sengupta, Jason Krone +2
Advances in neural modeling have achieved state-of-the-art (SOTA) results on public natural language processing (NLP) benchmarks, at times surpassing human performance. However, th…
Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings
He He, Anusha Balakrishnan, Mihail Eric +1
We study a symmetric collaborative dialogue setting in which two agents, each with private knowledge, must strategically communicate to achieve a common goal. The open-ended dialog…
Quizbowl: The Case for Incremental Question Answering
Pedro Rodriguez, Shi Feng, Mohit Iyyer +2
Scholastic trivia competitions test knowledge and intelligence through mastery of question answering. Modern question answering benchmarks are one variant of the Turing test. Speci…
Predicting Empirical AI Research Outcomes with Language Models
Jiaxin Wen, Chenglei Si, Yueh-han Chen +2
Many promising-looking ideas in AI research fail to deliver, but their validation takes substantial human labor and compute. Predicting an idea's chance of success is thus crucial…
The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models
Hannah Rose Kirk, Alexander Whitefield, Paul Röttger +9
Human feedback is central to the alignment of Large Language Models (LLMs). However, open questions remain about methods (how), domains (where), people (who) and objectives (to wha…
Unsupervised Elicitation of Language Models
Jiaxin Wen, Zachary Ankner, Arushi Somani +10
To steer pretrained language models for downstream tasks, today's post-training paradigm relies on humans to specify desired behaviors. However, for models with superhuman capabili…
Measuring LLM Novelty As The Frontier Of Original And High-Quality Output
Vishakh Padmakumar, Chen Yueh-Han, Jane Pan +2
As large language models (LLMs) are increasingly used for ideation and scientific discovery, it is important to evaluate their ability to generate novel output. Prior work evaluate…
Multi-View Broad Learning System for Primate Oculomotor Decision Decoding
Zhenhua Shi, Xiaomo Chen, Changming Zhao +3
Multi-view learning improves the learning performance by utilizing multi-view data: data collected from multiple sources, or feature sets extracted from the same data source. This…
ARGUS: Visualization of AI-Assisted Task Guidance in AR
Sonia Castelo, Joao Rulff, Erin McGowan +15
The concept of augmented reality (AR) assistants has captured the human imagination for decades, becoming a staple of modern science fiction. To pursue this goal, it is necessary t…
Does Writing with Language Models Reduce Content Diversity?
Vishakh Padmakumar, He He
Large language models (LLMs) have led to a surge in collaborative writing with model assistance. As different users incorporate suggestions from the same model, there is a risk of…
Text Generation by Learning from Demonstrations
Richard Yuanzhe Pang, He He
Current approaches to text generation largely rely on autoregressive models and maximum likelihood estimation. This paradigm leads to (i) diverse but low-quality samples due to mis…
Meta-learning via Language Model In-context Tuning
Yanda Chen, Ruiqi Zhong, Sheng Zha +2
The goal of meta-learning is to learn to adapt to a new task with only a few labeled examples. To tackle this problem in NLP, we propose , which recasts…
Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification
Anqi Zhang, Yulin Chen, Jane Pan +4
Reasoning models have achieved remarkable performance on tasks like math and logical reasoning thanks to their ability to search during reasoning. However, they still suffer from o…
Iterative Reasoning Preference Optimization
Richard Yuanzhe Pang, Weizhe Yuan, Kyunghyun Cho +3
Iterative preference optimization methods have recently been shown to perform well for general instruction tuning tasks, but typically make little improvement on reasoning tasks (Y…
Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning
Vishakh Padmakumar, Leonard Lausen, Miguel Ballesteros +3
Recent work has found that multi-task training with a large number of diverse tasks can uniformly improve downstream performance on unseen target tasks. In contrast, literature on…
Nuisances via Negativa: Adjusting for Spurious Correlations via Data Augmentation
Aahlad Puli, Nitish Joshi, Yoav Wald +2
In prediction tasks, there exist features that are related to the label in the same way across different settings for that task; these are semantic features or semantics. Features…
Amortized Noisy Channel Neural Machine Translation
Richard Yuanzhe Pang, He He, Kyunghyun Cho
Noisy channel models have been especially effective in neural machine translation (NMT). However, recent approaches like "beam search and rerank" (BSR) incur significant computatio…
Reasoning Models Struggle to Control their Chains of Thought
Chen Yueh-Han, Robert McCarthy, Bruce W. Lee +5
Chain-of-thought (CoT) monitoring is a promising tool for detecting misbehaviors and understanding the motivations of modern reasoning models. However, if models can control what t…
Extrapolative Controlled Sequence Generation via Iterative Refinement
Vishakh Padmakumar, Richard Yuanzhe Pang, He He +1
We study the problem of extrapolative controlled generation, i.e., generating sequences with attribute values beyond the range seen in training. This task is of significant importa…
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought
Abulhair Saparov, He He
Large language models (LLMs) have shown remarkable reasoning capabilities given chain-of-thought prompts (examples with intermediate reasoning steps). Existing benchmarks measure r…
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
Lifu Tu, Garima Lalwani, Spandana Gella +1
Recent work has shown that pre-trained language models such as BERT improve robustness to spurious correlations in the dataset. Intrigued by these results, we find that the key to…
A Dynamic Strategy Coach for Effective Negotiation
Yiheng Zhou, He He, Alan W Black +1
Negotiation is a complex activity involving strategic reasoning, persuasion, and psychology. An average person is often far from an expert in negotiation. Our goal is to assist hum…
Epitaxial strain induced magnetic transitions and phonon instabilities of the tetragonal SrRuO3
He He, Hang-Chen Ding, Yong-Chao Gao +3
Using density-functional theory calculations, we investigate the magnetic as well as the dynamical properties of tetragonal SrRuO3 (SRO) under the influence of epitaxial strain. It…
Training Continuous Chain of Thought Models: A Tale of Two Regimes
Varun Yerram, He He, Eunsol Choi
Continuous Chain-of-Thought methods replace verbose reasoning traces with a short sequence of dense latent representations. Earlier continuous CoT methods indirectly supervise the…
Machine-in-the-Loop Rewriting for Creative Image Captioning
Vishakh Padmakumar, He He
Machine-in-the-loop writing aims to enable humans to collaborate with models to complete their writing tasks more effectively. Prior work has found that providing humans a machine-…
GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing
Jian Guo, He He, Tong He +13
We present GluonCV and GluonNLP, the deep learning toolkits for computer vision and natural language processing based on Apache MXNet (incubating). These toolkits provide state-of-…
Learning to Search for Dependencies
Kai-Wei Chang, He He, Hal Daumé +1
We demonstrate that a dependency parser can be built using a credit assignment compiler which removes the burden of worrying about low-level machine learning details from the parse…
When Benchmarks Talk: Re-Evaluating Code LLMs with Interactive Feedback
Jane Pan, Ryan Shar, Jacob Pfau +3
Programming is a fundamentally interactive process, yet coding assistants are often evaluated using static benchmarks that fail to measure how well models collaborate with users. W…
Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual
He He, Sheng Zha, Haohan Wang
Statistical natural language inference (NLI) models are susceptible to learning dataset bias: superficial cues that happen to associate with the label on a particular dataset, but…
Types of Out-of-Distribution Texts and How to Detect Them
Udit Arora, William Huang, He He
Despite agreement on the importance of detecting out-of-distribution (OOD) examples, there is little consensus on the formal definition of OOD examples and how to best detect them.…
FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization
Esin Durmus, He He, Mona Diab
Neural abstractive summarization models are prone to generate content inconsistent with the source document, i.e. unfaithful. Existing automatic metrics do not capture such mistake…
Pun Generation with Surprise
He He, Nanyun Peng, Percy Liang
We tackle the problem of generating a pun sentence given a pair of homophones (e.g., "died" and "dyed"). Supervised text generation is inappropriate due to the lack of a large corp…
Spontaneous Reward Hacking in Iterative Self-Refinement
Jane Pan, He He, Samuel R. Bowman +1
Language models are capable of iteratively improving their outputs based on natural language feedback, thus enabling in-context optimization of user preference. In place of human u…
Your Co-Workers Matter: Evaluating Collaborative Capabilities of Language Models in Blocks World
Guande Wu, Chen Zhao, Claudio Silva +1
Language agents that interact with the world on their own have great potential for automating digital tasks. While large language model (LLM) agents have made progress in understan…
Hyperparameter Loss Surfaces Are Simple Near their Optima
Nicholas Lourie, He He, Kyunghyun Cho
Hyperparameters greatly impact models' capabilities; however, modern models are too large for extensive search. Instead, researchers design recipes that train well across scales ba…