14 papers
Verifier-Grounded Agentic RL for Provably Correct Code
George Ma, Yuhao Zhang, Bing He +5
Reinforcement learning for code usually rewards passing tests, which a model can do while remaining wrong on the inputs the tests do not cover. A program verifier instead decides w…
How RL Reshapes LLM Reasoning: Transferability, Coverage, and Scaling Laws
Ziheng Cheng, Yixiao Huang, Hanlin Zhu +1
Recent studies on reinforcement learning (RL) report seemingly conflicting evidence about large language model (LLM) reasoning. Training on mathematics can improve performance in o…
SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?
George Ma, Benjamin Mikek, Haoyu Li +9
Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check correctness with held-out test suit…
Transformers Provably Learn to Internalize Chain-of-Thought
Yixiao Huang, Hanlin Zhu, Zixuan Wang +4
Chain-of-Thought (CoT) prompting substantially improves the sample efficiency of transformers, reducing the complexity of tasks like parity learning from exponential to polynomial…
Multi-Objective Learning for Diffusion Models: A Statistical Theory under Semi-Supervised Learning
Ziheng Cheng, Yixiao Huang, Hanlin Zhu +5
Diffusion models are increasingly used as powerful conditional generators, yet real deployments often involve multiple target distributions arising from different tasks, e.g., dive…
ScribbleEdit: Synthetic Data for Image Editing with Scribbles and Text
Anya Ji, George Ma, Téa Wright +4
Recent progress in generative models has significantly advanced image editing capabilities, yet precise and intuitive user control remains difficult. Specifically, users often stru…
Reinforcement Learning via Value Gradient Flow
Haoran Xu, Kaiwen Hu, Somayeh Sojoudi +1
We study behavior-regularized reinforcement learning (RL), where regularization toward a reference distribution (the dataset in offline RL or the base model in LLM RL finetuning) i…
Breaking the Reversal Curse in Autoregressive Language Models via Identity Bridge
Xutao Ma, Yixiao Huang, Hanlin Zhu +1
Autoregressive large language models (LLMs) have achieved remarkable success in many complex tasks, yet they can still fail in very simple logical reasoning such as the "reversal c…
Do Sparse Autoencoders Identify Reasoning Features in Language Models?
George Ma, Zhongyuan Liang, Irene Y. Chen +1
We study how reliably sparse autoencoders (SAEs) support claims about reasoning-related internal features in large language models. We first give a stylized analysis showing that s…
Cross-attention Secretly Performs Orthogonal Alignment in Recommendation Models
Hyunin Lee, Yong Zhang, Hoang Vu Nguyen +8
Cross-domain sequential recommendation (CDSR) aims to align heterogeneous user behavior sequences collected from different domains. While cross-attention is widely used to enhance…
Reinforcement Learning for Flow-Matching Policies
Samuel Pfrommer, Yixiao Huang, Somayeh Sojoudi
Flow-matching policies have emerged as a powerful paradigm for generalist robotics. These models are trained to imitate an action chunk, conditioned on sensor observations and text…
Spooky Action at a Distance: Normalization Layers Enable Side-Channel Spatial Communication
Samuel Pfrommer, George Ma, Yixiao Huang +1
This work shows that normalization layers can facilitate a surprising degree of communication across the spatial dimensions of an input tensor. We study a toy localization task wit…
Generalization or Hallucination? Understanding Out-of-Context Reasoning in Transformers
Yixiao Huang, Hanlin Zhu, Tianyu Guo +5
Large language models (LLMs) can acquire new knowledge through fine-tuning, but this process exhibits a puzzling duality: models can generalize remarkably from new facts, yet are a…
OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Models
Ziheng Cheng, Yixiao Huang, Hui Xu +4
Text-to-Image (T2I) models have achieved remarkable success in generating visual content from text inputs. Although multiple safety alignment strategies have been proposed to preve…