11 papers · 1 filter
Verifier-Grounded Agentic RL for Provably Correct Code
George Ma, Yuhao Zhang, Bing He +5
Reinforcement learning for code usually rewards passing tests, which a model can do while remaining wrong on the inputs the tests do not cover. A program verifier instead decides w…
How RL Reshapes LLM Reasoning: Transferability, Coverage, and Scaling Laws
Ziheng Cheng, Yixiao Huang, Hanlin Zhu +1
Recent studies on reinforcement learning (RL) report seemingly conflicting evidence about large language model (LLM) reasoning. Training on mathematics can improve performance in o…
SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?
George Ma, Benjamin Mikek, Haoyu Li +9
Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check correctness with held-out test suit…
Transformers Provably Learn to Internalize Chain-of-Thought
Yixiao Huang, Hanlin Zhu, Zixuan Wang +4
Chain-of-Thought (CoT) prompting substantially improves the sample efficiency of transformers, reducing the complexity of tasks like parity learning from exponential to polynomial…
Multi-Objective Learning for Diffusion Models: A Statistical Theory under Semi-Supervised Learning
Ziheng Cheng, Yixiao Huang, Hanlin Zhu +5
Diffusion models are increasingly used as powerful conditional generators, yet real deployments often involve multiple target distributions arising from different tasks, e.g., dive…
Reinforcement Learning via Value Gradient Flow
Haoran Xu, Kaiwen Hu, Somayeh Sojoudi +1
We study behavior-regularized reinforcement learning (RL), where regularization toward a reference distribution (the dataset in offline RL or the base model in LLM RL finetuning) i…
Do Sparse Autoencoders Identify Reasoning Features in Language Models?
George Ma, Zhongyuan Liang, Irene Y. Chen +1
We study how reliably sparse autoencoders (SAEs) support claims about reasoning-related internal features in large language models. We first give a stylized analysis showing that s…
Cross-attention Secretly Performs Orthogonal Alignment in Recommendation Models
Hyunin Lee, Yong Zhang, Hoang Vu Nguyen +8
Cross-domain sequential recommendation (CDSR) aims to align heterogeneous user behavior sequences collected from different domains. While cross-attention is widely used to enhance…
Reinforcement Learning for Flow-Matching Policies
Samuel Pfrommer, Yixiao Huang, Somayeh Sojoudi
Flow-matching policies have emerged as a powerful paradigm for generalist robotics. These models are trained to imitate an action chunk, conditioned on sensor observations and text…
Spooky Action at a Distance: Normalization Layers Enable Side-Channel Spatial Communication
Samuel Pfrommer, George Ma, Yixiao Huang +1
This work shows that normalization layers can facilitate a surprising degree of communication across the spatial dimensions of an input tensor. We study a toy localization task wit…
OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Models
Ziheng Cheng, Yixiao Huang, Hui Xu +4
Text-to-Image (T2I) models have achieved remarkable success in generating visual content from text inputs. Although multiple safety alignment strategies have been proposed to preve…