16 papers
Does Latent Chain-of-Thought Actually Reason? A Single, Unifying Taxonomy for Latent Reasoning
Jingjie Qiu, Sheldon Yu, Xunyi Jiang +6
Latent chain-of-thought replaces written reasoning steps with continuous hidden states, giving models additional steps to compute before answering. It remains unclear what computat…
Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight
Sheldon Yu, Tong Yu, Nikki Lijing Kuang +6
When every rollout for a problem fails, GRPO receives neither a successful exemplar nor a nonzero group-relative advantage, so it learns nothing from exactly the problems it most n…
Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory
Minghao Guo, Qingyue Jiao, Xiujin Liu +77
Multimodal models and agents have emerged to enable AI systems to learn from and interact with the real world more like humans by integrating information across multiple modalities…
SonicWorld: Source-Centric Latent Dynamics for Spatial Audio Reasoning
Chenlong Wei, Hanling Wang, Mingjin Lou +6
Dynamic spatial audio reasoning requires a model to associate changing acoustic observations with the correct source and recover its spatial state at a queried time or interval, a…
Toward Latent Language Model Skills Steering and Optimization: An Empirical Study
Xunyi Jiang, Junda Wu, Yuxin Xiong +6
Skills, as a useful abstraction for the procedural capabilities of large language models (LLMs), capture how models perform structured, multi-step reasoning and program execution.…
Agent2UCB: Agentic System for Generative Engine Optimization
Sheldon Yu, Rui Wang, Tong Yu +4
Large language model driven search engines such as Google AI Overviews and Perplexity have created new opportunities for Generative Engine Optimization (GEO) the practice of refini…
Filesystem-Based Memory for LLM Agents: Organization, Evolution, and Sustainability
Sizhe Zhou, Sheldon Yu, Hui Wei +8
Deployed LLM agents increasingly keep their long-term memory as a filesystem: a directory tree of markdown files that the agent itself reads, writes, and reorganizes through generi…
How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectories
Hui Wei, Junda Wu, Sheldon Yu +8
Understanding how computational effort is allocated across individual chain-of-thought (CoT) reasoning steps remains an open challenge: existing interpretability methods rely on ou…
RRPO: Reference-Relative Policy Optimization with Stratified Conditional Rollouts
Yuxin Xiong, Xunyi Jiang, Rohan Surana +8
Group Relative Policy Optimization (GRPO) has shown strong effectiveness in reinforcement learning from verifiable feedback, where sampled rollouts can be compared within a group u…
Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering
Sheldon Yu, Tong Yu, Xunyi Jiang +6
Extended reasoning has become standard for frontier Large Language Models (LLMs), yet the trajectories these models produce remain largely uncontrollable. Existing methods for shap…
OLIVIA: Online Learning via Inference-time Action Adaptation for Decision Making in LLM ReAct Agents
Sheldon Yu, Junda Wu, Xintong Li +6
Large language model agents interleave reasoning, action selection, and observation to solve sequential decision-making tasks. In deployed settings where agents repeatedly handle r…
MASS-DPO: Multi-negative Active Sample Selection for Direct Policy Optimization
Rohan Surana, Xintong Li, Sheldon Yu +7
Multi-negative preference optimization under the Plackett--Luce (PL) model extends Direct Preference Optimization (DPO) by leveraging comparative signals across one preferred and m…
A Low-Latency Fraud Detection Layer for Detecting Adversarial Interaction Patterns in LLM-Powered Agents
Sheldon Yu, Yingcheng Sun, Hanqing Guo +1
Large Language Model (LLM)-powered agents demonstrate strong capabilities in autonomous task execution, tool use, and multi-step reasoning. However, their increasing autonomy also…
Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning
Rohan Surana, Gagan Mundada, Xunyi Jiang +19
Reinforcement learning (RL) has become a central post-training tool for improving the reasoning abilities of large language models (LLMs). In these systems, the rollout, the trajec…
WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning
Gagan Mundada, Zihan Huang, Rohan Surana +8
Group Relative Policy Optimization (GRPO) is effective for training language models on complex reasoning. However, since the objective is defined relative to a group of sampled tra…
Explainable Chain-of-Thought Reasoning: An Empirical Analysis on State-Aware Reasoning Dynamics
Sheldon Yu, Yuxin Xiong, Junda Wu +6
Recent advances in chain-of-thought (CoT) prompting have enabled large language models (LLMs) to perform multi-step reasoning. However, the explainability of such reasoning remains…