11 papers
IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems
A. Jian Yang, An Huang, Bo Dan +173
We present IQuest-Q1, a sparse Mixture-of-Experts (MoE) agentic foundation model with 320B total parameters (15B activated). IQuest-Q1 is developed through a three-stage process: (…
CUA-Skill: Develop Skills for Computer Using Agent
Tianyi Chen, Yinheng Li, Michael Solodko +12
Computer-Using Agents (CUAs) aim to autonomously operate computer systems to complete real-world tasks. However, existing agentic systems remain difficult to scale and lag behind h…
On the Tension Between Optimality and Adversarial Robustness in Policy Optimization
Haoran Li, Jiayu Lv, Congying Han +5
Achieving optimality and adversarial robustness in deep reinforcement learning has long been regarded as conflicting goals. Nonetheless, recent theoretical insights presented in CA…
Selecting Belief-State Approximations in Simulators with Latent States
Nan Jiang
State resetting is a fundamental but often overlooked capability of simulators. It supports sample-based planning by allowing resets to previously encountered simulation states, an…
Offline Reinforcement Learning in Large State Spaces: Algorithms and Guarantees
Nan Jiang, Tengyang Xie
This article introduces the theory of offline reinforcement learning in large state spaces, where good policies are learned from historical data without online interactions with th…
Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction
Junhong Shen, Hao Bai, Lunjun Zhang +8
The current paradigm of test-time scaling relies on generating long reasoning traces ("thinking" more) before producing a response. In agent problems that require interaction, this…
A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learning
Yuzheng Hu, Fan Wu, Haotian Ye +5
Online reinforcement learning (RL) excels in complex, safety-critical domains but suffers from sample inefficiency, training instability, and limited interpretability. Data attribu…
Reinforcement Learning under Latent Dynamics: Toward Statistical and Algorithmic Modularity
Philip Amortila, Dylan J. Foster, Nan Jiang +2
Real-world applications of reinforcement learning often involve environments where agents operate on complex, high-dimensional observations, but the underlying (''latent'') dynamic…
A Note on Loss Functions and Error Compounding in Model-based Reinforcement Learning
Nan Jiang
This note clarifies some confusions (and perhaps throws out more) around model-based reinforcement learning and their theoretical understanding in the context of deep RL. Main topi…
Harnessing Density Ratios for Online Reinforcement Learning
Philip Amortila, Dylan J. Foster, Nan Jiang +2
The theories of offline and online reinforcement learning, despite having evolved in parallel, have begun to show signs of the possibility for a unification, with algorithms and an…
Marginalized Importance Sampling for Off-Environment Policy Evaluation
Pulkit Katdare, Nan Jiang, Katherine Driggs-Campbell
Reinforcement Learning (RL) methods are typically sample-inefficient, making it challenging to train and deploy RL-policies in real world robots. Even a robust policy trained in si…