25 citations · 75 across the 12 of their papers we have counts for
12 papers
Miles v0.1: Production-Level Post-Training
RadixArk, :, Tom Chen +11
We present Miles v0.1, a full-stack, production-ready system for frontier post-training. Building upon the clean design of slime, Miles designs each stage of the reinforcement-lear…
SETA: Scaling Environments for Terminal Agents
Qijia Shen, Zhiqi Huang, Vamsidhar Kamanuru +19
Large language models (LLMs) are rapidly shifting toward agents that solve tasks through diverse interfaces, including web and graphical user interfaces (GUIs). Among these, the te…
RLHF and IIA: Perverse Incentives
Wanqiao Xu, Shi Dong, Xiuyuan Lu +3
Existing algorithms for reinforcement learning from human feedback (RLHF) can incentivize responses at odds with preferences because they are based on models that assume independen…
Fine-Tuning Language Models with Advantage-Induced Policy Alignment
Banghua Zhu, Hiteshi Sharma, Felipe Vieira Frujeri +4
Reinforcement learning from human feedback (RLHF) has emerged as a reliable approach to aligning large language models (LLMs) to human preferences. Among the plethora of RLHF techn…
Shattering the Agent-Environment Interface for Fine-Tuning Inclusive Language Models
Wanqiao Xu, Shi Dong, Dilip Arumugam +1
A centerpiece of the ever-popular reinforcement learning from human feedback (RLHF) approach to fine-tuning autoregressive language models is the explicit training of a reward mode…
Inclusive Artificial Intelligence
Dilip Arumugam, Shi Dong, Benjamin Van Roy
Prevailing methods for assessing and comparing generative AIs incentivize responses that serve a hypothetical representative individual. Evaluating models in these terms presumes h…
Posterior Sampling for Continuing Environments
Wanqiao Xu, Shi Dong, Benjamin Van Roy
We develop an extension of posterior sampling for reinforcement learning (PSRL) that is suited for a continuing agent-environment interface and integrates naturally into agent desi…
Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent States
Shi Dong, Benjamin Van Roy, Zhengyuan Zhou
We design a simple reinforcement learning (RL) agent that implements an optimistic version of -learning and establish through regret analysis that this agent can operate with so…
Provably Efficient Reinforcement Learning with Aggregated States
Shi Dong, Benjamin Van Roy, Zhengyuan Zhou
We establish that an optimistic variant of Q-learning applied to a fixed-horizon episodic Markov decision process with an aggregated state representation incurs regret $\tilde{\mat…
Comments on the Du-Kakade-Wang-Yang Lower Bounds
Benjamin Van Roy, Shi Dong
Du, Kakade, Wang, and Yang recently established intriguing lower bounds on sample complexity, which suggest that reinforcement learning with a misspecified representation is intrac…
On the Performance of Thompson Sampling on Logistic Bandits
Shi Dong, Tengyu Ma, Benjamin Van Roy
We study the logistic bandit, in which rewards are binary with success probability and actions and coefficients are within the …
An Information-Theoretic Analysis for Thompson Sampling with Many Actions
Shi Dong, Benjamin Van Roy
Information-theoretic Bayesian regret bounds of Russo and Van Roy capture the dependence of regret on prior uncertainty. However, this dependence is through entropy, which can beco…