15 citations · 61 across the 28 of their papers we have counts for
28 papers
LeAct: Learning to Reason from Expert Actions
Ziran Yang, Chengshuai Shi, Raj Ghugare +3
Modern reasoning models depend on reasoning data, today sourced from human annotations or distilled from stronger LLMs. However, a rich and largely untapped source of supervision l…
The Past Is Prologue: A Plug-in Controller for Selective Updates in Sequentially Evolving LLM Memory
Zihan Chen, Songwei Dong, Chengshuai Shi +4
Sequentially evolving LLM memory enables agents to reuse past experience, but existing systems usually deploy each locally generated memory update without checking whether it impro…
Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory
Songwei Dong, Zihan Chen, Chengshuai Shi +3
Memory plays a central role in enabling large language models (LLMs) to operate over sequential tasks by accumulating and reusing experience over time. However, existing evaluation…
Efficient Multi-objective Prompt Optimization via Pure-exploration Bandits
Donghao Li, Chengshuai Shi, Weijuan Ou +2
Prompt engineering has become central to eliciting the capabilities of large language models (LLMs). At its core lies prompt selection -- efficiently identifying the most effective…
Continual Harness: Online Adaptation for Self-Improving Foundation Agents
Seth Karten, Joel Zhang, Tersoo Upaa +5
Coding harnesses such as Claude Code and OpenHands wrap foundation models with tools, memory, and planning, but no equivalent exists for embodied agents' long-horizon partial-obser…
-Divergence Regularized RLHF: Two Tales of Sampling and Unified Analyses
Di Wu, Chengshuai Shi, Jing Yang +1
Reinforcement Learning from Human Feedback (RLHF) has become a cornerstone technique for post-training large language models. While most existing approaches rely on the reverse KL-…
MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI
Bohan Lyu, Yucheng Yang, Siqiao Huang +25
Modern AI progress has been driven by ML methods that are generalizable across settings and scalable to larger regimes. As large language models demonstrate advanced capabilities i…
Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learning
Chengshuai Shi, Wenzhe Li, Xinran Liang +10
Given the rapidly growing capabilities of vision-language models (VLMs), extending them to interactive decision-making tasks such as video games has emerged as a promising frontier…
Greedy Sampling Is Provably Efficient for RLHF
Di Wu, Chengshuai Shi, Jing Yang +1
Reinforcement Learning from Human Feedback (RLHF) has emerged as a key technique for post-training large language models. Despite its empirical success, the theoretical understandi…
From Cross-Task Examples to In-Task Prompts: A Graph-Based Pseudo-Labeling Framework for In-context Learning
Zihan Chen, Song Wang, Xingbo Fu +4
The capability of in-context learning (ICL) enables large language models (LLMs) to perform novel tasks without parameter updates by conditioning on a few input-output examples. Ho…
Augmenting Online RL with Offline Data is All You Need: A Unified Hybrid RL Algorithm Design and Analysis
Ruiquan Huang, Donghao Li, Chengshuai Shi +2
This paper investigates a hybrid learning framework for reinforcement learning (RL) in which the agent can leverage both an offline dataset and online interactions to learn the opt…
Cost-Aware Optimal Pairwise Pure Exploration
Di Wu, Chengshuai Shi, Ruida Zhou +1
Pure exploration is one of the fundamental problems in multi-armed bandits (MAB). However, existing works mostly focus on specific pure exploration tasks, without a holistic view o…
Transformers as Game Players: Provable In-context Game-playing Capabilities of Pre-trained Models
Chengshuai Shi, Kun Yang, Jing Yang +1
The in-context learning (ICL) capability of pre-trained models based on the transformer architecture has received growing interest in recent years. While theoretical understanding…
Building Math Agents with Multi-Turn Iterative Preference Learning
Wei Xiong, Chengshuai Shi, Jiaming Shen +10
Recent studies have shown that large language models' (LLMs) mathematical problem-solving capabilities can be enhanced by integrating external tools, such as code interpreters, and…
Efficient Prompt Optimization Through the Lens of Best Arm Identification
Chengshuai Shi, Kun Yang, Zihan Chen +3
The remarkable instruction-following capability of large language models (LLMs) has sparked a growing interest in automatically finding good prompts, i.e., prompt optimization. Mos…
Harnessing the Power of Federated Learning in Federated Contextual Bandits
Chengshuai Shi, Ruida Zhou, Kun Yang +1
Federated learning (FL) has demonstrated great potential in revolutionizing distributed machine learning, and tremendous efforts have been made to extend it beyond the original foc…
Provably Efficient Offline Reinforcement Learning with Perturbed Data Sources
Chengshuai Shi, Wei Xiong, Cong Shen +1
Existing theoretical studies on offline reinforcement learning (RL) mostly consider a dataset sampled directly from the target task. In practice, however, data often come from seve…
On High-dimensional and Low-rank Tensor Bandits
Chengshuai Shi, Cong Shen, Nicholas D. Sidiropoulos
Most existing studies on linear bandits focus on the one-dimensional characterization of the overall system. While being representative, this formulation may fail to model applicat…
Reward Teaching for Federated Multi-armed Bandits
Chengshuai Shi, Wei Xiong, Cong Shen +1
Most of the existing federated multi-armed bandits (FMAB) designs are based on the presumption that clients will implement the specified design to collaborate with the server. In r…
A Self-Play Posterior Sampling Algorithm for Zero-Sum Markov Games
Wei Xiong, Han Zhong, Chengshuai Shi +2
Existing studies on provably efficient algorithms for Markov games (MGs) almost exclusively build on the "optimism in the face of uncertainty" (OFU) principle. This work focuses on…
Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game
Wei Xiong, Han Zhong, Chengshuai Shi +3
Offline reinforcement learning (RL) aims at learning an optimal strategy using a pre-collected dataset without further interactions with the environment. While various algorithms h…
Heterogeneous Multi-player Multi-armed Bandits: Closing the Gap and Generalization
Chengshuai Shi, Wei Xiong, Cong Shen +1
Despite the significant interests and many progresses in decentralized multi-player multi-armed bandits (MP-MAB) problems in recent years, the regret gap to the natural centralized…
(Almost) Free Incentivized Exploration from Decentralized Learning Agents
Chengshuai Shi, Haifeng Xu, Wei Xiong +1
Incentivized exploration in multi-armed bandits (MAB) has witnessed increasing interests and many progresses in recent years, where a principal offers bonuses to agents to do explo…
Multi-player Multi-armed Bandits with Collision-Dependent Reward Distributions
Chengshuai Shi, Cong Shen
We study a new stochastic multi-player multi-armed bandits (MP-MAB) problem, where the reward distribution changes if a collision occurs on the arm. Existing literature always assu…