29 citations · 175 across the 25 of their papers we have counts for
25 papers
Longhorn: State Space Models are Amortized Online Learners
Bo Liu, Rui Wang, Lemeng Wu +3
Modern large language models are built on sequence modeling via next-token prediction. While the Transformer remains the dominant architecture for sequence modeling, its quadratic…
BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents
Zhiwei Liu, Weiran Yao, Jianguo Zhang +12
The massive successes of large language models (LLMs) encourage the emerging exploration of LLM-augmented Autonomous Agents (LAAs). An LAA is able to generate actions with its core…
Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization
Weiran Yao, Shelby Heinecke, Juan Carlos Niebles +12
Recent months have seen the emergence of a powerful new trend in which large language models (LLMs) are augmented to become autonomous language agents capable of performing objecti…
REX: Rapid Exploration and eXploitation for AI Agents
Rithesh Murthy, Shelby Heinecke, Juan Carlos Niebles +12
In this paper, we propose an enhanced approach for Rapid Exploration and eXploitation for AI Agents called REX. Existing AutoGPT-style techniques have inherent limitations, such as…
FAMO: Fast Adaptive Multitask Optimization
Bo Liu, Yihao Feng, Peter Stone +1
One of the grand enduring goals of AI is to create generalist agents that can learn multiple different tasks from diverse data via multitask learning (MTL). However, in practice, a…
LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
Bo Liu, Yifeng Zhu, Chongkai Gao +4
Lifelong learning offers a promising paradigm of building a generalist agent that learns and adapts over its lifespan. Unlike traditional lifelong learning problems in image and te…
Preference-grounded Token-level Guidance for Language Model Fine-tuning
Shentao Yang, Shujian Zhang, Congying Xia +3
Aligning language models (LMs) with preferences is an important problem in natural language generation. A key challenge is that preferences are typically provided at the sequence l…
UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild
Can Qin, Shu Zhang, Ning Yu +10
Achieving machine autonomy and human control often represent divergent objectives in the design of interactive AI systems. Visual generative foundation models such as Stable Diffus…
HIVE: Harnessing Human Feedback for Instructional Visual Editing
Shu Zhang, Xinyi Yang, Yihao Feng +9
Incorporating human feedback has been shown to be crucial to align text generated by large language models to human preferences. We hypothesize that state-of-the-art instructional…
Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systems
Yihao Feng, Shentao Yang, Shujian Zhang +4
When learning task-oriented dialogue (ToD) agents, reinforcement learning (RL) techniques can naturally be utilized to train dialogue strategies to achieve user-specific goals. Pri…
A Unified Framework for Alternating Offline Model Training and Policy Learning
Shentao Yang, Shujian Zhang, Yihao Feng +1
In offline model-based reinforcement learning (offline MBRL), we learn a dynamic model from historically collected data, and subsequently utilize the learned model and fixed datase…
Metric Residual Networks for Sample Efficient Goal-Conditioned Reinforcement Learning
Bo Liu, Yihao Feng, Qiang Liu +1
Goal-conditioned reinforcement learning (GCRL) has a wide range of potential real-world applications, including manipulation and navigation problems in robotics. Especially in such…
Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning
Shentao Yang, Yihao Feng, Shujian Zhang +1
Offline reinforcement learning (RL) extends the paradigm of classical RL algorithms to purely learning from static datasets, without interacting with the underlying environment dur…
A Behavior Regularized Implicit Policy for Offline Reinforcement Learning
Shentao Yang, Zhendong Wang, Huangjie Zheng +2
Offline reinforcement learning enables learning from a fixed dataset, without further interactions with the environment. The lack of environmental interactions makes the policy tra…
Operator Deep Q-Learning: Zero-Shot Reward Transferring in Reinforcement Learning
Ziyang Tang, Yihao Feng, Qiang Liu
Reinforcement learning (RL) has drawn increasing interests in recent years due to its tremendous success in various applications. However, standard RL algorithms can only be applie…
Unsupervised Out-of-Domain Detection via Pre-trained Transformers
Keyang Xu, Tongzheng Ren, Shikun Zhang +2
Deployed real-world machine learning applications are often subject to uncontrolled and even potentially malicious inputs. Those out-of-domain inputs can lead to unpredictable outp…
Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System
Congying Xia, Wenpeng Yin, Yihao Feng +1
Text classification is usually studied by labeling natural language texts with relevant categories from a predefined set. In the real world, new classes might keep challenging the…
Non-asymptotic Confidence Intervals of Off-policy Evaluation: Primal and Dual Bounds
Yihao Feng, Ziyang Tang, Na Zhang +1
Off-policy evaluation (OPE) is the task of estimating the expected reward of a given policy based on offline data previously collected under different policies. Therefore, OPE is a…
Off-Policy Interval Estimation with Lipschitz Value Iteration
Ziyang Tang, Yihao Feng, Na Zhang +2
Off-policy evaluation provides an essential tool for evaluating the effects of different policies or treatments using only observed data. When applied to high-stakes scenarios such…
Accountable Off-Policy Evaluation With Kernel Bellman Statistics
Yihao Feng, Tongzheng Ren, Ziyang Tang +1
We consider off-policy evaluation (OPE), which evaluates the performance of a new policy from observed data collected from previous experiments, without requiring the execution of…
Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimation
Ziyang Tang, Yihao Feng, Lihong Li +2
Infinite horizon off-policy policy evaluation is a highly challenging task due to the excessively large variance of typical importance sampling (IS) estimators. Recently, Liu et al…
A Kernel Loss for Solving the Bellman Equation
Yihao Feng, Lihong Li, Qiang Liu
Value function learning plays a central role in many state-of-the-art reinforcement-learning algorithms. Many popular algorithms like Q-learning do not optimize any objective funct…
Action-depedent Control Variates for Policy Optimization via Stein's Identity
Hao Liu, Yihao Feng, Yi Mao +3
Policy gradient methods have achieved remarkable successes in solving challenging reinforcement learning problems. However, it still often suffers from the large variance issue on…
Learning to Draw Samples with Amortized Stein Variational Gradient Descent
Yihao Feng, Dilin Wang, Qiang Liu
We propose a simple algorithm to train stochastic neural networks to draw samples from given target distributions for probabilistic inference. Our method is based on iteratively ad…