30 citations · 109 across the 31 of their papers we have counts for
31 papers
Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
Zhuolin Yang, Zihan Liu, Yang Chen +14
We introduce Nemotron-Cascade 2, an open 30B MoE model with 3B activated parameters that delivers best-in-class reasoning and strong agentic capabilities. Despite its compact size,…
Variance-Dependent Regret Lower Bounds for Contextual Bandits
Jiafan He, Quanquan Gu
Variance-dependent regret bounds for linear contextual bandits, which improve upon the classical regret bound to , wher…
Accelerated Preference Optimization for Large Language Model Alignment
Jiafan He, Huizhuo Yuan, Quanquan Gu
Reinforcement Learning from Human Feedback (RLHF) has emerged as a pivotal tool for aligning large language models (LLMs) with human preferences. Direct Preference Optimization (DP…
Nearly Optimal Algorithms for Contextual Dueling Bandits from Adversarial Feedback
Qiwei Di, Jiafan He, Quanquan Gu
Learning from human feedback plays an important role in aligning generative models, such as large language models (LLM). However, the effectiveness of this approach can be influenc…
Achieving Constant Regret in Linear Markov Decision Processes
Weitong Zhang, Zhiyuan Fan, Jiafan He +1
We study the constant regret guarantees in reinforcement learning (RL). Our objective is to design an algorithm that incurs only finite regret over infinite episodes with high prob…
Nearly Minimax Optimal Regret for Learning Linear Mixture Stochastic Shortest Path
Qiwei Di, Jiafan He, Dongruo Zhou +1
We study the Stochastic Shortest Path (SSP) problem with a linear mixture transition kernel, where an agent repeatedly interacts with a stochastic environment and seeks to reach ce…
Reinforcement Learning from Human Feedback with Active Queries
Kaixuan Ji, Jiafan He, Quanquan Gu
Aligning large language models (LLM) with human preference plays a key role in building modern generative models and can be achieved by reinforcement learning from human feedback (…
Towards Robust Model-Based Reinforcement Learning Against Adversarial Corruption
Chenlu Ye, Jiafan He, Quanquan Gu +1
This study tackles the challenges of adversarial corruption in model-based reinforcement learning (RL), where the transition dynamics can be corrupted by an adversary. Existing stu…
A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation
Heyang Zhao, Jiafan He, Quanquan Gu
The exploration-exploitation dilemma has been a central challenge in reinforcement learning (RL) with complex model classes. In this paper, we propose a new algorithm, Monotonic Q-…
Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning
Qiwei Di, Heyang Zhao, Jiafan He +1
Offline reinforcement learning (RL), where the agent aims to learn the optimal policy based on the data collected by a behavior policy, has attracted increasing attention in recent…
Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation
Yifei Min, Jiafan He, Tianhao Wang +1
We study multi-agent reinforcement learning in the setting of episodic Markov decision processes, where multiple agents cooperate via communication through a central server. We pro…
Integrating Tick-level Data and Periodical Signal for High-frequency Market Making
Jiafa He, Cong Zheng, Can Yang
We focus on the problem of market making in high-frequency trading. Market making is a critical function in financial markets that involves providing liquidity by buying and sellin…
Optimal Execution Using Reinforcement Learning
Cong Zheng, Jiafa He, Can Yang
This work is about optimal order execution, where a large order is split into several small orders to maximize the implementation shortfall. Based on the diversity of cryptocurrenc…
Option Dynamic Hedging Using Reinforcement Learning
Cong Zheng, Jiafa He, Can Yang
This work focuses on the dynamic hedging of financial derivatives, where a reinforcement learning algorithm is designed to minimize the variance of the delta hedging process. In co…
Horizon-free Reinforcement Learning in Adversarial Linear Mixture MDPs
Kaixuan Ji, Qingyue Zhao, Jiafan He +2
Recent studies have shown that episodic reinforcement learning (RL) is no harder than bandits when the total reward is bounded by , and proved regret bounds that have a polyloga…
Uniform-PAC Guarantees for Model-Based RL with Bounded Eluder Dimension
Yue Wu, Jiafan He, Quanquan Gu
Recently, there has been remarkable progress in reinforcement learning (RL) with general function approximation. However, all these works only provide regret or sample complexity g…
On the Interplay Between Misspecification and Sub-optimality Gap in Linear Contextual Bandits
Weitong Zhang, Jiafan He, Zhiyuan Fan +1
We study linear contextual bandits in the misspecified setting, where the expected reward function can be approximated by a linear function class up to a bounded misspecification l…
Variance-Dependent Regret Bounds for Linear Bandits and Reinforcement Learning: Adaptivity and Computational Efficiency
Heyang Zhao, Jiafan He, Dongruo Zhou +2
Recently, several studies (Zhou et al., 2021a; Zhang et al., 2021b; Kim et al., 2021; Zhou and Gu, 2022) have provided variance-dependent regret bounds for linear contextual bandit…
Nearly Minimax Optimal Reinforcement Learning for Linear Markov Decision Processes
Jiafan He, Heyang Zhao, Dongruo Zhou +1
We study reinforcement learning (RL) with linear function approximation. For episodic time-inhomogeneous linear Markov decision processes (linear MDPs) whose transition probability…
A Simple and Provably Efficient Algorithm for Asynchronous Federated Contextual Linear Bandits
Jiafan He, Tianhao Wang, Yifei Min +1
We study federated contextual linear bandits, where agents cooperate with each other to solve a global contextual linear bandit problem with the help of a central server. We co…
Nearly Optimal Algorithms for Linear Contextual Bandits with Adversarial Corruptions
Jiafan He, Dongruo Zhou, Tong Zhang +1
We study the linear contextual bandit problem in the presence of adversarial corruption, where the reward at each round is corrupted by an adversary, and the corruption level (i.e.…
Optimal Online Generalized Linear Regression with Stochastic Noise and Its Application to Heteroscedastic Bandits
Heyang Zhao, Dongruo Zhou, Jiafan He +1
We study the problem of online generalized linear regression in the stochastic setting, where the label is generated from a generalized linear model with possibly unbounded additiv…
Locally Differentially Private Reinforcement Learning for Linear Mixture Markov Decision Processes
Chonghua Liao, Jiafan He, Quanquan Gu
Reinforcement learning (RL) algorithms can be used to provide personalized services, which rely on users' private and sensitive data. To protect the users' privacy, privacy-preserv…
Learning Stochastic Shortest Path with Linear Function Approximation
Yifei Min, Jiafan He, Tianhao Wang +1
We study the stochastic shortest path (SSP) problem in reinforcement learning with linear function approximation, where the transition kernel is represented as a linear mixture of…