49 citations · 235 across the 48 of their papers we have counts for
49 papers
Spectral Ghost in Representation Learning: from Component Analysis to Self-Supervised Learning
Bo Dai, Na Li, Dale Schuurmans
Self-supervised learning (SSL) has improved empirical performance by unleashing the power of unlabeled data for practical applications. Specifically, SSL extracts the representatio…
Spectral Representation-based Reinforcement Learning
Chenxiao Gao, Haotian Sun, Na Li +2
In real-world applications with large state and action spaces, reinforcement learning (RL) typically employs function approximations to represent core components like the policies,…
Judging with Confidence: Calibrating Autoraters to Preference Distributions
Zhuohang Li, Xiaowei Li, Chengyu Huang +11
The alignment of large language models (LLMs) with human values increasingly relies on using other LLMs as automated judges, or ``autoraters''. However, their reliability is limite…
Rethinking the Global Convergence of Softmax Policy Gradient with Linear Function Approximation: The Case of Multi-Armed Bandits
Max Qiushi Lin, Jincheng Mei, Matin Aghaei +6
Policy gradient (PG) methods have played an essential role in the empirical successes of reinforcement learning. In order to handle large state-action spaces, PG methods are typica…
Representation Learning via Non-Contrastive Mutual Information
Zhaohan Daniel Guo, Bernardo Avila Pires, Khimya Khetarpal +2
Labeling data is often very time consuming and expensive, leaving us with a majority of unlabeled data. Self-supervised representation learning methods such as SimCLR (Chen et al.,…
Ordering-based Conditions for Global Convergence of Policy Gradient Methods
Jincheng Mei, Bo Dai, Alekh Agarwal +3
We prove that, for finite-arm bandits with linear function approximation, the global convergence of policy gradient (PG) methods depends on inter-related properties between the pol…
Small steps no more: Global convergence of stochastic gradient bandits for arbitrary learning rates
Jincheng Mei, Bo Dai, Alekh Agarwal +4
We provide a new understanding of the stochastic gradient bandit algorithm by showing that it converges to a globally optimal policy almost surely using \emph{any} constant learnin…
Faster WIND: Accelerating Iterative Best-of- Distillation for LLM Alignment
Tong Yang, Jincheng Mei, Hanjun Dai +5
Recent advances in aligning large language models with human preferences have corroborated the growing importance of best-of-N distillation (BOND). However, the iterative BOND algo…
UQE: A Query Engine for Unstructured Databases
Hanjun Dai, Bethany Yixin Wang, Xingchen Wan +7
Analytics on structured data is a mature field with many successful methods. However, most real world data exists in unstructured form, such as images and conversations. We investi…
Target Networks and Over-parameterization Stabilize Off-policy Bootstrapping with Function Approximation
Fengdi Che, Chenjun Xiao, Jincheng Mei +6
We prove that the combination of a target network and over-parameterized linear function approximation establishes a weaker convergence condition for bootstrapped value estimation…
Stochastic Gradient Succeeds for Bandits
Jincheng Mei, Zixin Zhong, Bo Dai +3
We show that the \emph{stochastic gradient} bandit algorithm converges to a \emph{globally optimal} policy at an rate, even with a \emph{constant} step size. Remarkably, g…
Beyond Expectations: Learning with Stochastic Dominance Made Practical
Shicong Cen, Jincheng Mei, Hanjun Dai +3
Stochastic dominance serves as a general framework for modeling a broad spectrum of decision preferences under uncertainty, with risk aversion as one notable example, as it natural…
Provable Representation with Efficient Planning for Partial Observable Reinforcement Learning
Hongming Zhang, Tongzheng Ren, Chenjun Xiao +2
In most real-world reinforcement learning applications, state information is only partially observable, which breaks the Markov decision process assumption and leads to inferior pe…
Scalable Diffusion for Materials Generation
Sherry Yang, KwangHwan Cho, Amil Merchant +4
Generative models trained on internet-scale data are capable of generating novel and realistic texts, images, and videos. A natural next question is whether these models can advanc…
Probabilistic Adaptation of Text-to-Video Models
Mengjiao Yang, Yilun Du, Bo Dai +3
Large text-to-video models trained on internet-scale data have demonstrated exceptional capabilities in generating high-fidelity videos from arbitrary textual descriptions. However…
The Role of Baselines in Policy Gradient Optimization
Jincheng Mei, Wesley Chung, Valentin Thomas +3
We study the effect of baselines in on-policy stochastic policy gradient optimization, and close the gap between the theory and practice of policy optimization methods. Our first c…
A Simple Decentralized Cross-Entropy Method
Zichen Zhang, Jun Jin, Martin Jagersand +2
Cross-Entropy Method (CEM) is commonly used for planning in model-based reinforcement learning (MBRL) where a centralized approach is typically utilized to update the sampling dist…
Managing Temporal Resolution in Continuous Value Estimation: A Fundamental Trade-off
Zichen Zhang, Johannes Kirschner, Junxi Zhang +4
A default assumption in reinforcement learning (RL) and optimal control is that observations arrive at discrete time points on a fixed clock cycle. Yet, many applications involve c…
Latent Variable Representation for Reinforcement Learning
Tongzheng Ren, Chenjun Xiao, Tianjun Zhang +5
Deep latent variable models have achieved significant empirical successes in model-based reinforcement learning (RL) due to their expressiveness in modeling complex transition dyna…
TEMPERA: Test-Time Prompting via Reinforcement Learning
Tianjun Zhang, Xuezhi Wang, Denny Zhou +2
Careful prompt design is critical to the use of large language models in zero-shot or few-shot learning. As a consequence, there is a growing interest in automated methods to desig…
Learning to Optimize with Stochastic Dominance Constraints
Hanjun Dai, Yuan Xue, Niao He +4
In real-world decision-making, uncertainty is important yet difficult to handle. Stochastic dominance provides a theoretically sound approach for comparing uncertain quantities, bu…
Dichotomy of Control: Separating What You Can Control from What You Cannot
Mengjiao Yang, Dale Schuurmans, Pieter Abbeel +1
Future- or return-conditioned supervised learning is an emerging paradigm for offline reinforcement learning (RL), where the future outcome (i.e., return) associated with an observ…
A Parametric Class of Approximate Gradient Updates for Policy Optimization
Ramki Gummadi, Saurabh Kumar, Junfeng Wen +1
Approaches to policy optimization have been motivated from diverse principles, based on how the parametric model is interpreted (e.g. value versus policy representation) or how the…
Discrete Langevin Sampler via Wasserstein Gradient Flow
Haoran Sun, Hanjun Dai, Bo Dai +2
It is known that gradient-based MCMC samplers for continuous spaces, such as Langevin Monte Carlo (LMC), can be derived as particle versions of a gradient flow that minimizes KL di…