11 papers
Hypergradient-based Bilevel Reinforcement Learning with Improved Sample Complexity
Naman Saxena, Mudit Gaur, Vaneet Aggarwal
Bilevel reinforcement learning (RL) is an important framework within the literature of RL that can be used to formalize various categories of problems, such as meta-learning, hiera…
Oracle-Robust Online Alignment for Large Language Models
Zimeng Li, Mudit Gaur, Vaneet Aggarwal
We study online alignment of large language models under misspecified preference feedback, where the observed preference oracle deviates from an ideal but unknown ground-truth orac…
Order-Optimal Sample Complexity of Rectified Flows
Hari Krishna Sahoo, Mudit Gaur, Vaneet Aggarwal
Recently, flow-based generative models have shown superior efficiency compared to diffusion models. In this paper, we study rectified flow models, which constrain transport traject…
Generative Modeling with Continuous Flows: Sample Complexity of Flow Matching
Mudit Gaur, Prashant Trivedi, Shuchin Aeron +3
Flow matching has recently emerged as a promising alternative to diffusion-based generative models, offering faster sampling and simpler training by learning continuous flows gover…
Discrete State Diffusion Models: A Sample Complexity Perspective
Aadithya Srikanth, Mudit Gaur, Vaneet Aggarwal
Diffusion models have demonstrated remarkable performance in generating high-dimensional samples across domains such as vision, language, and the sciences. Although continuous-stat…
ArchXBench: A Complex Digital Systems Benchmark Suite for LLM Driven RTL Synthesis
Suresh Purini, Siddhant Garg, Mudit Gaur +3
Modern SoC datapaths include deeply pipelined, domain-specific accelerators, but their RTL implementation and verification are still mostly done by hand. While large language model…
Improved Sample Complexity For Diffusion Model Training Without Empirical Risk Minimizer Access
Mudit Gaur, Prashant Trivedi, Sasidhar Kunapuli +2
Diffusion models have demonstrated state-of-the-art performance across vision, language, and scientific domains. Despite their empirical success, prior theoretical analyses of the…
On The Sample Complexity Bounds In Bilevel Reinforcement Learning
Mudit Gaur, Utsav Singh, Amrit Singh Bedi +2
Bilevel reinforcement learning (BRL) has emerged as a powerful framework for aligning generative models, yet its theoretical foundations, especially sample complexity bounds, remai…
On The Global Convergence Of Online RLHF With Neural Parametrization
Mudit Gaur, Amrit Singh Bedi, Raghu Pasupathy +1
The importance of Reinforcement Learning from Human Feedback (RLHF) in aligning large language models (LLMs) with human values cannot be overstated. RLHF is a three-stage process t…
On the Global Convergence of Natural Actor-Critic with Two-layer Neural Network Parametrization
Mudit Gaur, Amrit Singh Bedi, Di Wang +1
Actor-critic algorithms have shown remarkable success in solving state-of-the-art decision-making problems. However, despite their empirical effectiveness, their theoretical underp…
On the Global Convergence of Fitted Q-Iteration with Two-layer Neural Network Parametrization
Mudit Gaur, Vaneet Aggarwal, Mridul Agarwal
Deep Q-learning based algorithms have been applied successfully in many decision making problems, while their theoretical foundations are not as well understood. In this paper, we…