16 citations · 36 across the 34 of their papers we have counts for
20 papers · 1 filter
AdvantageFlow: Regularized Advantage-Weighted RL in Flow Models
Branislav Kveton, Anup Rao, Subhojyoti Mukherjee +2
We present AdvantageFlow, a forward-process reinforcement learning (RL) algorithm for rectified flow models. The algorithm minimizes an advantage-weighted prediction loss, which ma…
Stepwise Credit Assignment for GRPO on Flow-Matching Models
Yash Savani, Branislav Kveton, Yuchen Liu +5
Flow-GRPO successfully applies reinforcement learning to flow models, but uses uniform credit assignment across all steps. This ignores the temporal structure of diffusion generati…
Agentic Planning with Reasoning for Image Styling via Offline RL
Subhojyoti Mukherjee, Stefano Petrangeli, Branislav Kveton +3
Direct prompt-based editing often fails on complex transformations because vague and subjective prompts often require nuanced understanding of what should be changed in the image.…
Partial Policy Gradients for RL in LLMs
Puneet Mathur, Branislav Kveton, Subhojyoti Mukherjee +1
Reinforcement learning is a framework for learning to act sequentially in an unknown environment. We propose a natural approach for modeling policy structure in policy gradients. T…
Learning to Reason in LLMs by Expectation Maximization
Junghyun Lee, Branislav Kveton, Anup Rao +4
Large language models (LLMs) solve reasoning problems by first generating a rationale and then answering. We formalize reasoning as a latent variable model and derive a reward-base…
From Selection to Generation: A Survey of LLM-based Active Learning
Yu Xia, Subhojyoti Mukherjee, Zhouhang Xie +31
Active Learning (AL) has been a powerful paradigm for improving model efficiency and performance by selecting the most informative data points for labeling and training. In recent…
Logits are All We Need to Adapt Closed Models
Gaurush Hiranandani, Haolun Wu, Subhojyoti Mukherjee +1
Many commercial Large Language Models (LLMs) are often closed-source, limiting developers to prompt tuning for aligning content generation with specific applications. While these m…
Multi-Objective Alignment of Large Language Models Through Hypervolume Maximization
Subhojyoti Mukherjee, Anusha Lalitha, Sailik Sengupta +2
Multi-objective alignment from human feedback (MOAHF) in large language models (LLMs) is a challenging problem as human preferences are complex, multifaceted, and often conflicting…
Off-Policy Evaluation from Logged Human Feedback
Aniruddha Bhargava, Lalit Jain, Branislav Kveton +2
Learning from human feedback has been central to recent advances in artificial intelligence and machine learning. Since the collection of human feedback is costly, a natural questi…
SaVeR: Optimal Data Collection Strategy for Safe Policy Evaluation in Tabular MDP
Subhojyoti Mukherjee, Josiah P. Hanna, Robert Nowak
In this paper, we study safe data collection for the purpose of policy evaluation in tabular Markov decision processes (MDPs). In policy evaluation, we are given a \textit{target}…
Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning
Subhojyoti Mukherjee, Josiah P. Hanna, Qiaomin Xie +1
We study learning to learn for the multi-task structured bandit problem where the goal is to learn a near-optimal algorithm that minimizes cumulative regret. The tasks share a comm…
Experimental Design for Active Transductive Inference in Large Language Models
Subhojyoti Mukherjee, Anusha Lalitha, Aniket Deshmukh +3
One emergent ability of large language models (LLMs) is that query-specific examples can be included in the prompt at inference time. In this work, we use active learning for adapt…
Optimal Design for Human Preference Elicitation
Subhojyoti Mukherjee, Anusha Lalitha, Kousha Kalantari +4
Learning of preference models from human feedback has been central to recent advances in artificial intelligence. Motivated by the cost of obtaining high-quality human annotations,…
Multi-task Representation Learning for Pure Exploration in Bilinear Bandits
Subhojyoti Mukherjee, Qiaomin Xie, Josiah P. Hanna +1
We study multi-task representation learning for the problem of pure exploration in bilinear bandits. In bilinear bandits, an action takes the form of a pair of arms from two differ…
Efficient and Interpretable Bandit Algorithms
Subhojyoti Mukherjee, Ruihao Zhu, Branislav Kveton
Motivated by the importance of explainability in modern machine learning, we design bandit algorithms that are efficient and interpretable. A bandit algorithm is interpretable if i…
Safety Aware Changepoint Detection for Piecewise i.i.d. Bandits
Subhojyoti Mukherjee
In this paper, we consider the setting of piecewise i.i.d. bandits under a safety constraint. In this piecewise i.i.d. setting, there exists a finite number of changepoints where t…
ReVar: Strengthening Policy Evaluation via Reduced Variance Sampling
Subhojyoti Mukherjee, Josiah P. Hanna, Robert Nowak
This paper studies the problem of data collection for policy evaluation in Markov decision processes (MDPs). In policy evaluation, we are given a target policy and asked to estimat…
Distribution-dependent and Time-uniform Bounds for Piecewise i.i.d Bandits
Subhojyoti Mukherjee, Odalric-Ambrym Maillard
We consider the setup of stochastic multi-armed bandits in the case when reward distributions are piecewise i.i.d. and bounded with unknown changepoints. We focus on the case when…
Efficient-UCBV: An Almost Optimal Algorithm using Variance Estimates
Subhojyoti Mukherjee, K. P. Naveen, Nandan Sudarsanam +1
We propose a novel variant of the UCB algorithm (referred to as Efficient-UCB-Variance (EUCBV)) for minimizing cumulative regret in the stochastic multi-armed bandit (MAB) setting.…
Thresholding Bandits with Augmented UCB
Subhojyoti Mukherjee, K. P. Naveen, Nandan Sudarsanam +1
In this paper we propose the Augmented-UCB (AugUCB) algorithm for a fixed-budget version of the thresholding bandit problem (TBP), where the objective is to identify a set of arms…