23 citations · 57 across the 29 of their papers we have counts for
27 papers · 1 filter
Privacy Without Regret: Differentially Private Inference-Time Alignment
Ishi Jain, Nandini Bhattad, Sayak Ray Chowdhury
Best-of-N (BoN) sampling is the simplest and most widely deployed inference-time alignment strategy, but it suffers from two distinct problems: reward hacking, in which the selecte…
Improved Algorithms for Nash Welfare in Linear Bandits
Dhruv Sarkar, Nishant Pandey, Sayak Ray Chowdhury
Nash regret has recently emerged as a principled fairness-aware performance metric for stochastic multi-armed bandits, motivated by the Nash Social Welfare objective. Although this…
Revisiting Social Welfare in Bandits: UCB is (Nearly) All You Need
Dhruv Sarkar, Nishant Pandey, Sayak Ray Chowdhury
Regret in stochastic multi-armed bandits traditionally measures the difference between the highest reward and either the arithmetic mean of accumulated rewards or the final reward.…
Why DPO is a Misspecified Estimator and How to Fix It
Aditya Gopalan, Sayak Ray Chowdhury, Debangshu Banerjee
Direct alignment algorithms such as Direct Preference Optimization (DPO) fine-tune models based on preference data, using only supervised learning instead of two-stage reinforcemen…
Constrained Adversarial Perturbation
Virendra Nishad, Bhaskar Mukhoty, Hilal AlQuabeh +2
Deep neural networks have achieved remarkable success in a wide range of classification tasks. However, they remain highly susceptible to adversarial examples - inputs that are sub…
KITE: Kernelized and Information Theoretic Exemplars for In-Context Learning
Vaibhav Singh, Soumya Suvra Ghosal, Kapu Nirmal Joshua +2
In-context learning (ICL) has emerged as a powerful paradigm for adapting large language models (LLMs) to new and data-scarce tasks using only a few carefully selected task-specifi…
DP-NCB: Privacy Preserving Fair Bandits
Dhruv Sarkar, Nishant Pandey, Sayak Ray Chowdhury
Multi-armed bandit algorithms are fundamental tools for sequential decision-making under uncertainty, with widespread applications across domains such as clinical trials and person…
Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift
Seongho Son, William Bankes, Sayak Ray Chowdhury +2
Current Large Language Model (LLM) preference optimization algorithms do not account for temporal preference drift, which can lead to severe misalignment. To address this limitatio…
Provably Robust DPO: Aligning Language Models with Noisy Feedback
Sayak Ray Chowdhury, Anush Kini, Nagarajan Natarajan
Learning from preference-based feedback has recently gained traction as a promising approach to align language models with human interests. While these aligned generative models ha…
Active Preference Optimization for Sample Efficient RLHF
Nirjhar Das, Souradip Chakraborty, Aldo Pacchiano +1
Large Language Models (LLMs) aligned using Reinforcement Learning from Human Feedback (RLHF) have shown remarkable generation abilities in numerous tasks. However, collecting high-…
Differentially Private Reward Estimation with Preference Feedback
Sayak Ray Chowdhury, Xingyu Zhou, Nagarajan Natarajan
Learning from preference-based feedback has recently gained considerable traction as a promising approach to align generative models with human interests. Instead of relying on num…
Differentially Private Episodic Reinforcement Learning with Heavy-tailed Rewards
Yulian Wu, Xingyu Zhou, Sayak Ray Chowdhury +1
In this paper, we study the problem of (finite horizon tabular) Markov decision processes (MDPs) with heavy-tailed rewards under the constraint of differential privacy (DP). Compar…
On Differentially Private Federated Linear Contextual Bandits
Xingyu Zhou, Sayak Ray Chowdhury
We consider cross-silo federated linear contextual bandit (LCB) problem under differential privacy, where multiple silos (agents) interact with the local users and communicate via…
Exploration in Linear Bandits with Rich Action Sets and its Implications for Inference
Debangshu Banerjee, Avishek Ghosh, Sayak Ray Chowdhury +1
We present a non-asymptotic lower bound on the eigenspectrum of the design matrix generated by any linear bandit algorithm with sub-linear regret when the action set has well-behav…
Distributed Differential Privacy in Multi-Armed Bandits
Sayak Ray Chowdhury, Xingyu Zhou
We consider the standard -armed bandit problem under a distributed trust model of differential privacy (DP), which enables to guarantee privacy without a trustworthy server. Und…
Shuffle Private Linear Contextual Bandits
Sayak Ray Chowdhury, Xingyu Zhou
Differential privacy (DP) has been recently introduced to linear contextual bandits to formally address the privacy concerns in its associated personalized services to participatin…
Bregman Deviations of Generic Exponential Families
Sayak Ray Chowdhury, Patrick Saux, Odalric-Ambrym Maillard +1
We revisit the method of mixture technique, also known as the Laplace method, to study the concentration phenomenon in generic exponential families. Combining the properties of Bre…
Differentially Private Regret Minimization in Episodic Markov Decision Processes
Sayak Ray Chowdhury, Xingyu Zhou
We study regret minimization in finite horizon tabular Markov decision processes (MDPs) under the constraints of differential privacy (DP). This is motivated by the widespread appl…
Adaptive Control of Differentially Private Linear Quadratic Systems
Sayak Ray Chowdhury, Xingyu Zhou, Ness Shroff
In this paper, we study the problem of regret minimization in reinforcement learning (RL) under differential privacy constraints. This work is motivated by the wide range of RL app…
Value Function Approximations via Kernel Embeddings for No-Regret Reinforcement Learning
Sayak Ray Chowdhury, Rafael Oliveira
We consider the regret minimization problem in reinforcement learning (RL) in the episodic setting. In many real-world RL environments, the state and action spaces are continuous o…
No-regret Algorithms for Multi-task Bayesian Optimization
Sayak Ray Chowdhury, Aditya Gopalan
We consider multi-objective optimization (MOO) of an unknown vector-valued function in the non-parametric Bayesian optimization (BO) setting, with the aim being to learn points on…
On Online Learning in Kernelized Markov Decision Processes
Sayak Ray Chowdhury, Aditya Gopalan
We develop algorithms with low regret for learning episodic Markov decision processes based on kernel approximation techniques. The algorithms are based on both the Upper Confidenc…
On Batch Bayesian Optimization
Sayak Ray Chowdhury, Aditya Gopalan
We present two algorithms for Bayesian optimization in the batch feedback setting, based on Gaussian process upper confidence bound and Thompson sampling approaches, along with fre…
Bayesian Optimization under Heavy-tailed Payoffs
Sayak Ray Chowdhury, Aditya Gopalan
We consider black box optimization of an unknown function in the nonparametric Gaussian process setting when the noise in the observed function values can be heavy tailed. This is…