23 citations · 57 across the 29 of their papers we have counts for
33 papers
Axiom Satisfiability of Linear Rewards in Alignment
Soumya Nasipuri, Sayak Ray Chowdhury, Sanjukta Roy
Learning from human preference data is the dominant route to aligning language models with human values. In linear social choice, where rewards are linear in a fixed feature repres…
Instruction Distillation: Text Instructions as Visual Examples
Hardik Jindal, Soumyabrata Pal, Sayak Ray Chowdhury
Visual in-context learning (ICL) with multimodal large language models (MLLMs) is effective for fine-grained visual classification, but each retrieved image example consumes severa…
Privacy Without Regret: Differentially Private Inference-Time Alignment
Ishi Jain, Nandini Bhattad, Sayak Ray Chowdhury
Best-of-N (BoN) sampling is the simplest and most widely deployed inference-time alignment strategy, but it suffers from two distinct problems: reward hacking, in which the selecte…
Price of Fairness in Bandits: A Tight Minimax Characterization
Dhruv Sarkar, Soumyadeep Dutta, Sayak Ray Chowdhury
In bandit problems, standard regret-minimizing algorithms treat exploration as an amortized cost, which can expose early participants to unfair ex-ante losses in settings such as c…
Improved Algorithms for Nash Welfare in Linear Bandits
Dhruv Sarkar, Nishant Pandey, Sayak Ray Chowdhury
Nash regret has recently emerged as a principled fairness-aware performance metric for stochastic multi-armed bandits, motivated by the Nash Social Welfare objective. Although this…
Revisiting Social Welfare in Bandits: UCB is (Nearly) All You Need
Dhruv Sarkar, Nishant Pandey, Sayak Ray Chowdhury
Regret in stochastic multi-armed bandits traditionally measures the difference between the highest reward and either the arithmetic mean of accumulated rewards or the final reward.…
Why DPO is a Misspecified Estimator and How to Fix It
Aditya Gopalan, Sayak Ray Chowdhury, Debangshu Banerjee
Direct alignment algorithms such as Direct Preference Optimization (DPO) fine-tune models based on preference data, using only supervised learning instead of two-stage reinforcemen…
Constrained Adversarial Perturbation
Virendra Nishad, Bhaskar Mukhoty, Hilal AlQuabeh +2
Deep neural networks have achieved remarkable success in a wide range of classification tasks. However, they remain highly susceptible to adversarial examples - inputs that are sub…
KITE: Kernelized and Information Theoretic Exemplars for In-Context Learning
Vaibhav Singh, Soumya Suvra Ghosal, Kapu Nirmal Joshua +2
In-context learning (ICL) has emerged as a powerful paradigm for adapting large language models (LLMs) to new and data-scarce tasks using only a few carefully selected task-specifi…
DP-NCB: Privacy Preserving Fair Bandits
Dhruv Sarkar, Nishant Pandey, Sayak Ray Chowdhury
Multi-armed bandit algorithms are fundamental tools for sequential decision-making under uncertainty, with widespread applications across domains such as clinical trials and person…
Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift
Seongho Son, William Bankes, Sayak Ray Chowdhury +2
Current Large Language Model (LLM) preference optimization algorithms do not account for temporal preference drift, which can lead to severe misalignment. To address this limitatio…
Provably Robust DPO: Aligning Language Models with Noisy Feedback
Sayak Ray Chowdhury, Anush Kini, Nagarajan Natarajan
Learning from preference-based feedback has recently gained traction as a promising approach to align language models with human interests. While these aligned generative models ha…
Active Preference Optimization for Sample Efficient RLHF
Nirjhar Das, Souradip Chakraborty, Aldo Pacchiano +1
Large Language Models (LLMs) aligned using Reinforcement Learning from Human Feedback (RLHF) have shown remarkable generation abilities in numerous tasks. However, collecting high-…
GAR-meets-RAG Paradigm for Zero-Shot Information Retrieval
Daman Arora, Anush Kini, Sayak Ray Chowdhury +3
Given a query and a document corpus, the information retrieval (IR) task is to output a ranked list of relevant documents. Combining large language models (LLMs) with embedding-bas…
Differentially Private Reward Estimation with Preference Feedback
Sayak Ray Chowdhury, Xingyu Zhou, Nagarajan Natarajan
Learning from preference-based feedback has recently gained considerable traction as a promising approach to align generative models with human interests. Instead of relying on num…
Differentially Private Episodic Reinforcement Learning with Heavy-tailed Rewards
Yulian Wu, Xingyu Zhou, Sayak Ray Chowdhury +1
In this paper, we study the problem of (finite horizon tabular) Markov decision processes (MDPs) with heavy-tailed rewards under the constraint of differential privacy (DP). Compar…
On Differentially Private Federated Linear Contextual Bandits
Xingyu Zhou, Sayak Ray Chowdhury
We consider cross-silo federated linear contextual bandit (LCB) problem under differential privacy, where multiple silos (agents) interact with the local users and communicate via…
Model Selection in Reinforcement Learning with General Function Approximations
Avishek Ghosh, Sayak Ray Chowdhury
We consider model selection for classic Reinforcement Learning (RL) environments -- Multi Armed Bandits (MABs) and Markov Decision Processes (MDPs) -- under general function approx…
Exploration in Linear Bandits with Rich Action Sets and its Implications for Inference
Debangshu Banerjee, Avishek Ghosh, Sayak Ray Chowdhury +1
We present a non-asymptotic lower bound on the eigenspectrum of the design matrix generated by any linear bandit algorithm with sub-linear regret when the action set has well-behav…
Distributed Differential Privacy in Multi-Armed Bandits
Sayak Ray Chowdhury, Xingyu Zhou
We consider the standard -armed bandit problem under a distributed trust model of differential privacy (DP), which enables to guarantee privacy without a trustworthy server. Und…
Shuffle Private Linear Contextual Bandits
Sayak Ray Chowdhury, Xingyu Zhou
Differential privacy (DP) has been recently introduced to linear contextual bandits to formally address the privacy concerns in its associated personalized services to participatin…
Bregman Deviations of Generic Exponential Families
Sayak Ray Chowdhury, Patrick Saux, Odalric-Ambrym Maillard +1
We revisit the method of mixture technique, also known as the Laplace method, to study the concentration phenomenon in generic exponential families. Combining the properties of Bre…
Differentially Private Regret Minimization in Episodic Markov Decision Processes
Sayak Ray Chowdhury, Xingyu Zhou
We study regret minimization in finite horizon tabular Markov decision processes (MDPs) under the constraints of differential privacy (DP). This is motivated by the widespread appl…
Adaptive Control of Differentially Private Linear Quadratic Systems
Sayak Ray Chowdhury, Xingyu Zhou, Ness Shroff
In this paper, we study the problem of regret minimization in reinforcement learning (RL) under differential privacy constraints. This work is motivated by the wide range of RL app…