34 citations · 150 across the 33 of their papers we have counts for
13 papers · 1 filter
Bradley-Terry Policy Optimization for Generative Preference Modeling
Shengyu Feng, Yun He, Shuang Ma +12
Reinforcement learning (RL) has recently proven effective at scaling chain-of-thought (CoT) reasoning in large language models for tasks with verifiable answers. However, extending…
Preference Optimization with Multi-Sample Comparisons
Chaoqi Wang, Zhuokai Zhao, Chen Zhu +8
Recent advancements in generative models, particularly large language models (LLMs) and diffusion models, have been driven by extensive pretraining on large datasets followed by po…
The Perfect Blend: Redefining RLHF with Mixture of Judges
Tengyu Xu, Eryk Helenowski, Karthik Abinav Sankararaman +17
Reinforcement learning from human feedback (RLHF) has become the leading approach for fine-tuning large language models (LLM). However, RLHF has limitations in multi-task learning…
On the Equivalence of Graph Convolution and Mixup
Xiaotian Han, Hanqing Zeng, Yu Chen +9
This paper investigates the relationship between graph convolution and Mixup techniques. Graph convolution in a graph neural network involves aggregating features from neighboring…
Improved Adaptive Algorithm for Scalable Active Learning with Weak Labeler
Yifang Chen, Karthik Sankararaman, Alessandro Lazaric +6
Active learning with strong and weak labelers considers a practical setting where we have access to both costly but accurate strong labelers and inaccurate but cheap predictions pr…
Contextual Bandits with Packing and Covering Constraints: A Modular Lagrangian Approach via Regression
Aleksandrs Slivkins, Xingyu Zhou, Karthik Abinav Sankararaman +1
We consider contextual bandits with linear constraints (CBwLC), a variant of contextual bandits in which the algorithm consumes multiple resources subject to linear constraints on…
Beyond Regret for Decentralized Bandits in Matching Markets
Soumya Basu, Karthik Abinav Sankararaman, Abishek Sankararaman
We design decentralized algorithms for regret minimization in the two-sided matching market with one-sided bandit feedback that significantly improves upon the prior works (Liu et…
Robust Identifiability in Linear Structural Equation Models of Causal Inference
Karthik Abinav Sankararaman, Anand Louis, Navin Goyal
In this work, we consider the problem of robust parameter estimation from observational data in the context of linear structural equation models (LSEMs). LSEMs are a popular and we…
Dominate or Delete: Decentralized Competing Bandits in Serial Dictatorship
Abishek Sankararaman, Soumya Basu, Karthik Abinav Sankararaman
Online learning in a two-sided matching market, with demand side agents continuously competing to be matched with supply side (arms), abstracts the complex interactions under parti…
Bandits with Knapsacks beyond the Worst-Case
Karthik Abinav Sankararaman, Aleksandrs Slivkins
Bandits with Knapsacks (BwK) is a general model for multi-armed bandits under supply/budget constraints. While worst-case regret bounds for BwK are well-understood, we present thre…
Stability of Linear Structural Equation Models of Causal Inference
Karthik Abinav Sankararaman, Anand Louis, Navin Goyal
We consider the numerical stability of the parameter recovery problem in Linear Structural Equation Model ($\LSEM$) of causal inference. A long line of work starting from Wright (1…
The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent
Karthik A. Sankararaman, Soham De, Zheng Xu +2
This paper studies how neural network architecture affects the speed of training. We introduce a simple concept called gradient confusion to help formally analyze this. When gradie…
Combinatorial Semi-Bandits with Knapsacks
Karthik Abinav Sankararaman, Aleksandrs Slivkins
We unify two prominent lines of work on multi-armed bandits: bandits with knapsacks (BwK) and combinatorial semi-bandits. The former concerns limited "resources" consumed by the al…