249 citations · 447 across the 22 of their papers we have counts for
23 papers
ConMeZO: Adaptive Descent-Direction Sampling for Gradient-Free Finetuning of Large Language Models
Lejs Deen Behric, Liang Zhang, Bingcong Li +1
Zeroth-order or derivative-free optimization (MeZO) is an attractive strategy for finetuning large language models (LLMs) because it eliminates the memory overhead of backpropagati…
RewardRank: Optimizing True Learning-to-Rank Utility
Gaurav Bhatt, Kiran Koshy Thekumparampil, Tanmay Gangwani +2
Traditional ranking systems optimize offline proxy objectives that rely on oversimplified assumptions about user behavior, often neglecting factors such as position bias and item d…
Zeroth-Order Optimization Finds Flat Minima
Liang Zhang, Bingcong Li, Kiran Koshy Thekumparampil +3
Zeroth-order methods are extensively used in machine learning applications where gradients are infeasible or expensive to compute, such as black-box attacks, reinforcement learning…
FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain
Rohan Deb, Kiran Thekumparampil, Kousha Kalantari +3
Supervised fine-tuning (SFT) is a standard approach to adapting large language models (LLMs) to new domains. In this work, we improve the statistical efficiency of SFT by selecting…
Comparing Few to Rank Many: Active Human Preference Learning using Randomized Frank-Wolfe
Kiran Koshy Thekumparampil, Gaurush Hiranandani, Kousha Kalantari +2
We study learning of human preferences from a limited comparison feedback. This task is ubiquitous in machine learning. Its applications such as reinforcement learning from human f…
A Sinkhorn-type Algorithm for Constrained Optimal Transport
Xun Tang, Holakou Rahmanian, Michael Shavlovsky +3
Entropic optimal transport (OT) and the Sinkhorn algorithm have made it practical for machine learning practitioners to perform the fundamental task of calculating transport distan…
Accelerating Sinkhorn Algorithm with Sparse Newton Iterations
Xun Tang, Michael Shavlovsky, Holakou Rahmanian +4
Computing the optimal transport distance between statistical distributions is a fundamental task in machine learning. One remarkable recent advancement is entropic regularization a…
DPZero: Private Fine-Tuning of Language Models without Backpropagation
Liang Zhang, Bingcong Li, Kiran Koshy Thekumparampil +2
The widespread practice of fine-tuning large language models (LLMs) on domain-specific data faces two major challenges in memory and privacy. First, as the size of LLMs continues t…
Pretrained deep models outperform GBDTs in Learning-To-Rank under label scarcity
Charlie Hou, Kiran Koshy Thekumparampil, Michael Shavlovsky +3
On tabular data, a significant body of literature has shown that current deep learning (DL) models perform at best similarly to Gradient Boosted Decision Trees (GBDTs), while signi…
Bring Your Own Algorithm for Optimal Differentially Private Stochastic Minimax Optimization
Liang Zhang, Kiran Koshy Thekumparampil, Sewoong Oh +1
We study differentially private (DP) algorithms for smooth stochastic minimax optimization, with stochastic minimization as a byproduct. The holy grail of these settings is to guar…
Lifted Primal-Dual Method for Bilinearly Coupled Smooth Minimax Optimization
Kiran Koshy Thekumparampil, Niao He, Sewoong Oh
We study the bilinearly coupled minimax problem: , where and are both strongly convex smooth functions and admit first-order gra…
FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learning
Charlie Hou, Kiran K. Thekumparampil, Giulia Fanti +1
Federated learning (FL) aims to minimize the communication complexity of training a model over heterogeneous data distributed across many clients. A common approach is local method…
Sample Efficient Linear Meta-Learning by Alternating Minimization
Kiran Koshy Thekumparampil, Prateek Jain, Praneeth Netrapalli +1
Meta-learning synthesizes and leverages the knowledge from a given set of tasks to rapidly learn new tasks using very little data. Meta-learning of linear regression tasks, where t…
Efficient Algorithms for Federated Saddle Point Optimization
Charlie Hou, Kiran K. Thekumparampil, Giulia Fanti +1
We consider strongly convex-concave minimax problems in the federated setting, where the communication constraint is the main bottleneck. When clients are arbitrarily heterogeneous…
Projection Efficient Subgradient Method and Optimal Nonsmooth Frank-Wolfe Method
Kiran Koshy Thekumparampil, Prateek Jain, Praneeth Netrapalli +1
We consider the classical setting of optimizing a nonsmooth Lipschitz continuous convex function over a convex constraint set, when having access to a (stochastic) first-order orac…
Efficient Algorithms for Smooth Minimax Optimization
Kiran Koshy Thekumparampil, Prateek Jain, Praneeth Netrapalli +1
This paper studies first order methods for solving smooth minimax optimization problems where is smooth and is concave for each…
Robust conditional GANs under missing or uncertain labels
Kiran Koshy Thekumparampil, Sewoong Oh, Ashish Khetan
Matching the performance of conditional Generative Adversarial Networks with little supervision is an important task, especially in venturing into new domains. We design a new trai…
InfoGAN-CR and ModelCentrality: Self-supervised Model Training and Selection for Disentangling GANs
Zinan Lin, Kiran Koshy Thekumparampil, Giulia Fanti +1
Disentangled generative models map a latent code vector to a target space, while enforcing that a subset of the learned latent codes are interpretable and associated with distinct…
Robustness of Conditional GANs to Noisy Labels
Kiran Koshy Thekumparampil, Ashish Khetan, Zinan Lin +1
We study the problem of learning conditional generators from noisy labeled samples, where the labels are corrupted by random noise. A standard training of conditional GANs will not…
Attention-based Graph Neural Network for Semi-supervised Learning
Kiran K. Thekumparampil, Chong Wang, Sewoong Oh +1
Recently popularized graph neural networks achieve the state-of-the-art accuracy on a number of standard benchmark datasets for graph-based semi-supervised learning, improving sign…
Learning from Comparisons and Choices
Sahand Negahban, Sewoong Oh, Kiran K. Thekumparampil +1
When tracking user-specific online activities, each user's preference is revealed in the form of choices and comparisons. For example, a user's purchase history is a record of her…
Collaboratively Learning Preferences from Ordinal Data
Sewoong Oh, Kiran K. Thekumparampil, Jiaming Xu
In applications such as recommendation systems and revenue management, it is important to predict preferences on items that have not been seen by a user or predict outcomes of comp…
Sub-Modularity of Waterfilling with Applications to Online Basestation Allocation
Kiran Koshy Thekumparampil, Andrew Thangaraj, Rahul Vaze
We show that the popular water-filling algorithm for maximizing the mutual information in parallel Gaussian channels is sub-modular. The sub-modularity of water-filling algorithm i…