5 papers
GiVA: Gradient-Informed Bases for Vector-Based Adaptation
Neeraj Gangwar, Rishabh Deshmukh, Michael Shavlovsky +4
As model sizes continue to grow, parameter-efficient fine-tuning has emerged as a powerful alternative to full fine-tuning. While LoRA is widely adopted among these methods, recent…
Pretrained deep models outperform GBDTs in Learning-To-Rank under label scarcity
Charlie Hou, Kiran Koshy Thekumparampil, Michael Shavlovsky +3
On tabular data, a significant body of literature has shown that current deep learning (DL) models perform at best similarly to Gradient Boosted Decision Trees (GBDTs), while signi…
An efficient algorithm for entropic optimal transport under martingale-type constraints
Xun Tang, Michael Shavlovsky, Holakou Rahmanian +2
This work introduces novel computational methods for entropic optimal transport (OT) problems under martingale-type conditions. The considered problems include the discrete marting…
COS-DPO: Conditioned One-Shot Multi-Objective Fine-Tuning Framework
Yinuo Ren, Tesi Xiao, Michael Shavlovsky +2
In LLM alignment and many other ML applications, one often faces the Multi-Objective Fine-Tuning (MOFT) problem, i.e., fine-tuning an existing model with datasets labeled w.r.t. di…
Orbit: A Framework for Designing and Evaluating Multi-objective Rankers
Chenyang Yang, Tesi Xiao, Michael Shavlovsky +2
Machine learning in production needs to balance multiple objectives: This is particularly evident in ranking or recommendation models, where conflicting objectives such as user eng…