310 citations · 607 across the 39 of their papers we have counts for
42 papers
Cost-Aware Routing for Efficient Text-To-Image Generation
Qinchan Li, Kenneth Chen, Changyue Su +3
Diffusion models are well known for their ability to generate a high-fidelity image for an input prompt through an iterative denoising process. Unfortunately, the high fidelity als…
Bipartite Ranking From Multiple Labels: On Loss Versus Label Aggregation
Michal Lukasik, Lin Chen, Harikrishna Narasimhan +7
Bipartite ranking is a fundamental supervised learning problem, with the goal of learning a ranking over instances with maximal Area Under the ROC Curve (AUC) against a single bina…
Gatekeeper: Improving Model Cascades Through Confidence Tuning
Stephan Rabanser, Nathalie Rauschmayr, Achin Kulshrestha +5
Large-scale machine learning models deliver strong performance across a wide range of tasks but come with significant computational and resource constraints. To mitigate these chal…
Universal Model Routing for Efficient LLM Inference
Wittawat Jitkrittum, Harikrishna Narasimhan, Ankit Singh Rawat +9
Model routing is a simple technique for reducing the inference cost of large language models (LLMs), wherein one maintains a pool of candidate LLMs, and learns to route each prompt…
A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs
Ankit Singh Rawat, Veeranjaneyulu Sadhanala, Afshin Rostamizadeh +12
A primary challenge in large language model (LLM) development is their onerous pre-training cost. Typically, such pre-training involves optimizing a self-supervised objective (such…
Cascade-Aware Training of Language Models
Congchao Wang, Sean Augenstein, Keith Rush +5
Reducing serving cost and latency is a fundamental concern for the deployment of language models (LMs) in business applications. To address this, cascades of LMs offer an effective…
Faster Cascades via Speculative Decoding
Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat +4
Cascades and speculative decoding are two common approaches to improving language models' inference efficiency. Both approaches involve interleaving models of different sizes, but…
Language Model Cascades: Token-level uncertainty and beyond
Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum +3
Recent advances in language models (LMs) have led to significant improvements in quality on complex NLP tasks, but at the expense of increased inference costs. Cascading offers a s…
It's an Alignment, Not a Trade-off: Revisiting Bias and Variance in Deep Models
Lin Chen, Michal Lukasik, Wittawat Jitkrittum +2
Classical wisdom in machine learning holds that the generalization error can be decomposed into bias and variance, and these two terms exhibit a \emph{trade-off}. However, in this…
When Does Confidence-Based Cascade Deferral Suffice?
Wittawat Jitkrittum, Neha Gupta, Aditya Krishna Menon +3
Cascades are a classical strategy to enable inference cost to vary adaptively across samples, wherein a sequence of classifiers are invoked in turn. A deferral rule determines whet…
Plugin estimators for selective classification with out-of-distribution detection
Harikrishna Narasimhan, Aditya Krishna Menon, Wittawat Jitkrittum +1
Real-world classifiers can benefit from the option of abstaining from predicting on samples where they have low confidence. Such abstention is particularly useful on samples which…
EmbedDistill: A Geometric Knowledge Distillation for Information Retrieval
Seungyeon Kim, Ankit Singh Rawat, Manzil Zaheer +6
Large neural models (such as Transformers) achieve state-of-the-art performance for information retrieval (IR). In this paper, we aim to improve distillation methods that pave the…
A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketch
Patsorn Sangkloy, Wittawat Jitkrittum, Diyi Yang +1
We address the problem of retrieving images with both a sketch and a text query. We present TASK-former (Text And SKetch transformer), an end-to-end trainable model for image retri…
Discussion of `Multiscale Fisher's Independence Test for Multivariate Dependence'
Antonin Schrab, Wittawat Jitkrittum, Zoltán Szabó +2
We discuss how MultiFIT, the Multiscale Fisher's Independence Test for Multivariate Dependence proposed by Gorsky and Ma (2022), compares to existing linear-time kernel tests based…
ELM: Embedding and Logit Margins for Long-Tail Learning
Wittawat Jitkrittum, Aditya Krishna Menon, Ankit Singh Rawat +1
Long-tail learning is the problem of learning under skewed label distributions, which pose a challenge for standard learners. Several recent approaches for the problem have propose…
HD-cos Networks: Efficient Neural Architectures for Secure Multi-Party Computation
Wittawat Jitkrittum, Michal Lukasik, Ananda Theertha Suresh +2
Multi-party computation (MPC) is a branch of cryptography where multiple non-colluding parties execute a well designed protocol to securely compute a function. With the non-colludi…
Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces
Ankit Singh Rawat, Aditya Krishna Menon, Wittawat Jitkrittum +4
Negative sampling schemes enable efficient training given a large number of classes, by offering a means to approximate a computationally expensive loss function that takes all lab…
A Witness Two-Sample Test
Jonas M. Kübler, Wittawat Jitkrittum, Bernhard Schölkopf +1
The Maximum Mean Discrepancy (MMD) has been the state-of-the-art nonparametric test for tackling the two-sample problem. Its statistic is given by the difference in expectations of…
Kernel Distributionally Robust Optimization
Jia-Jie Zhu, Wittawat Jitkrittum, Moritz Diehl +1
We propose kernel distributionally robust optimization (Kernel DRO) using insights from the robust optimization theory and functional analysis. Our method uses reproducing kernel H…
Learning Kernel Tests Without Data Splitting
Jonas M. Kübler, Wittawat Jitkrittum, Bernhard Schölkopf +1
Modern large-scale kernel-based tests such as maximum mean discrepancy (MMD) and kernelized Stein discrepancy (KSD) optimize kernel hyperparameters on a held-out sample via data sp…
Worst-Case Risk Quantification under Distributional Ambiguity using Kernel Mean Embedding in Moment Problem
Jia-Jie Zhu, Wittawat Jitkrittum, Moritz Diehl +1
In order to anticipate rare and impactful events, we propose to quantify the worst-case risk under distributional ambiguity using a recent development in kernel methods -- the kern…
Testing Goodness of Fit of Conditional Density Models with Kernels
Wittawat Jitkrittum, Heishiro Kanagawa, Bernhard Schölkopf
We propose two nonparametric statistical tests of goodness of fit for conditional distributions: given a conditional probability density function and a joint sample, decid…
Kernel Conditional Moment Test via Maximum Moment Restriction
Krikamol Muandet, Wittawat Jitkrittum, Jonas Kübler
We propose a new family of specification tests called kernel conditional moment (KCM) tests. Our tests are built on a novel representation of conditional moment restrictions in a r…
Kernel-Guided Training of Implicit Generative Models with Stability Guarantees
Arash Mehrjou, Wittawat Jitkrittum, Krikamol Muandet +1
Modern implicit generative models such as generative adversarial networks (GANs) are generally known to suffer from issues such as instability, uninterpretability, and difficulty i…