20 citations · 30 across the 14 of their papers we have counts for
16 papers
What Matters for NVFP4 Training? A Comparative Study of Low-Precision Pretraining Recipes
Anjulie Agrusa, Andrei Panferov, Elizabeth Wei +7
Training large language models directly in 4-bit floating-point (FP) formats promises substantial improvements in throughput and energy efficiency. While some recipe design choices…
Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
Alexandra Volkova, Matin Ansaripour, Erik Schultheis +2
Quantization-aware training (QAT) leverages lower-precision arithmetic to reduce the cost of LLM deployment, but aggressive quantization degrades final model performance. A common…
ELMO: Efficiency via Low-precision and Peak Memory Optimization in Large Output Spaces
Jinbin Zhang, Nasib Ullah, Erik Schultheis +1
Large output spaces, also referred to as Extreme multilabel classification (XMC), is a setting that arises, e.g., in large-scale tagging and product-to-product recommendation, and…
DynaSpec: Context-aware Dynamic Speculative Sampling for Large-Vocabulary Language Models
Jinbin Zhang, Nasib Ullah, Erik Schultheis +1
Speculative decoding accelerates LLM inference by letting a small drafter propose multiple tokens which a large target model verifies once per speculation step. As vocabularies sca…
Labels in Extremes: How Well Calibrated are Extreme Multi-label Classifiers?
Nasib Ullah, Erik Schultheis, Jinbin Zhang +1
Extreme multilabel classification (XMLC) problems occur in settings such as related product recommendation, large-scale document tagging, or ad prediction, and are characterized by…
Navigating Extremes: Dynamic Sparsity in Large Output Spaces
Nasib Ullah, Erik Schultheis, Mike Lasby +2
In recent years, Dynamic Sparse Training (DST) has emerged as an alternative to post-training pruning for generating efficient models. In principle, DST allows for a more memory ef…
A General Online Algorithm for Optimizing Complex Performance Metrics
Wojciech Kotłowski, Marek Wydmuch, Erik Schultheis +2
We consider sequential maximization of performance metrics that are general functions of a confusion matrix of a classifier (such as precision, F-measure, or G-mean). Such metrics…
Learning label-label correlations in Extreme Multi-label Classification via Label Features
Siddhant Kharbanda, Devaansh Gupta, Erik Schultheis +3
Extreme Multi-label Text Classification (XMC) involves learning a classifier that can assign an input with a subset of most relevant labels from millions of label choices. Recent w…
Consistent algorithms for multi-label classification with macro-at- metrics
Erik Schultheis, Wojciech Kotłowski, Marek Wydmuch +3
We consider the optimization of complex performance metrics in multi-label classification under the population utility framework. We mainly focus on metrics linearly decomposable i…
Generalized test utilities for long-tail performance in extreme multi-label classification
Erik Schultheis, Marek Wydmuch, Wojciech Kotłowski +2
Extreme multi-label classification (XMLC) is the task of selecting a small subset of relevant labels from a very large set of possible labels. As such, it is characterized by long-…
Towards Memory-Efficient Training for Extremely Large Output Spaces -- Learning with 500k Labels on a Single Commodity GPU
Erik Schultheis, Rohit Babbar
In classification problems with large output spaces (up to millions of labels), the last layer can require an enormous amount of memory. Using sparse connectivity would drastically…
CascadeXML: Rethinking Transformers for End-to-end Multi-resolution Training in Extreme Multi-label Classification
Siddhant Kharbanda, Atmadeep Banerjee, Erik Schultheis +1
Extreme Multi-label Text Classification (XMC) involves learning a classifier that can assign an input with a subset of most relevant labels from millions of label choices. Recent a…
On Missing Labels, Long-tails and Propensities in Extreme Multi-label Classification
Erik Schultheis, Marek Wydmuch, Rohit Babbar +1
The propensity model introduced by Jain et al. 2016 has become a standard approach for dealing with missing and long-tail labels in extreme multi-label classification (XMLC). In th…
Speeding-up One-vs-All Training for Extreme Classification via Smart Initialization
Erik Schultheis, Rohit Babbar
In this paper we show that a simple, data dependent way of setting the initial vector can be used to substantially speed up the training of linear one-versus-all (OVA) classifiers…
Unbiased Loss Functions for Multilabel Classification with Missing Labels
Erik Schultheis, Rohit Babbar
This paper considers binary and multilabel classification problems in a setting where labels are missing independently and with a known rate. Missing labels are a ubiquitous phenom…
Unbiased Loss Functions for Extreme Classification With Missing Labels
Erik Schultheis, Mohammadreza Qaraei, Priyanshu Gupta +1
The goal in extreme multi-label classification (XMC) is to tag an instance with a small subset of relevant labels from an extremely large set of possible labels. In addition to the…