4 papers
DynaSpec: Context-aware Dynamic Speculative Sampling for Large-Vocabulary Language Models
Jinbin Zhang, Nasib Ullah, Erik Schultheis +1
Speculative decoding accelerates LLM inference by letting a small drafter propose multiple tokens which a large target model verifies once per speculation step. As vocabularies sca…
ELMO: Efficiency via Low-precision and Peak Memory Optimization in Large Output Spaces
Jinbin Zhang, Nasib Ullah, Erik Schultheis +1
Large output spaces, also referred to as Extreme multilabel classification (XMC), is a setting that arises, e.g., in large-scale tagging and product-to-product recommendation, and…
Navigating Extremes: Dynamic Sparsity in Large Output Spaces
Nasib Ullah, Erik Schultheis, Mike Lasby +2
In recent years, Dynamic Sparse Training (DST) has emerged as an alternative to post-training pruning for generating efficient models. In principle, DST allows for a more memory ef…
Labels in Extremes: How Well Calibrated are Extreme Multi-label Classifiers?
Nasib Ullah, Erik Schultheis, Jinbin Zhang +1
Extreme multilabel classification (XMLC) problems occur in settings such as related product recommendation, large-scale document tagging, or ad prediction, and are characterized by…