1 citations · 1 across the 1 of their papers we have counts for
5 papers
REAP the Experts: Why Pruning Prevails for One-Shot MoE compression
Mike Lasby, Ivan Lazarevich, Nish Sinnadurai +3
Sparsely-activated Mixture-of-Experts (SMoE) models offer efficient pre-training and low latency but their large parameter counts create significant memory overhead, motivating res…
TapOut: A Bandit-Based Approach to Dynamic Speculative Decoding
Aditya Sridhar, Nish Sinnadurai, Sean Lie +1
Speculative decoding accelerates LLMs by using a lightweight draft model to generate tokens autoregressively before verifying them in parallel with a larger target model. However,…
SD: Self-Distilled Sparse Drafters
Mike Lasby, Nish Sinnadurai, Valavan Manohararajah +3
Speculative decoding is a powerful technique for reducing the latency of Large Language Models (LLMs), offering a fault-tolerant framework that enables the use of highly compressed…
MASSV: Multimodal Adaptation and Self-Data Distillation for Speculative Decoding of Vision-Language Models
Mugilan Ganesan, Shane Segal, Ankur Aggarwal +3
Speculative decoding significantly accelerates language model inference by enabling a lightweight draft model to propose multiple tokens that a larger target model verifies simulta…
Self-Data Distillation for Recovering Quality in Pruned Large Language Models
Vithursan Thangarasa, Ganesh Venkatesh, Mike Lasby +2
Large language models have driven significant progress in natural language processing, but their deployment requires substantial compute and memory resources. As models scale, comp…