4 citations · 5 across the 5 of their papers we have counts for
5 papers
H-Spec: Parallel Speculative Decoding Without a Drafter-Side KV Cache
Weifan Jiang, Krishna Teja Chitty-Venkata, Megan Flynn +6
Speculative decoding losslessly accelerates large language model inference by having a lightweight draft model predict future tokens for verification by the target model. Recent bl…
An Interpretable Latency Model for Speculative Decoding in LLM Serving
Linghao Kong, Megan Flynn, Michael Peng +3
Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller draft model to propose multiple tokens that are verified by a larger target model in p…
Comparing large language models for supervised analysis of students' lab notes
Rebeckah K. Fussell, Megan Flynn, Anil Damle +2
Recent advancements in large language models (LLMs) hold significant promise in improving physics education research that uses machine learning. In this study, we compare the appli…
STAT: Shrinking Transformers After Training
Megan Flynn, Alexander Wang, Dean Edward Alvarez +2
We present STAT: a simple algorithm to prune transformer models without any fine-tuning. STAT eliminates both attention heads and neurons from the network, while preserving accurac…
Model Preserving Compression for Neural Networks
Jerry Chee, Megan Renz, Anil Damle +1
After training complex deep learning models, a common task is to compress the model to reduce compute and storage demands. When compressing, it is desirable to preserve the origina…