3 papers
cs.LG2026
An Interpretable Latency Model for Speculative Decoding in LLM Serving
Linghao Kong, Megan Flynn, Michael Peng +3
Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller draft model to propose multiple tokens that are verified by a larger target model in p…
physics.ed-ph2024
Comparing large language models for supervised analysis of students' lab notes
Rebeckah K. Fussell, Megan Flynn, Anil Damle +2
Recent advancements in large language models (LLMs) hold significant promise in improving physics education research that uses machine learning. In this study, we compare the appli…
cs.LG2024
STAT: Shrinking Transformers After Training
Megan Flynn, Alexander Wang, Dean Edward Alvarez +2
We present STAT: a simple algorithm to prune transformer models without any fine-tuning. STAT eliminates both attention heads and neurons from the network, while preserving accurac…