6 papers · 1 filter
Compressing Language Models for Specialized Domains
Miles Williams, George Chrysostomou, Vitor Jeronymo +1
Language models (LMs) excel at tasks across diverse domains, yet require substantial computational resources during inference. Compression techniques such as pruning and quantizati…
Speculative Decoding with a Speculative Vocabulary
Miles Williams, Young D. Kwon, Rui Li +2
Speculative decoding has rapidly emerged as a leading approach for accelerating language model (LM) inference, as it offers substantial speedups while yielding identical outputs. T…
Vocabulary-level Memory Efficiency for Language Model Fine-tuning
Miles Williams, Nikolaos Aletras
The extensive memory footprint of language model (LM) fine-tuning poses a challenge for both researchers and practitioners. LMs use an embedding matrix to represent extensive vocab…
Self-calibration for Language Model Quantization and Pruning
Miles Williams, George Chrysostomou, Nikolaos Aletras
Quantization and pruning are fundamental approaches for model compression, enabling efficient inference for language models. In a post-training setting, state-of-the-art quantizati…
Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization
George Chrysostomou, Zhixue Zhao, Miles Williams +1
Despite the remarkable performance of generative large language models (LLMs) on abstractive summarization, they face two significant challenges: their considerable size and tenden…
On the Impact of Calibration Data in Post-training Quantization and Pruning
Miles Williams, Nikolaos Aletras
Quantization and pruning form the foundation of compression for neural networks, enabling efficient inference for large language models (LLMs). Recently, various quantization and p…