15 citations · 15 across the 6 of their papers we have counts for
3 papers · 1 filter
ConfLayers: Adaptive Confidence-based Layer Skipping for Self-Speculative Decoding
Walaa Amer, Uday das, Fadi Kurdahi
Self-speculative decoding is an inference technique for large language models designed to speed up generation without sacrificing output quality. It combines fast, approximate deco…
Mixed-Precision Quantization for Language Models: Techniques and Prospects
Mariam Rakka, Marios Fournarakis, Olga Krestinskaya +5
The rapid scaling of language models (LMs) has resulted in unprecedented computational, memory, and energy requirements, making their training and deployment increasingly unsustain…
Mixed-Precision Neural Networks: A Survey
Mariam Rakka, Mohammed E. Fouda, Pramod Khargonekar +1
Mixed-precision Deep Neural Networks achieve the energy efficiency and throughput needed for hardware deployment, particularly when the resources are limited, without sacrificing a…