148 citations
Counting papers with at most 25 authors, so collaboration author lists do not stand in for the people here.
- Carnegie Mellon UniversityUS9 papers
- Ben-Gurion University of the NegevIL5 papers
- Centre National de la Recherche ScientifiqueFR5 papers
- ETH ZurichCH5 papers
- University of TorontoCA4 papers
- Chalmers University of TechnologySE3 papers
- Tsinghua UniversityCN3 papers
- University of Hong KongHK3 papers
- University of Illinois Urbana-ChampaignUS3 papers
- Aalto UniversityFI2 papers
- Adobe Systems (United States)US2 papers
- Columbia UniversityUS2 papers
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2024★ 40 cited
LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding
Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich +10
We present LayerSkip, an end-to-end solution to speed-up inference of large language models (LLMs). First, during training we apply layer dropout, with low dropout rates for earlie…
cs.CL2023★ 47 cited
LLM-FP4: 4-Bit Floating-Point Quantized Transformers
Shih-yang Liu, Zechun Liu, Xijie Huang +2
We propose LLM-FP4 for quantizing both weights and activations in large language models (LLMs) down to 4-bit floating-point values, in a post-training manner. Existing post-trainin…