3 papers
cs.LG2025
Out-of-Vocabulary Sampling Boosts Speculative Decoding
Nadav Timor, Jonathan Mamou, Oren Pereg +2
Speculative decoding relies on fast and accurate drafters. Recent state-of-the-art language models employ larger and larger vocabularies, which significantly slows down drafters. O…
cs.LG2025
NdLinear: Preserving Multi-Dimensional Structure for Parameter-Efficient Neural Networks
Alex Reneau, Jerry Yao-Chieh Hu, Zhongfang Zhuang +6
In deep learning, processing multidimensional inputs (e.g., images, medical scans, and time series) is an important task that often requires flattening the inputs. We introduce $\m…
cs.CL2025
Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies
Nadav Timor, Jonathan Mamou, Daniel Korat +5
Accelerating the inference of large language models (LLMs) is a critical challenge in generative AI. Speculative decoding (SD) methods offer substantial efficiency gains by generat…