4 papers · 1 filter
MineDraft: A Framework for Batch Parallel Speculative Decoding
Zhenwei Tang, Arun Verma, Zijian Zhou +4
Speculative decoding (SD) accelerates large language model inference by using a smaller draft model to propose draft tokens that are subsequently verified by a larger target model.…
MeMo: Memory as a Model
Ryan Wei Heng Quek, Sanghyuk Lee, Alfred Wei Lun Leong +6
Large language models (LLMs) achieve strong performance across a wide range of tasks, but remain frozen after pretraining until subsequent updates. Many real-world applications req…
ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignment
Xiaoqiang Lin, Arun Verma, Zhongxiang Dai +3
The recent success in using human preferences to align large language models (LLMs) has significantly improved their performance in various downstream tasks, such as question answe…
Preconditioned DeltaNet: Curvature-aware Sequence Modeling for Linear Recurrences
Neehal Tumma, Noel Loo, Daniela Rus
To address the increasing long-context compute limitations of softmax attention, several subquadratic recurrent operators have been developed. This work includes models such as Mam…