39 citations · 70 across the 8 of their papers we have counts for
4 papers · 1 filter
A Practical Investigation of Training-free Relaxed Speculative Decoding
Guoxuan Xia, Luka Ribar, Paul Balanca
Speculative decoding accelerates sampling from an autoregressive LLM by using a faster auxiliary model to draft tokens which are then verified in parallel by the LLM. Standard spec…
Optimal Formats for Weight Quantisation
Douglas Orr, Luka Ribar, Carlo Luschi
Weight quantisation is an essential technique for enabling efficient training and deployment of modern deep learning models. However, the recipe book of quantisation formats is lar…
Approximate Top- for Increased Parallelism
Oscar Key, Luka Ribar, Alberto Cattaneo +2
We present an evaluation of bucketed approximate top- algorithms. Computing top- exactly suffers from limited parallelism, because the largest values must be aggregated a…
SparQ Attention: Bandwidth-Efficient LLM Inference
Luka Ribar, Ivan Chelombiev, Luke Hudlass-Galley +3
The computational difficulties of large language model (LLM) inference remain a significant obstacle to their widespread deployment. The need for many applications to support long…