Fast and Expressive Multi-Byte Prediction with Probabilistic Circuits

Andreas Grivas, Lorenzo Loconte, Emile van Krieken +6

Multi-token prediction (MTP) is a prominent strategy to significantly speed up generation in large language models (LLMs), especially in byte-level LLMs, which are tokeniser-free b…