RadixMLP -- Intra-batch Deduplication for Causal Transformers

Michael Feil, Julius Lipp

Batch inference workloads for causal transformer models frequently process sequences that share common prefixes, such as system prompts, few-shot examples, or shared queries. Stand…