4 papers
AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs
Peize He, Zichen Wen, Yubo Wang +13
Processing long-form audio is a major challenge for Large Audio Language models (LALMs). These models struggle with the quadratic cost of attention () and with modeling lon…
Self Speculative Decoding for Diffusion Large Language Models
Yifeng Gao, Ziang Ji, Yuxuan Wang +3
Diffusion-based Large Language Models (dLLMs) have emerged as a competitive alternative to autoregressive models, offering unique advantages through bidirectional attention and par…
TPLA: Tensor Parallel Latent Attention for Efficient Disaggregated Prefill and Decode Inference
Xiaojuan Tang, Fanxu Meng, Pingzhi Tang +4
Multi-Head Latent Attention (MLA), introduced in DeepSeek-V2, compresses key-value states into a low-rank latent vector, caching only this vector to reduce memory. In tensor parall…
Thinking Inside the Mask: In-Place Prompting in Diffusion LLMs
Xiangqi Jin, Yuxuan Wang, Yifeng Gao +4
Despite large language models (LLMs) have achieved remarkable success, their prefix-only prompting paradigm and sequential generation process offer limited flexibility for bidirect…