8 papers
Residual Context Diffusion Language Models
Yuezhou Hu, Harman Singh, Monishwaran Maheswaran +10
Diffusion Large Language Models (dLLMs) have emerged as a promising alternative to purely autoregressive language models because they can decode multiple tokens in parallel. Howeve…
LoSA: Locality Aware Sparse Attention for Block-Wise Diffusion Language Models
Haocheng Xi, Harman Singh, Yuezhou Hu +9
Block-wise diffusion language models (DLMs) generate multiple tokens in any order, offering a promising alternative to the autoregressive decoding pipeline. However, they still rem…
Reward Under Attack: Analyzing the Robustness and Hackability of Process Reward Models
Rishabh Tiwari, Aditya Tomar, Udbhav Bamba +5
Process Reward Models (PRMs) are rapidly becoming the backbone of LLM reasoning pipelines, yet we demonstrate that state-of-the-art PRMs are systematically exploitable under advers…
Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models
Minseo Kim, Coleman Hooper, Aditya Tomar +5
Large Language Models (LLMs) have achieved state-of-the-art performance on a broad range of Natural Language Processing (NLP) tasks, including document processing and code generati…
XQuant: Breaking the Memory Wall for LLM Inference with KV Cache Rematerialization
Aditya Tomar, Coleman Hooper, Minjae Lee +7
Although LLM inference has emerged as a critical workload for many downstream applications, efficiently inferring LLMs is challenging due to the substantial memory footprint and ba…
Can Transformers Break Encryption Schemes via In-Context Learning?
Jathin Korrapati, Patrick Mendoza, Aditya Tomar +1
In-context learning (ICL) has emerged as a powerful capability of transformer-based language models, enabling them to perform tasks by conditioning on a small number of examples pr…