5 papers
Blackboard Intelligence Can Surpass Autoregressive on Globally Constrained Problems
Woosang Jeon, Jaeyeon Kim, Sham Kakade +6
Next-token prediction has driven remarkable progress in large language models, yet a growing body of evidence suggests that they can struggle on problems governed by complex global…
Test-Time Scaling in Diffusion LLMs via Hidden Semi-Autoregressive Experts
Jihoon Lee, Hoyeon Moon, Kevin Zhai +6
Diffusion-based large language models (dLLMs) are trained flexibly to model extreme dependence in the data distribution; however, how to best utilize this information at inference…
SMaRT: Select, Mix, and ReinvenT -- A Strategy Fusion Framework for LLM-Driven Reasoning and Planning
Nikhil Verma, Manasa Bharadwaj, Wonjun Jang +4
Large Language Models (LLMs) have redefined complex task automation with exceptional generalization capabilities. Despite these advancements, state-of-the-art methods rely on singl…
Cross-Attention Speculative Decoding
Wei Zhong, Manasa Bharadwaj, Yixiao Wang +2
Speculative decoding (SD) is a widely adopted approach for accelerating inference in large language models (LLMs), particularly when the draft and target models are well aligned. H…
What Dense Graph Do You Need for Self-Attention?
Yuxin Wang, Chu-Tak Lee, Qipeng Guo +4
Transformers have made progress in miscellaneous tasks, but suffer from quadratic computational and memory complexities. Recent works propose sparse Transformers with attention on…