3 papers
cs.CL2026
FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling
Ted Zadouri, Markus Hoehnerbach, Jay Shah +3
Attention, as a core layer of the ubiquitous Transformer architecture, is the bottleneck for large language models and long-context applications. While FlashAttention-3 optimized a…
cs.AI2026
AI+HW 2035: Shaping the Next Decade
Deming Chen, Jason Cong, Azalia Mirhoseini +27
Artificial intelligence (AI) and hardware (HW) are advancing at unprecedented rates, yet their trajectories have become inseparably intertwined. The global research community lacks…
cs.LG2026
Speculative Speculative Decoding
Tanishq Kumar, Tri Dao, Avner May
Autoregressive decoding is bottlenecked by its sequential nature. Speculative decoding has become a standard way to accelerate inference by using a fast draft model to predict upco…