2 papers
cs.CL2026
A Systematic Analysis of Hybrid Linear Attention
Dustin Wang, Rui-Jie Zhu, Steven Abreu +9
Transformers face quadratic complexity and memory issues with long sequences, prompting the adoption of linear attention mechanisms using fixed-size hidden states. However, linear…
cs.NE2025
Neuromorphic Principles for Efficient Large Language Models on Intel Loihi 2
Steven Abreu, Sumit Bam Shrestha, Rui-Jie Zhu +1
Large language models (LLMs) deliver impressive performance but require large amounts of energy. In this work, we present a MatMul-free LLM architecture adapted for Intel's neuromo…