3 papers
cs.LG2026
ZeroS: Zero-Sum Linear Attention for Efficient Transformers
Jiecheng Lu, Xu Han, Yan Sun +4
Linear attention methods offer Transformers complexity but typically underperform standard softmax attention. We identify two fundamental limitations affecting these approac…
cs.LG2026
CAPS: Unifying Attention, Recurrence, and Alignment in Transformer-based Time Series Forecasting
Viresh Pati, Yubin Kim, Vinh Pham +3
This paper presents (Clock-weighted Aggregation with Prefix-products and Softmax), a structured attention mechanism for time series forecasting that decouples three…
cs.LG2025
Linear Transformers as VAR Models: Aligning Autoregressive Attention Mechanisms with Autoregressive Forecasting
Jiecheng Lu, Shihao Yang
Autoregressive attention-based time series forecasting (TSF) has drawn increasing interest, with mechanisms like linear attention sometimes outperforming vanilla attention. However…