4 papers
ZeroS: Zero-Sum Linear Attention for Efficient Transformers
Jiecheng Lu, Xu Han, Yan Sun +4
Linear attention methods offer Transformers complexity but typically underperform standard softmax attention. We identify two fundamental limitations affecting these approac…
CAPS: Unifying Attention, Recurrence, and Alignment in Transformer-based Time Series Forecasting
Viresh Pati, Yubin Kim, Vinh Pham +3
This paper presents (Clock-weighted Aggregation with Prefix-products and Softmax), a structured attention mechanism for time series forecasting that decouples three…
Physics-Informed Inference Time Scaling for Solving High-Dimensional PDE via Defect Correction
Zexi Fan, Yan Sun, Shihao Yang +1
Solving high-dimensional partial differential equations (PDEs) is a critical challenge where modern data-driven solvers often lack reliability and rigorous error guarantees. We int…
Linear Transformers as VAR Models: Aligning Autoregressive Attention Mechanisms with Autoregressive Forecasting
Jiecheng Lu, Shihao Yang
Autoregressive attention-based time series forecasting (TSF) has drawn increasing interest, with mechanisms like linear attention sometimes outperforming vanilla attention. However…