Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
ZeroS: Zero-Sum Linear Attention for Efficient Transformers
Jiecheng Lu, Xu Han, Yan Sun +4
Linear attention methods offer Transformers complexity but typically underperform standard softmax attention. We identify two fundamental limitations affecting these approac…
cs.LG2024
WAVE: Weighted Autoregressive Varying Gate for Time Series Forecasting
Jiecheng Lu, Xu Han, Yan Sun +1
We propose a Weighted Autoregressive Varying gatE (WAVE) attention mechanism equipped with both Autoregressive (AR) and Moving-average (MA) components. It can adapt to various atte…
cs.LG2024
In-context Time Series Predictor
Jiecheng Lu, Yan Sun, Shihao Yang
Recent Transformer-based large language models (LLMs) demonstrate in-context learning ability to perform various functions based solely on the provided context, without updating mo…