13 papers
Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMs
Wenhao Yu, Zhicong Lu, Bo Lv +4
Knowledge editing aims to efficiently update factual information in Large Language Models (LLMs) without full retraining. However, existing methods still suffer from performance de…
Beyond Similarity: Temporal Operator Attention for Time Series Analysis
Jevon Twitty, Vinh Pham, Nitiwith Rotchanarak +4
A persistent paradox in time-series forecasting is that structurally simple MLP and linear models often outperform high-capacity Transformers. We argue that this gap arises from a…
HyperMLP: An Integrated Perspective for Sequence Modeling
Jiecheng Lu, Shihao Yang
Self-attention is often viewed as probabilistic query-key lookup, motivating designs that preserve normalized attention scores and fixed positional semantics. We advocate a simpler…
StretchTime: Adaptive Time Series Forecasting via Symplectic Attention
Yubin Kim, Viresh Pati, Jevon Twitty +3
Transformer architectures have established strong baselines in time series forecasting, yet they typically rely on positional encodings that assume uniform, index-based temporal pr…
Free Energy Mixer
Jiecheng Lu, Shihao Yang
Standard attention stores keys/values losslessly but reads them via a per-head convex average, blocking channel-wise selection. We propose the Free Energy Mixer (FEM): a free-energ…
ZeroS: Zero-Sum Linear Attention for Efficient Transformers
Jiecheng Lu, Xu Han, Yan Sun +4
Linear attention methods offer Transformers complexity but typically underperform standard softmax attention. We identify two fundamental limitations affecting these approac…