6 citations · 7 across the 11 of their papers we have counts for
13 papers
Target-Token Asymmetry Can Masquerade As Depth Specialization
Danlin Li, Miao Li, Xu Han +1
Layer-ablation studies often perturb early or late layers of a language model, compare the damage to prediction on two corpora, and read a larger early-layer effect on one corpus a…
Match the Canvas, Not Just the Answer: Predictability-Aware SFT for Diffusion Language Models
Miao Li, Hanyang Jiang, Hengyu Fu +8
Diffusion language models (DLMs) enable flexible parallel generation by resolving tokens according to the model's evolving predictability rather than a fixed left-to-right order. T…
Beyond Similarity: Temporal Operator Attention for Time Series Analysis
Jevon Twitty, Vinh Pham, Nitiwith Rotchanarak +4
A persistent paradox in time-series forecasting is that structurally simple MLP and linear models often outperform high-capacity Transformers. We argue that this gap arises from a…
HyperMLP: An Integrated Perspective for Sequence Modeling
Jiecheng Lu, Shihao Yang
Self-attention is often viewed as probabilistic query-key lookup, motivating designs that preserve normalized attention scores and fixed positional semantics. We advocate a simpler…
StretchTime: Adaptive Time Series Forecasting via Symplectic Attention
Yubin Kim, Viresh Pati, Jevon Twitty +3
Transformer architectures have established strong baselines in time series forecasting, yet they typically rely on positional encodings that assume uniform, index-based temporal pr…
Free Energy Mixer
Jiecheng Lu, Shihao Yang
Standard attention stores keys/values losslessly but reads them via a per-head convex average, blocking channel-wise selection. We propose the Free Energy Mixer (FEM): a free-energ…
ZeroS: Zero-Sum Linear Attention for Efficient Transformers
Jiecheng Lu, Xu Han, Yan Sun +4
Linear attention methods offer Transformers complexity but typically underperform standard softmax attention. We identify two fundamental limitations affecting these approac…
CAPS: Unifying Attention, Recurrence, and Alignment in Transformer-based Time Series Forecasting
Viresh Pati, Yubin Kim, Vinh Pham +3
This paper presents (Clock-weighted Aggregation with Prefix-products and Softmax), a structured attention mechanism for time series forecasting that decouples three…
Linear Transformers as VAR Models: Aligning Autoregressive Attention Mechanisms with Autoregressive Forecasting
Jiecheng Lu, Shihao Yang
Autoregressive attention-based time series forecasting (TSF) has drawn increasing interest, with mechanisms like linear attention sometimes outperforming vanilla attention. However…
WAVE: Weighted Autoregressive Varying Gate for Time Series Forecasting
Jiecheng Lu, Xu Han, Yan Sun +1
We propose a Weighted Autoregressive Varying gatE (WAVE) attention mechanism equipped with both Autoregressive (AR) and Moving-average (MA) components. It can adapt to various atte…
In-context Time Series Predictor
Jiecheng Lu, Yan Sun, Shihao Yang
Recent Transformer-based large language models (LLMs) demonstrate in-context learning ability to perform various functions based solely on the provided context, without updating mo…
CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables
Jiecheng Lu, Xu Han, Yan Sun +1
For Multivariate Time Series Forecasting (MTSF), recent deep learning applications show that univariate models frequently outperform multivariate ones. To address the difficiency i…
ARM: Refining Multivariate Forecasting with Adaptive Temporal-Contextual Learning
Jiecheng Lu, Xu Han, Shihao Yang
Long-term time series forecasting (LTSF) is important for various domains but is confronted by challenges in handling the complex temporal-contextual relationships. As multivariate…