2 citations · 4 across the 9 of their papers we have counts for
11 papers
Gated Slot Attention-2: Two-Sided Associative Memory Correction in Linear Attention
Ruijie Li, Shengnan Ding, Weimin Zhang +6
Linear attention models have emerged as efficient alternatives to standard attention, but effectively managing their fixed-size recurrent memory remains challenging. To improve mem…
Native Hybrid Attention for Efficient Sequence Modeling
Jusen Du, Jiaxi Hu, Tao Zhang +2
Transformers excel at sequence modeling but face quadratic complexity, while linear attention offers improved efficiency but often compromises recall accuracy over long contexts. I…
Speed Always Wins: A Survey on Efficient Architectures for Large Language Models
Weigao Sun, Jiaxi Hu, Yucheng Zhou +12
Large Language Models (LLMs) have delivered impressive results in language understanding, generation, reasoning, and pushes the ability boundary of multimodal models. Transformer m…
Comba: Improving Bilinear RNNs with Closed-loop Control
Jiaxi Hu, Yongqi Pan, Jusen Du +5
Recent efficient sequence modeling methods such as Gated DeltaNet, TTT, and RWKV-7 have achieved performance improvements by supervising the recurrent memory management through Del…
Multi-Order Wavelet Derivative Transform for Deep Time Series Forecasting
Ziyu Zhou, Jiaxi Hu, Qingsong Wen +2
In deep time series forecasting, the Fourier Transform (FT) is extensively employed for frequency representation learning. However, it often struggles in capturing multi-scale, tim…
Liger: Linearizing Large Language Models to Gated Recurrent Structures
Disen Lan, Weigao Sun, Jiaxi Hu +2
Transformers with linear recurrent modeling offer linear-time training and constant-memory inference. Despite their demonstrated efficiency and performance, pretraining such non-st…
MoM: Linear Sequence Modeling with Mixture-of-Memories
Jusen Du, Weigao Sun, Disen Lan +2
Linear sequence modeling methods, such as linear attention, state space modeling, and linear RNNs, offer significant efficiency improvements by reducing the complexity of training…
Toward Physics-guided Time Series Embedding
Jiaxi Hu, Bowen Zhang, Qingsong Wen +2
In various scientific and engineering fields, the primary research areas have revolved around physics-based dynamical systems modeling and data-driven time series analysis. Accordi…
TwinS: Revisiting Non-Stationarity in Multivariate Time Series Forecasting
Jiaxi Hu, Qingsong Wen, Sijie Ruan +2
Recently, multivariate time series forecasting tasks have garnered increasing attention due to their significant practical applications, leading to the emergence of various deep fo…
Time-SSM: Simplifying and Unifying State Space Models for Time Series Forecasting
Jiaxi Hu, Disen Lan, Ziyu Zhou +2
State Space Models (SSMs) have emerged as a potent tool in sequence modeling tasks in recent years. These models approximate continuous systems using a set of basis functions and d…
Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective
Jiaxi Hu, Yuehong Hu, Wei Chen +4
In long-term time series forecasting (LTSF) tasks, an increasing number of models have acknowledged that discrete time series originate from continuous dynamic systems and have att…