5 citations · 14 across the 15 of their papers we have counts for
4 papers · 1 filter
Modern Transformers Are Implicit Hybrids: From Functional Differentiation to Principled Hybrid Architecture Design
Runlin Shi, Bojian Yin, Guoqi Li
Hybrid architectures combining Full Attention (FA) and Linear Attention (LA) are increasingly prominent, yet their allocation remains heuristic. We seek an evidence-grounded basis…
Full-Stack FP4: Stable LLM Pretraining with Quantized Projections, Optimizers, and Attention
Siyu Ding, Mingchuan Ma, Jiabo Tong +3
Recent NVFP4 pretraining work has primarily optimized Transformer linear projections, leaving persistent optimizer states, optimizer computation, and low-precision attention forwar…
NeuroCogMap Reveals Cognitive Organization of Large Language Models
Zhongxiang Sun, Haolang Lu, Qiang Ma +11
Understanding how complex cognitive functions are organized within artificial systems is central to interpreting large language models (LLMs) and relating them to biological cognit…
Efficient Sparse Selective-Update RNNs for Long-Range Sequence Modeling
Bojian Yin, Shurong Wang, Haoyu Tan +3
Real-world sequential signals, such as audio or video, contain critical information that is often embedded within long periods of silence or noise. While recurrent neural networks…