3 papers
cs.LG2026
LongSpike: Fractional Order Spiking State Space Models for Efficient Long Sequence Learning
Xinrui He, Qiyu Kang, Xuhao Li +1
Spiking Neural Networks (SNNs) are well-regarded for their biological plausibility and energy efficiency in processing sequential data. However, dominant SNN architectures typicall…
cs.LG2026
Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts
Ruizhe Wang, Yucheng Ding, Xiao Liu +5
As the computational demands for pre-training Large Language Models (LLMs) continue to surge, the need for efficient training paradigms becomes critical. Despite the vast resources…
cs.LG2025
Optimizing Large Language Model Training Using FP4 Quantization
Ruizhe Wang, Yeyun Gong, Xiao Liu +5
The growing computational demands of training large language models (LLMs) necessitate more efficient methods. Quantized training presents a promising solution by enabling low-bit…