3 papers
cs.LG2026
Accelerating LLM Pre-Training through Flat-Direction Dynamics Enhancement
Shuchen Zhu, Rizhen Hu, Mingze Wang +4
Pre-training Large Language Models requires immense computational resources, making optimizer efficiency essential. The optimization landscape is highly anisotropic, with loss redu…
cs.AI2025
SimDiff: Simpler Yet Better Diffusion Model for Time Series Point Forecasting
Hang Ding, Xue Wang, Tian Zhou +1
Diffusion models have recently shown promise in time series forecasting, particularly for probabilistic predictions. However, they often fail to achieve state-of-the-art point esti…
cs.LG2025
MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
Yuxi Liu, Renjia Deng, Yutong He +3
The substantial memory demands of pre-training and fine-tuning large language models (LLMs) require memory-efficient optimization algorithms. One promising approach is layer-wise o…