2 papers
cs.LG2025
SeWA: Selective Weight Average via Probabilistic Masking
Peng Wang, Shengchao Hu, Zerui Tao +5
Weight averaging has become a standard technique for enhancing model performance. However, methods such as Stochastic Weight Averaging (SWA) and Latest Weight Averaging (LAWA) ofte…
cs.LG2024
A Unified Analysis for Finite Weight Averaging
Peng Wang, Li Shen, Zerui Tao +3
Averaging iterations of Stochastic Gradient Descent (SGD) have achieved empirical success in training deep learning models, such as Stochastic Weight Averaging (SWA), Exponential M…