Trainable Weight Averaging: Accelerating Training and Improving Generalization

Tao Li, Zhehao Huang, Yingwen Wu +4

Weight averaging is a widely used technique for accelerating training and improving the generalization of deep neural networks (DNNs). While existing approaches like stochastic wei…