Soft Merging: A Flexible and Robust Soft Model Merging Approach for Enhanced Neural Network Performance

Hao Chen, Yusen Wu, Phuong Nguyen +2

Stochastic Gradient Descent (SGD), a widely used optimization algorithm in deep learning, is often limited to converging to local optima due to the non-convex nature of the problem…