3 papers
cs.LG2026
Misalignment of Low-Loss Regions Causes Grokking
Yongding Tian, Zaid Al-Ars, Maksim Kitsak +1
Grokking refers to the delayed emergence of validation-set generalization after a model has already overfit the training set. Although first observed in small algorithmic tasks tra…
cs.LG2025
Connecting Independently Trained Modes via Layer-Wise Connectivity
Yongding Tian, Zaid Al-Ars, Maksim Kitsak +1
Empirical studies have shown that continuous low-loss paths can be constructed between independently trained neural network models. This phenomenon, known as mode connectivity, ref…
cs.LG2024
Vanishing Variance Problem in Fully Decentralized Neural-Network Systems
Yongding Tian, Zaid Al-Ars, Maksim Kitsak +1
Federated learning and gossip learning are emerging methodologies designed to mitigate data privacy concerns by retaining training data on client devices and exclusively sharing lo…