555 citations · 2.5k across the 105 of their papers we have counts for
3 papers · 2 filters
Demystifying Oversmoothing in Attention-Based Graph Neural Networks
Xinyi Wu, Amir Ajorlou, Zihui Wu +1
Oversmoothing in Graph Neural Networks (GNNs) refers to the phenomenon where increasing network depth leads to homogeneous node representations. While previous work has established…
How to escape sharp minima with random perturbations
Kwangjun Ahn, Ali Jadbabaie, Suvrit Sra
Modern machine learning applications have witnessed the remarkable success of optimization algorithms that are designed to find flat minima. Motivated by this design choice, we und…
Variance-reduced Clipping for Non-convex Optimization
Amirhossein Reisizadeh, Haochuan Li, Subhro Das +1
Gradient clipping is a standard training technique used in deep learning applications such as large-scale language modeling to mitigate exploding gradients. Recent experimental stu…