5 papers
Learning Neural Networks by Neuron Pursuit
Akshay Kumar, Jarvis Haupt
The first part of this paper studies the evolution of gradient flow for homogeneous neural networks near a class of saddle points exhibiting a sparsity structure. The choice of the…
Deploying AI for Signal Processing education: Selected challenges and intriguing opportunities
Jarvis Haupt, Qin Lu, Yanning Shen +5
Powerful artificial intelligence (AI) tools that have emerged in recent years -- including large language models, automated coding assistants, and advanced image and speech generat…
Towards Understanding Gradient Flow Dynamics of Homogeneous Neural Networks Beyond the Origin
Akshay Kumar, Jarvis Haupt
Recent works exploring the training dynamics of homogeneous neural network weights under gradient flow with small initialization have established that in the early stages of traini…
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations
Akshay Kumar, Jarvis Haupt
This paper studies the gradient flow dynamics that arise when training deep homogeneous neural networks assumed to have locally Lipschitz gradients and an order of homogeneity stri…
Directional Convergence Near Small Initializations and Saddles in Two-Homogeneous Neural Networks
Akshay Kumar, Jarvis Haupt
This paper examines gradient flow dynamics of two-homogeneous neural networks for small initializations, where all weights are initialized near the origin. For both square and logi…