Behind the Scenes of Gradient Descent: A Trajectory Analysis via Basis Function Decomposition

Jianhao Ma, Lingjun Guo, Salar Fattahi

This work analyzes the solution trajectory of gradient-based algorithms via a novel basis function decomposition. We show that, although solution trajectories of gradient-based alg…