The Expected Loss of Preconditioned Langevin Dynamics Reveals the Hessian Rank

Amitay Bar, Rotem Mulayoff, Tomer Michaeli +1

Langevin dynamics (LD) is widely used for sampling from distributions and for optimization. In this work, we derive a closed-form expression for the expected loss of preconditioned…