FOSI: Hybrid First and Second Order Optimization

Hadar Sivan, Moshe Gabel, Assaf Schuster

Popular machine learning approaches forgo second-order information due to the difficulty of computing curvature in high dimensions. We present FOSI, a novel meta-algorithm that imp…