Locally Optimal Descent for Dynamic Stepsize Scheduling

Gilad Yehudai, Alon Cohen, Amit Daniely +3

We introduce a novel dynamic learning-rate scheduling scheme grounded in theory with the goal of simplifying the manual and time-consuming tuning of schedules in practice. Our appr…