Stable Anderson Acceleration for Deep Learning

Massimiliano Lupo Pasini, Junqi Yin, Viktor Reshniak +1

Anderson acceleration (AA) is an extrapolation technique designed to speed-up fixed-point iterations like those arising from the iterative training of DL models. Training DL models…