Loss Patterns of Neural Networks

Ivan Skorokhodov, Mikhail Burtsev

We present multi-point optimization: an optimization technique that allows to train several models simultaneously without the need to keep the parameters of each one individually.…