Neighbourhood Distillation: On the benefits of non end-to-end distillation

Laëtitia Shao, Max Moroz, Elad Eban +1

End-to-end training with back propagation is the standard method for training deep neural networks. However, as networks become deeper and bigger, end-to-end training becomes more…