Pretraining a Neural Network before Knowing Its Architecture

Boris Knyazev

Training large neural networks is possible by training a smaller hypernetwork that predicts parameters for the large ones. A recently released Graph HyperNetwork (GHN) trained this…