Weight Squeezing: Reparameterization for Knowledge Transfer and Model Compression

Artem Chumachenko, Daniil Gavrilov, Nikita Balagansky +1

In this work, we present a novel approach for simultaneous knowledge transfer and model compression called Weight Squeezing. With this method, we perform knowledge transfer from a…