Simplifying Neural Networks During Training

Lorenzo Sciandra, Samuele Fonio, Roberto Esposito

The paper proposes a training framework that monitors representation dynamics using the Inverse Fisher Criterion to identify when and where to replace later layers of a deep networ…