RMT-KD: Random Matrix Theoretic Causal Knowledge Distillation

Davide Ettori, Nastaran Darabi, Sureshkumar Senthilkumar +1

Large deep learning models such as BERT and ResNet achieve state-of-the-art performance but are costly to deploy at the edge due to their size and compute demands. We present RMT-K…