SQuaT: Self-Supervised Knowledge Distillation via Student-Aware Quantized Teacher Features

HyeonJun Lee, Hyeonsik Jo, Jinwoo Chung +1

Quantization-Aware Training (QAT) enables the deployment of quantized models with minimal accuracy degradation. However, in practical scenarios, training labels are often unavailab…