3 papers
cs.LG2024
Improving Quantization-aware Training of Low-Precision Network via Block Replacement on Full-Precision Counterpart
Chengting Yu, Shu Yang, Fengzhao Zhang +3
Quantization-aware training (QAT) is a common paradigm for network quantization, in which the training phase incorporates the simulation of the low-precision computation to optimiz…
cs.LG2024
Decoupling Dark Knowledge via Block-wise Logit Distillation for Feature-level Alignment
Chengting Yu, Fengzhao Zhang, Ruizhe Chen +4
Knowledge Distillation (KD), a learning manner with a larger teacher network guiding a smaller student network, transfers dark knowledge from the teacher to the student via logits…
cs.LG2023
Go beyond End-to-End Training: Boosting Greedy Local Learning with Context Supply
Chengting Yu, Fengzhao Zhang, Hanzhi Ma +2
Traditional end-to-end (E2E) training of deep networks necessitates storing intermediate activations for back-propagation, resulting in a large memory footprint on GPUs and restric…