adaptive coefficient regulation 1feature decorrelation 1heterogeneous models 1knowledge distillation 1redundancy suppression 1
From the 1 of 2 linked papers with an AI index.
2 papers
cs.LG2026
CoCaRS: Correlation Calibration-Based Redundancy Suppression for Heterogeneous Knowledge Distillation
Fengming Yu, Haiwei Pan, Kejia Zhang +3
The paper introduces CoCaRS, a method for heterogeneous knowledge distillation that calibrates feature decorrelation to suppress redundancy while preserving structural information,…
cs.CV2025
UHKD: A Unified Framework for Heterogeneous Knowledge Distillation via Frequency-Domain Representations
Fengming Yu, Haiwei Pan, Kejia Zhang +2
Knowledge distillation (KD) is an effective model compression technique that transfers knowledge from a high-performance teacher to a lightweight student, reducing computational an…