adaptive coefficient regulation 1feature decorrelation 1heterogeneous models 1knowledge distillation 1redundancy suppression 1
From the 1 of 3 linked papers with an AI index.
3 papers
cs.LG2026
CoCaRS: Correlation Calibration-Based Redundancy Suppression for Heterogeneous Knowledge Distillation
Fengming Yu, Haiwei Pan, Kejia Zhang +3
The paper introduces CoCaRS, a method for heterogeneous knowledge distillation that calibrates feature decorrelation to suppress redundancy while preserving structural information,…
cs.LG2025
Efficient Mathematical Reasoning Models via Dynamic Pruning and Knowledge Distillation
Fengming Yu, Qingyu Meng, Haiwei Pan +1
With the rapid development of deep learning, large language models have shown strong capabilities in complex reasoning tasks such as mathematical equation solving. However, their s…
cs.CV2025
UHKD: A Unified Framework for Heterogeneous Knowledge Distillation via Frequency-Domain Representations
Fengming Yu, Haiwei Pan, Kejia Zhang +2
Knowledge distillation (KD) is an effective model compression technique that transfers knowledge from a high-performance teacher to a lightweight student, reducing computational an…