Publications (4)
UHKD: A Unified Framework for Heterogeneous Knowledge Distillation via Frequency-Domain Representations
Fengming Yu, Haiwei Pan, Kejia Zhang +2
Knowledge distillation (KD) is an effective model compression technique that transfers knowledge from a high-performance teacher to a lightweight student, reducing computational an…
CoCaRS: Correlation Calibration-Based Redundancy Suppression for Heterogeneous Knowledge Distillation
Fengming Yu, Haiwei Pan, Kejia Zhang +3
The paper introduces CoCaRS, a method for heterogeneous knowledge distillation that calibrates feature decorrelation to suppress redundancy while preserving structural information,…
MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusion
Zhe Li, Haiwei Pan, Kejia Zhang +2
Multi-modality image fusion (MMIF) aims to integrate complementary information from different modalities into a single fused image to represent the imaging scene and facilitate dow…
Efficient Mathematical Reasoning Models via Dynamic Pruning and Knowledge Distillation
Fengming Yu, Qingyu Meng, Haiwei Pan +1
With the rapid development of deep learning, large language models have shown strong capabilities in complex reasoning tasks such as mathematical equation solving. However, their s…