3 papers
cs.LG2026
BicKD: Bilateral Contrastive Knowledge Distillation
Jiangnan Zhu, Yukai Xu, Li Xiong +4
Knowledge distillation (KD) is a machine learning framework that transfers knowledge from a teacher model to a student model. The vanilla KD proposed by Hinton et al. has been the…
cs.LG2026
Gradient Compression May Hurt Generalization: A Remedy by Synthetic Data Guided Sharpness Aware Minimization
Yujie Gu, Richeng Jin, Zhaoyang Zhang +1
It is commonly believed that gradient compression in federated learning (FL) enjoys significant improvement in communication efficiency with negligible performance degradation. In…
cs.CR2026
Differentially Private and Communication Efficient Large Language Model Split Inference via Stochastic Quantization and Soft Prompt
Yujie Gu, Richeng Jin, Xiaoyu Ji +2
Large Language Models (LLMs) have achieved remarkable performance and received significant research interest. The enormous computational demands, however, hinder the local deployme…