3 papers
cs.LG2025
Quantize Once, Train Fast: Allreduce-Compatible Compression with Provable Guarantees
Jihao Xin, Marco Canini, Peter Richtárik +1
Distributed training enables large-scale deep learning, but suffers from high communication overhead, especially as models and datasets grow. Gradient compression, particularly qua…
cs.LG2025
Practical Insights into Knowledge Distillation for Pre-Trained Models
Norah Alballa, Ahmed M. Abdelmoniem, Marco Canini
This research investigates the enhancement of knowledge distillation (KD) processes in pre-trained models, an emerging field in knowledge transfer with significant implications for…
cs.LG2025
Query-based Knowledge Transfer for Heterogeneous Learning Environments
Norah Alballa, Wenxuan Zhang, Ziquan Liu +3
Decentralized collaborative learning under data heterogeneity and privacy constraints has rapidly advanced. However, existing solutions like federated learning, ensembles, and tran…