7 papers
Knowledge Cascade: Reverse Knowledge Distillation on Nonparametric Multivariate Functional Estimation
Luyang Fang, Haoran Lu, Yongkai Chen +2
As machine learning models and datasets continue to grow, developing complex models has become increasingly computationally demanding. Knowledge distillation reduces deployment cos…
Multi-Teacher Knowledge Distillation via Teacher-Informed Mixture Priors
Luyang Fang, Yongkai Chen, Jiazhang Cai +2
Knowledge distillation is a powerful method for model compression, enabling the efficient deployment of complex deep learning models (teachers), including large language models. Ho…
Wahkon: A Statistically Principled Deep RKHS Superposition Network
Yongkai Chen, Wenxuan Zhong, Ping Ma
Deep learning excels at prediction but often lacks finite-sample guarantees and calibrated uncertainty; RKHS (Reproducing Kernel Hilbert Space)-based methods provide those guarante…
Quantum Statistical Bootstrap
Yongkai Chen, Ping Ma, Wenxuan Zhong
The bootstrap is a foundational tool in statistical inference, but its classical implementation relies on Monte Carlo resampling, introducing approximation error and incurring high…
Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
Luyang Fang, Xiaowei Yu, Jiazhang Cai +23
The exponential growth of Large Language Models (LLMs) continues to highlight the need for efficient strategies to meet ever-expanding computational and data demands. This survey p…
Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
Haoran Lu, Luyang Fang, Ruidong Zhang +47
Due to the remarkable capabilities and growing impact of large language models (LLMs), they have been deeply integrated into many aspects of society. Thus, ensuring their alignment…