4 papers · 1 filter
Diverse Image Priors for Black-box Data-free Knowledge Distillation
Tri-Nhan Vo, Dang Nguyen, Trung Le +2
Knowledge distillation (KD) represents a vital mechanism to transfer expertise from complex teacher networks to efficient student models. However, in decentralized or secure AI eco…
Sharpness-Aware Minimization in Logit Space Efficiently Enhances Direct Preference Optimization
Haocheng Luo, Zehang Deng, Thanh-Toan Do +3
Direct Preference Optimization (DPO) has emerged as a popular algorithm for aligning pretrained large language models with human preferences, owing to its simplicity and training s…
Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influence
Quoc Minh Nguyen, Trung Le, Jing Wu +2
Fine-tuning-as-a-service introduces a threat to Large Language Models' safety when service providers fine-tune their models on poisoned user-submitted datasets, a process known as…
Exemplar-Free Continual Learning for State Space Models
Isaac Ning Lee, Leila Mahmoodi, Trung Le +1
State-Space Models (SSMs) excel at capturing long-range dependencies with structured recurrence, making them well-suited for sequence modeling. However, their evolving internal sta…