4 citations · 11 across the 18 of their papers we have counts for
9 papers · 1 filter
The Bridge-Garden Dilemma in LLM Distillation: Why Mixing Hard and Soft Labels Works
Guanghui Wang, Kaiwen Lv Kacuila, Zhiyong Yang +5
Knowledge distillation (KD) transfers knowledge from a large teacher model to a smaller student. In language modeling, the student is trained either on tokens sampled from the teac…
ABKD: Pursuing a Proper Allocation of the Probability Mass in Knowledge Distillation via --Divergence
Guanghui Wang, Zhiyong Yang, Zitai Wang +3
Knowledge Distillation (KD) transfers knowledge from a large teacher model to a smaller student model by minimizing the divergence between their output distributions, typically usi…
Focal-SAM: Focal Sharpness-Aware Minimization for Long-Tailed Classification
Sicong Li, Qianqian Xu, Zhiyong Yang +4
Real-world datasets often follow a long-tailed distribution, making generalization to tail classes difficult. Recent methods resorted to long-tail variants of Sharpness-Aware Minim…
OpenworldAUC: Towards Unified Evaluation and Optimization for Open-world Prompt Tuning
Cong Hua, Qianqian Xu, Zhiyong Yang +3
Prompt tuning adapts Vision-Language Models like CLIP to open-world tasks with minimal training costs. In this direction, one typical paradigm evaluates model performance separatel…
HGOE: Hybrid External and Internal Graph Outlier Exposure for Graph Out-of-Distribution Detection
Junwei He, Qianqian Xu, Yangbangyan Jiang +3
With the progressive advancements in deep graph learning, out-of-distribution (OOD) detection for graph data has emerged as a critical challenge. While the efficacy of auxiliary da…
Top-K Pairwise Ranking: Bridging the Gap Among Ranking-Based Measures for Multi-Label Classification
Zitai Wang, Qianqian Xu, Zhiyong Yang +4
Multi-label ranking, which returns multiple top-ranked labels for each instance, has a wide range of applications for visual tasks. Due to its complicated setting, prior arts have…