2 papers
cs.LG2026
Investigating the Robustness of Subtask Distillation under Spurious Correlation
Pattarawat Chormai, Klaus-Robert Müller, Grégoire Montavon
Subtask distillation is an emerging paradigm in which compact, specialized models are extracted from large, general-purpose 'foundation models' for deployment in environments with…
cs.LG2026
Distilling Lightweight Domain Experts from Large ML Models by Identifying Relevant Subspaces
Pattarawat Chormai, Ali Hashemi, Klaus-Robert Müller +1
Knowledge distillation involves transferring the predictive capabilities of large, high-performing AI models (teachers) to smaller models (students) that can operate in environment…