4 papers · 1 filter
Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning
Dongkwan Lee, Junhoo Lee, Nojun Kwak
We introduce the Deep Edge Filter, a novel approach that applies high-pass filtering to deep neural network features to improve model generalizability. Our method is motivated by o…
The Role of Teacher Calibration in Knowledge Distillation
Suyoung Kim, Seonguk Park, Junhoo Lee +1
Knowledge Distillation (KD) has emerged as an effective model compression technique in deep learning, enabling the transfer of knowledge from a large teacher model to a compact stu…
Deep Support Vectors
Junhoo Lee, Hyunho Lee, Kyomin Hwang +1
Deep learning has achieved tremendous success. However, unlike SVMs, which provide direct decision criteria and can be trained with a small dataset, it still has significant weakne…
Practical Dataset Distillation Based on Deep Support Vectors
Hyunho Lee, Junhoo Lee, Nojun Kwak
Conventional dataset distillation requires significant computational resources and assumes access to the entire dataset, an assumption impractical as it presumes all data resides o…