From the 1 of 46 linked papers with an AI index.
7 papers · 1 filter
Zero-Shot Quantization for Object Detectors using Off-the-Shelf Generative Models
Hyunho Lee, Kyomin Hwang, Hyeonjin Kim +3
The paper proposes GoodQ, a method that uses off-the-shelf generative models to create synthetic training data for zero-shot quantization of object detectors, enabling low-bit quan…
Toward Structural Multimodal Representations: Specialization, Selection, and Sparsification via Mixture-of-Experts
Hahyeon Choi, Nojun Kwak
We propose S3 (Specialization, Selection, Sparsification), a framework that rethinks multimodal learning through a structural perspective. Instead of encoding all signals into a fi…
Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning
Dongkwan Lee, Junhoo Lee, Nojun Kwak
We introduce the Deep Edge Filter, a novel approach that applies high-pass filtering to deep neural network features to improve model generalizability. Our method is motivated by o…
Understanding Differential Transformer Unchains Pretrained Self-Attentions
Chaerin Kong, Jiho Jang, Nojun Kwak
Differential Transformer has recently gained significant attention for its impressive empirical performance, often attributed to its ability to perform noise canceled attention. Ho…
The Role of Teacher Calibration in Knowledge Distillation
Suyoung Kim, Seonguk Park, Junhoo Lee +1
Knowledge Distillation (KD) has emerged as an effective model compression technique in deep learning, enabling the transfer of knowledge from a large teacher model to a compact stu…
Deep Support Vectors
Junhoo Lee, Hyunho Lee, Kyomin Hwang +1
Deep learning has achieved tremendous success. However, unlike SVMs, which provide direct decision criteria and can be trained with a small dataset, it still has significant weakne…