3 citations · 5 across the 12 of their papers we have counts for
7 papers · 1 filter
N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models
Donghoon Han, Minseop Jung, Dong Woo Kwak +2
Conditional memory expands language-model capacity through selective lookup, but the full embedding table must remain available for every possible input. We introduce Conditional P…
Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning
Dongkwan Lee, Junhoo Lee, Nojun Kwak
We introduce the Deep Edge Filter, a novel approach that applies high-pass filtering to deep neural network features to improve model generalizability. Our method is motivated by o…
The Role of Teacher Calibration in Knowledge Distillation
Suyoung Kim, Seonguk Park, Junhoo Lee +1
Knowledge Distillation (KD) has emerged as an effective model compression technique in deep learning, enabling the transfer of knowledge from a large teacher model to a compact stu…
Practical Dataset Distillation Based on Deep Support Vectors
Hyunho Lee, Junhoo Lee, Nojun Kwak
Conventional dataset distillation requires significant computational resources and assumes access to the entire dataset, an assumption impractical as it presumes all data resides o…
Deep Support Vectors
Junhoo Lee, Hyunho Lee, Kyomin Hwang +1
Deep learning has achieved tremendous success. However, unlike SVMs, which provide direct decision criteria and can be trained with a small dataset, it still has significant weakne…
Any-Way Meta Learning
Junhoo Lee, Yearim Kim, Hyunho Lee +1
Although meta-learning seems promising performance in the realm of rapid adaptability, it is constrained by fixed cardinality. When faced with tasks of varying cardinalities that w…
SHOT: Suppressing the Hessian along the Optimization Trajectory for Gradient-Based Meta-Learning
JunHoo Lee, Jayeon Yoo, Nojun Kwak
In this paper, we hypothesize that gradient-based meta-learning (GBML) implicitly suppresses the Hessian along the optimization trajectory in the inner loop. Based on this hypothes…