559 citations · 811 across the 14 of their papers we have counts for
9 papers · 1 filter
SeedFlood: A Step Toward Scalable Decentralized Fine-Tuning of LLMs
Jihun Kim, Namhoon Lee, Dongyeop Lee
This work presents SeedFlood, a new approach to decentralized LLM fine-tuning designed to scale across large models, large collaborations, and complex network topologies while achi…
MemEIC: A Step Toward Continual and Compositional Knowledge Editing
Jin Seong, Jiyun Park, Wencke Liermann +5
The dynamic nature of information necessitates continuously updating large vision-language models (LVLMs). While recent knowledge editing techniques hint at promising directions, t…
JaxPruner: A concise library for sparsity research
Joo Hyung Lee, Wonpyo Park, Nicole Mitchell +22
This paper introduces JaxPruner, an open-source JAX-based pruning and sparse training library for machine learning research. JaxPruner aims to accelerate research on sparse neural…
SpReME: Sparse Regression for Multi-Environment Dynamic Systems
MoonJeong Park, Youngbin Choi, Namhoon Lee +1
Learning dynamical systems is a promising avenue for scientific discoveries. However, capturing the governing dynamics in multiple environments still remains a challenge: model-bas…
A Closer Look at the Intervention Procedure of Concept Bottleneck Models
Sungbin Shin, Yohan Jo, Sungsoo Ahn +1
Concept bottleneck models (CBMs) are a class of interpretable neural network models that predict the target response of a given input based on its high-level concepts. Unlike the s…
The Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformers
Seungwoo Son, Jegwang Ryu, Namhoon Lee +1
Knowledge distillation is an effective method for training lightweight vision models. However, acquiring teacher supervision for training samples is often costly, especially from l…
Meta-Learning Sparse Implicit Neural Representations
Jaeho Lee, Jihoon Tack, Namhoon Lee +1
Implicit neural representations are a promising new avenue of representing general signals by learning a continuous function that, parameterized as a neural network, maps the domai…
Understanding the Effects of Data Parallelism and Sparsity on Neural Network Training
Namhoon Lee, Thalaiyasingam Ajanthan, Philip H. S. Torr +1
We study two factors in neural network training: data parallelism and sparsity; here, data parallelism means processing training data in parallel using distributed systems (or equi…
A Signal Propagation Perspective for Pruning Neural Networks at Initialization
Namhoon Lee, Thalaiyasingam Ajanthan, Stephen Gould +1
Network pruning is a promising avenue for compressing deep neural networks. A typical approach to pruning starts by training a model and then removing redundant parameters while mi…