5 papers
Block Recursive Transformers for Structured Parameter Sharing
Josep Marques, Hyunyoung A. Cho, Youngeun Kim +4
Recursive Transformers depend not only on how many parameters are shared, but also on how shared parameters are organized across depth. Existing static schemes adopt two extreme re…
Dynamic Rank Adjustment for Accurate and Efficient Neural Network Training
Hyuntak Shin, Aecheon Jung, Sungeun Hong +1
Low-rank training methods reduce the number of trainable parameters by re-parameterizing the weights with matrix decompositions (e.g., singular value decomposition). However, enfor…
Task Vector Quantization for Memory-Efficient Model Merging
Youngeun Kim, Seunghwan Lee, Aecheon Jung +2
Model merging enables efficient multi-task models by combining task-specific fine-tuned checkpoints. However, storing multiple task-specific checkpoints requires significant memory…
IAM: Enhancing RGB-D Instance Segmentation with New Benchmarks
Aecheon Jung, Soyun Choi, Junhong Min +1
Image segmentation is a vital task for providing human assistance and enhancing autonomy in our daily lives. In particular, RGB-D segmentation-leveraging both visual and depth cues…
SyMerge: From Non-Interference to Synergistic Merging via Single-Layer Adaptation
Aecheon Jung, Seunghwan Lee, Dongyoon Han +1
Model merging combines independently trained models into a single multi-task model. However, most existing approaches focus primarily on avoiding task interference. We argue that i…