2 papers
cs.CV2026
Enhancing Mixture-of-Experts Specialization via Cluster-Aware Upcycling
Sanghyeok Chu, Pyunghwan Ahn, Gwangmo Song +3
Sparse Upcycling provides an efficient way to initialize a Mixture-of-Experts (MoE) model from pretrained dense weights instead of training from scratch. However, since all experts…
cs.CV2024
Exploring the Spectrum of Visio-Linguistic Compositionality and Recognition
Youngtaek Oh, Pyunghwan Ahn, Jinhyung Kim +4
Vision and language models (VLMs) such as CLIP have showcased remarkable zero-shot recognition abilities yet face challenges in visio-linguistic compositionality, particularly in l…