Toward Structural Multimodal Representations: Specialization, Selection, and Sparsification via Mixture-of-Experts

Hahyeon Choi, Nojun Kwak

We propose S3 (Specialization, Selection, Sparsification), a framework that rethinks multimodal learning through a structural perspective. Instead of encoding all signals into a fi…