Virtual Encoders in Multimodal Transformers

Katsuya Ogata, Yuta Nakashima

Multimodal language models traditionally rely on dedicated perceptual encoders to construct task-usable representations. More integrated architectures have recently emerged, which…