Brain encoding models based on multimodal transformers can transfer across language and vision

Jerry Tang, Meng Du, Vy A. Vo +2

Encoding models have been used to assess how the human brain represents concepts in language and vision. While language and vision rely on similar concept representations, current…