Multimodal Neurons in Pretrained Text-Only Transformers

Sarah Schwettmann, Neil Chowdhury, Samuel Klein +2

Language models demonstrate remarkable capacity to generalize representations learned in one modality to downstream tasks in other modalities. Can we trace this ability to individu…