5 papers
Brain alignment of reasoning and action representations from vision-language and action models during naturalistic gameplay
Subba Reddy Oota, Anant Khandelwal, Khushbu Pahwa +4
Understanding how humans and artificial intelligence systems predict and plan by interacting with their environment is a fundamental challenge at the intersection of neuroscience a…
Task-conditioned probing of instruction-tuned multimodal LLMs: Region-specific brain alignment patterns under naturalistic stimuli
Subba Reddy Oota, Khushbu Pahwa, Prachi Jindal +5
Recent voxel-wise multimodal brain encoding studies have shown that multimodal large language models (MLLMs) exhibit a higher degree of brain alignment compared to unimodal models.…
Linguistic properties and model scale in brain encoding: from small to compressed language models
Subba Reddy Oota, Vijay Rowtula, Satya Sai Srinath Namburi +5
Recent work has shown that scaling large language models (LLMs) improves their alignment with human brain activity, yet it remains unclear what drives these gains and which represe…
Pretrained Hybrids with MAD Skills
Nicholas Roberts, Samuel Guo, Zhiqi Gao +5
While Transformers underpin modern large language models (LMs), there is a growing list of alternative architectures with new capabilities, promises, and tradeoffs. This makes choo…
Correlating instruction-tuning (in multimodal models) with vision-language processing (in the brain)
Subba Reddy Oota, Akshett Jindal, Ishani Mondal +6
Transformer-based language models, though not explicitly trained to mimic brain recordings, have demonstrated surprising alignment with brain activity. Progress in these models-thr…