140 citations · 359 across the 24 of their papers we have counts for
26 papers
SCRIBE: Scalable Contextual Representations of Interpretable Brain-Grounded Semantics
Camila Kolling, Omer Moussa, Emin Çelik +1
Understanding how the human brain constructs meaning from real-world language remains a fundamental challenge, as semantics depend on context that varies in natural settings. To en…
Decomposing Brain-LLM Alignment with Interpretable Semantic Representations
Camila Kolling, Mariya Toneva
Large language models (LLMs) predict neural responses during naturalistic language comprehension, but the linguistic information underlying this brain alignment remains difficult t…
RABBiT: Rapidly adaptive BOLD foundation model via brain-tuning for accurate zero-shot and few-shot prediction of speech-elicited responses in the brain
Omer Moussa, Mariya Toneva
Language understanding in the brain is context-dependent, varying across experimental stimuli and individuals, which makes it difficult to build computational models that generaliz…
Neural Signatures of Programming Expertise: Classifying Programmer Skill Levels Using EEG Data
Maurice Rekrut, Mahima Mahabaleshwar Acharya, Taisiia Ulianova +5
Accurately assessing a programmer's skill level is critical for hiring, team composition, and performance evaluation in the software industry. Conventional methods, such as coding…
Temporal Context Reinstatement Drives Episodic-Like Order Memory in Long-Context Language Models
Mathis Pink, Vy Ai Vo, Qinyuan Wu +7
Human episodic memory supports the retrieval of experiences that unfold over extended timescales, yet the computational mechanisms underlying this ability remain debated due to the…
Tracking Equivalent Mechanistic Interpretations Across Neural Networks
Alan Sun, Mariya Toneva
Mechanistic interpretability (MI) is an emerging framework for interpreting neural networks. Given a task and model, MI aims to discover a succinct algorithmic process, an interpre…
When Language Models Lose Their Mind: The Consequences of Brain Misalignment
Gabriele Merlin, Mariya Toneva
While brain-aligned large language models (LLMs) have garnered attention for their potential as cognitive models and for potential for enhanced safety and trustworthiness in AI, th…
Fine-grained Analysis of Brain-LLM Alignment through Input Attribution
Michela Proietti, Roberto Capobianco, Mariya Toneva
Understanding the alignment between large language models (LLMs) and human brain activity can reveal computational principles underlying language processing. We introduce a fine-gr…
Brain-tuned Speech Models Better Reflect Speech Processing Stages in the Brain
Omer Moussa, Mariya Toneva
Pretrained self-supervised speech models excel in speech tasks but do not reflect the hierarchy of human speech processing, as they encode rich semantics in middle layers and poor…
Improving Semantic Understanding in Speech Language Models via Brain-tuning
Omer Moussa, Dietrich Klakow, Mariya Toneva
Speech language models align with human brain responses to natural language to an impressive degree. However, current models rely heavily on low-level speech features, indicating t…
Vision-Language Integration in Multimodal Video Transformers (Partially) Aligns with the Brain
Dota Tianai Dong, Mariya Toneva
Integrating information from multiple modalities is arguably one of the essential prerequisites for grounding artificial intelligence systems with an understanding of the real worl…
Speech language models lack important brain-relevant semantics
Subba Reddy Oota, Emin Çelik, Fatma Deniz +1
Despite known differences between reading and listening in the brain, recent work has shown that text-based language models predict both text-evoked and speech-evoked brain activit…
Perturbed examples reveal invariances shared by language models
Ruchit Rawal, Mariya Toneva
The rapid growth in natural language processing (NLP) research has led to numerous new models, outpacing our understanding of how they compare to established ones. One major reason…
What Happens During Finetuning of Vision Transformers: An Invariance Based Investigation
Gabriele Merlin, Vedant Nanda, Ruchit Rawal +1
The pretrain-finetune paradigm usually improves downstream performance over training a model from scratch on the same task, becoming commonplace across many areas of machine learni…
Investigating the Effects of Fairness Interventions Using Pointwise Representational Similarity
Camila Kolling, Till Speicher, Vedant Nanda +2
Machine learning (ML) algorithms can often exhibit discriminatory behavior, negatively affecting certain populations across protected groups. To address this, numerous debiasing me…
Large language models can segment narrative events similarly to humans
Sebastian Michelmann, Manoj Kumar, Kenneth A. Norman +1
Humans perceive discrete events such as "restaurant visits" and "train rides" in their continuous experience. One important prerequisite for studying human event perception is the…
Training language models to summarize narratives improves brain alignment
Khai Loong Aw, Mariya Toneva
Building systems that achieve a deeper understanding of language is one of the central goals of natural language processing (NLP). Towards this goal, recent works have begun to tra…
Joint processing of linguistic properties in brains and language models
Subba Reddy Oota, Manish Gupta, Mariya Toneva
Language models have been shown to be very effective in predicting brain recordings of subjects experiencing complex language stimuli. For a deeper understanding of this alignment,…
Language models and brains align due to more than next-word prediction and word-level information
Gabriele Merlin, Mariya Toneva
Pretrained language models have been shown to significantly predict brain recordings of people comprehending language. Recent work suggests that the prediction of the next word is…
Same Cause; Different Effects in the Brain
Mariya Toneva, Jennifer Williams, Anand Bollu +2
To study information processing in the brain, neuroscientists manipulate experimental stimuli while recording participant brain activity. They can then use encoding models to find…
A roadmap to reverse engineering real-world generalization by combining naturalistic paradigms, deep sampling, and predictive computational models
Peer Herholz, Eddy Fortier, Mariya Toneva +3
Real-world generalization, e.g., deciding to approach a never-seen-before animal, relies on contextual information as well as previous experiences. Such a seemingly easy behavioral…
Does injecting linguistic structure into language models lead to better alignment with brain recordings?
Mostafa Abdou, Ana Valeria Gonzalez, Mariya Toneva +2
Neuroscientists evaluate deep neural networks for natural language processing as possible candidate models for how language is processed in the brain. These models are often traine…
Modeling Task Effects on Meaning Representation in the Brain via Zero-Shot MEG Prediction
Mariya Toneva, Otilia Stretcu, Barnabas Poczos +2
How meaning is represented in the brain is still one of the big open questions in neuroscience. Does a word (e.g., bird) always have the same representation, or does the task under…
Inducing brain-relevant bias in natural language processing models
Dan Schwartz, Mariya Toneva, Leila Wehbe
Progress in natural language processing (NLP) models that estimate representations of word sequences has recently been leveraged to improve the understanding of language processing…