2 citations · 5 across the 20 of their papers we have counts for
20 papers
A Mechanistic Understanding of Pronoun Fidelity in LLMs
Katharina Trinley, Jesujoba O. Alabi, Dietrich Klakow +1
Faithful and robust pronoun use is important for fair and coherent generations, yet large language models largely fail when multiple referents use different pronouns. To study the…
Making a Name for Myself: On Academic Naming Policies and their Impact
A Pranav, Vagrant Gautam, Martin Mundt +6
In academic publishing, names connect scholars to their work. When scholars change their names, including for marriage, academic recognition, or gender transition, they may lose cr…
Your Multimodal Speech Model Says I Have a Face for Radio
Maya K. Nachesa, Vlad Niculae, Vagrant Gautam
As large neural models have become better at language tasks, researchers are increasingly building multi- and omnimodal models that handle more modalities of data. One example is t…
GRUFF: LLM Pronoun Fidelity, Reasoning, and Biases in German
Fabian Mewes, Anne Lauscher, Vagrant Gautam
Third-person singular pronouns have long been used to study stereotypical biases in language models and to test their abilities to reason about reference. More recently, the interp…
Queer NLP: A Critical Survey on Literature Gaps, Biases and Trends
Sabine Weber, Angelina Wang, Ankush Gupta +16
Natural language processing (NLP) technologies are rapidly reshaping how language is created, processed, and interpreted by humans. With current and potential applications in hirin…
Whose Facts Win? LLM Source Preferences under Knowledge Conflicts
Jakob Schuster, Vagrant Gautam, Katja Markert
As large language models (LLMs) are more frequently used in retrieval-augmented generation pipelines, it is increasingly relevant to study their behavior under knowledge conflicts.…
Teaching and Critiquing Conceptualization and Operationalization in NLP
Vagrant Gautam
NLP researchers regularly invoke abstract concepts like "interpretability," "bias," "reasoning," and "stereotypes," without defining them. Each subfield has a shared understanding…
Decoding Partial Differential Equations: Cross-Modal Adaptation of Decoder-only Models to PDEs
Paloma García-de-Herreros, Philipp Slusallek, Dietrich Klakow +1
While large language models are primarily used on natural language tasks, they have also shown great promise when adapted to new modalities, e.g., for scientific machine learning t…
Colombian Waitresses y Jueces canadienses: Gender and Country Biases in Occupation Recommendations from LLMs
Elisa Forcada Rodríguez, Olatz Perez-de-Viñaspre, Jon Ander Campos +2
One of the goals of fairness research in NLP is to measure and mitigate stereotypical biases that are propagated by NLP systems. However, such work tends to focus on single axes of…
Agree to Disagree? A Meta-Evaluation of LLM Misgendering
Arjun Subramonian, Vagrant Gautam, Preethi Seshadri +3
Numerous methods have been proposed to measure LLM misgendering, including probability-based evaluations (e.g., automatically with templatic sentences) and generation-based evaluat…
Aligned Probing: Relating Toxic Behavior and Model Internals
Andreas Waldis, Vagrant Gautam, Anne Lauscher +2
We introduce aligned probing, a novel interpretability framework that aligns the behavior of language models (LMs), based on their outputs, and their internal representations (inte…
WinoPron: Revisiting English Winogender Schemas for Consistency, Coverage, and Grammatical Case
Vagrant Gautam, Julius Steuer, Eileen Bingert +3
While measuring bias and robustness in coreference resolution are important goals, such measurements are only as good as the tools we use to measure them. Winogender Schemas (Rudin…
From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP
Marius Mosbach, Vagrant Gautam, Tomás Vergara-Browne +2
Interpretability and analysis (IA) research is a growing subfield within NLP with the goal of developing a deeper understanding of the behavior or inner workings of NLP systems and…
Understanding "Democratization" in NLP and ML Research
Arjun Subramonian, Vagrant Gautam, Dietrich Klakow +1
Recent improvements in natural language processing (NLP) and machine learning (ML) and increased mainstream adoption have led to researchers frequently discussing the "democratizat…
Stop! In the Name of Flaws: Disentangling Personal Names and Sociodemographic Attributes in NLP
Vagrant Gautam, Arjun Subramonian, Anne Lauscher +1
Personal names simultaneously differentiate individuals and categorize them in ways that are important in a given society. While the natural language processing community has thus…
Robust Pronoun Fidelity with English LLMs: Are they Reasoning, Repeating, or Just Biased?
Vagrant Gautam, Eileen Bingert, Dawei Zhu +2
Robust, faithful and harm-free pronoun use for individuals is an important goal for language model development as their use increases, but prior work tends to study only one or two…
What explains the success of cross-modal fine-tuning with ORCA?
Paloma García-de-Herreros, Vagrant Gautam, Philipp Slusallek +2
ORCA (Shen et al., 2023) is a recent technique for cross-modal fine-tuning, i.e., applying pre-trained transformer models to modalities beyond their training data. The technique co…
The Impact of Demonstrations on Multilingual In-Context Learning: A Multidimensional Analysis
Miaoran Zhang, Vagrant Gautam, Mingyang Wang +4
In-context learning is a popular inference strategy where large language models solve a task using only a few labeled demonstrations without needing any parameter updates. Although…
A Lightweight Method to Generate Unanswerable Questions in English
Vagrant Gautam, Miaoran Zhang, Dietrich Klakow
If a question cannot be answered with the available information, robust systems for question answering (QA) should know _not_ to answer. One way to build QA models that do this is…
Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairness
Anaelia Ovalle, Arjun Subramonian, Vagrant Gautam +2
Intersectionality is a critical framework that, through inquiry and praxis, allows us to examine how social inequalities persist through domains of structure and discipline. Given…