1 citations · 1 across the 11 of their papers we have counts for
12 papers
Opening LLM Judges: Recovering Preference Signals Beyond the Final Verdict
Sourabrata Mukherjee, Sunayana Sitaram
LLM judges are widely used to evaluate model outputs, but their verdicts can be unreliable: a judge may favor the worse answer for its position, length, or other surface features.…
Actions Speak Louder than Words: Measuring Cross-Lingual Policy Retention in Tool-Using Agents
Sourabrata Mukherjee, Kalika Bali, Sunayana Sitaram
When a tool-using agent is given the same task in a different language, does it still take the same steps? Multilingual evaluation rarely asks: it compares final answers and discar…
The Geometry of LLM-as-Judge: Why Inter-LLM Consensus Is Not Human Alignment
Sourabrata Mukherjee, Hamna Hamna, Kalika Bali +1
LLM judges now score most open-ended NLP output, and their mutual agreement is routinely read as evidence that the scores can be trusted. That reading is unsafe: judges may agree b…
Building Benchmarks from the Ground Up: Community-Centered Evaluation of LLMs in Healthcare Chatbot Settings
Hamna Hamna, Gayatri Bhat, Sourabrata Mukherjee +5
Large Language Models (LLMs) are typically evaluated through general or domain-specific benchmarks testing capabilities that often lack grounding in the lived realities of end user…
Evaluating Text Style Transfer Evaluation: Are There Any Reliable Metrics?
Sourabrata Mukherjee, Atul Kr. Ojha, John P. McCrae +1
Text style transfer (TST) is the task of transforming a text to reflect a particular style while preserving its original content. Evaluating TST outputs is a multidimensional chall…
Women, Infamous, and Exotic Beings: A Comparative Study of Honorific Usages in Wikipedia and LLMs for Bengali and Hindi
Sourabrata Mukherjee, Atharva Mehta, Sougata Saha +2
The obligatory use of third-person honorifics is a distinctive feature of several South Asian languages, encoding nuanced socio-pragmatic cues such as power, age, gender, fame, and…
A Survey of Text Style Transfer: Applications and Ethical Implications
Sourabrata Mukherjee, Mateusz Lango, Zdenek Kasner +1
Text style transfer (TST) is an important task in controllable text generation, which aims to control selected attributes of language use, such as politeness, formality, or sentime…
Text Style Transfer: An Introductory Overview
Sourabrata Mukherjee, Ondrej Dušek
Text Style Transfer (TST) is a pivotal task in natural language generation to manipulate text style attributes while preserving style-independent content. The attributes targeted i…
Are Large Language Models Actually Good at Text Style Transfer?
Sourabrata Mukherjee, Atul Kr. Ojha, Ondřej Dušek
We analyze the performance of large language models (LLMs) on Text Style Transfer (TST), specifically focusing on sentiment transfer and text detoxification across three languages:…
Multilingual Text Style Transfer: Datasets & Models for Indian Languages
Sourabrata Mukherjee, Atul Kr. Ojha, Akanksha Bansal +3
Text style transfer (TST) involves altering the linguistic style of a text while preserving its core content. This paper focuses on sentiment transfer, a popular TST subtask, acros…
Text Detoxification as Style Transfer in English and Hindi
Sourabrata Mukherjee, Akanksha Bansal, Atul Kr. Ojha +2
This paper focuses on text detoxification, i.e., automatically converting toxic text into non-toxic text. This task contributes to safer and more respectful online communication an…
Balancing the Style-Content Trade-Off in Sentiment Transfer Using Polarity-Aware Denoising
Sourabrata Mukherjee, Zdeněk Kasner, Ondřej Dušek
Text sentiment transfer aims to flip the sentiment polarity of a sentence (positive to negative or vice versa) while preserving its sentiment-independent content. Although current…