105 citations · 349 across the 67 of their papers we have counts for
12 papers · 1 filter
Can Activation Steering Capture Multidimensional Authorship Style?
Hieu Tran, Calvin Bao, Marine Carpuat
Activation steering has shown promise for controlling LLM generation along well-defined attributes, but it remains unclear whether it can handle the multidimensional and hard-to-de…
Do Evaluation Metrics Detect Errors in Classical Chinese to English Translations?
Osvaldo Quinjica, Eric Bennett, Xinchen Yang +2
Although large language models can translate some historical languages surprisingly well, their usefulness in digital humanities workflows is limited by the lack of reliable evalua…
Contrastive ESA: Human Evaluation of Multiple Translations at Once
Vilém Zouhar, Roman Grundkiewicz, Sara Rajaee +6
Current human evaluation of machine translation typically assesses single outputs in isolation, a paradigm that suffers from high annotator noise and cost. We introduce Contrastive…
Measuring User's Mental Models of Speech Translation in Human-AI Collaboration
HyoJung Han, Nishant Balepur, Jordan Boyd-Graber +1
Millions of people use machine translation (MT) tools daily, yet little is known about their perception of what systems can and cannot do. This paper studies users' mental models o…
AdaMame: A Training Recipe for Adaptive Multilingual Reasoning
Dayeon Ki, Kevin Duh, Marine Carpuat
While Large Reasoning Models (LRMs) show strong performance in English, they often fail to reason in the language of the query, a phenomenon known as language collapse. Existing RL…
Ouvia: A User-centered Framework for Measuring Usability of Speech Translation in Real-World Communication Scenarios
Giuseppe Attanasio, Beatrice Savoldi, Daniel Chechelnitsky +4
Speech translation (ST) is increasingly adopted in user applications, yet its evaluation largely focuses on decontextualized testbeds and holistic quality, rather than end users' c…
Can You Make It Sound Like You? Post-Editing LLM-Generated Text for Personal Style
Connor Baumler, Calvin Bao, Huy Nghiem +3
Despite the growing use of large language models (LLMs) for writing tasks, users may hesitate to rely on LLMs when personal style is important. Post-editing LLM-generated drafts or…
Reheat Nachos for Dinner? Evaluating AI Support for Cross-Cultural Communication of Neologisms
Dayeon Ki, Yu Hou, Rachel Rudinger +3
Neologisms and emerging slang are central to daily conversation, yet challenging for non-native speakers (NNS) to interpret and use appropriately in cross-cultural communication wi…
Should We be Pedantic About Reasoning Errors in Machine Translation?
Calvin Bao, Marine Carpuat
Across multiple language pairings (English \{Spanish, French, German, Mandarin, Japanese, Urdu, Cantonese\}), we find reasoning errors in translation. To quantify how often t…
What Makes Good Multilingual Reasoning? Disentangling Traces with Measurable Features
Dayeon Ki, Kevin Duh, Marine Carpuat
Large Reasoning Models (LRMs) still exhibit large performance gaps between English and other languages, yet much current work assumes these gaps can be closed simply by making reas…
Pragmatics Meets Culture: Culturally-adapted Artwork Description Generation and Evaluation
Lingjun Zhao, Dayeon Ki, Marine Carpuat +1
Language models are known to exhibit various forms of cultural bias in decision-making tasks, yet much less is known about their degree of cultural familiarity in open-ended text g…
Take Out Your Calculators: Estimating the Real Difficulty of Question Items with LLM Student Simulations
Christabel Acquaye, Yi Ting Huang, Marine Carpuat +1
Standardized math assessments require expensive human pilot studies to establish the difficulty of test items. We investigate the predictive value of open-source large language mod…