105 citations · 349 across the 67 of their papers we have counts for
67 papers · 1 filter
Can Activation Steering Capture Multidimensional Authorship Style?
Hieu Tran, Calvin Bao, Marine Carpuat
Activation steering has shown promise for controlling LLM generation along well-defined attributes, but it remains unclear whether it can handle the multidimensional and hard-to-de…
Do Evaluation Metrics Detect Errors in Classical Chinese to English Translations?
Osvaldo Quinjica, Eric Bennett, Xinchen Yang +2
Although large language models can translate some historical languages surprisingly well, their usefulness in digital humanities workflows is limited by the lack of reliable evalua…
Contrastive ESA: Human Evaluation of Multiple Translations at Once
Vilém Zouhar, Roman Grundkiewicz, Sara Rajaee +6
Current human evaluation of machine translation typically assesses single outputs in isolation, a paradigm that suffers from high annotator noise and cost. We introduce Contrastive…
Measuring User's Mental Models of Speech Translation in Human-AI Collaboration
HyoJung Han, Nishant Balepur, Jordan Boyd-Graber +1
Millions of people use machine translation (MT) tools daily, yet little is known about their perception of what systems can and cannot do. This paper studies users' mental models o…
AdaMame: A Training Recipe for Adaptive Multilingual Reasoning
Dayeon Ki, Kevin Duh, Marine Carpuat
While Large Reasoning Models (LRMs) show strong performance in English, they often fail to reason in the language of the query, a phenomenon known as language collapse. Existing RL…
Ouvia: A User-centered Framework for Measuring Usability of Speech Translation in Real-World Communication Scenarios
Giuseppe Attanasio, Beatrice Savoldi, Daniel Chechelnitsky +4
Speech translation (ST) is increasingly adopted in user applications, yet its evaluation largely focuses on decontextualized testbeds and holistic quality, rather than end users' c…
Can You Make It Sound Like You? Post-Editing LLM-Generated Text for Personal Style
Connor Baumler, Calvin Bao, Huy Nghiem +3
Despite the growing use of large language models (LLMs) for writing tasks, users may hesitate to rely on LLMs when personal style is important. Post-editing LLM-generated drafts or…
Reheat Nachos for Dinner? Evaluating AI Support for Cross-Cultural Communication of Neologisms
Dayeon Ki, Yu Hou, Rachel Rudinger +3
Neologisms and emerging slang are central to daily conversation, yet challenging for non-native speakers (NNS) to interpret and use appropriately in cross-cultural communication wi…
Should We be Pedantic About Reasoning Errors in Machine Translation?
Calvin Bao, Marine Carpuat
Across multiple language pairings (English \{Spanish, French, German, Mandarin, Japanese, Urdu, Cantonese\}), we find reasoning errors in translation. To quantify how often t…
What Makes Good Multilingual Reasoning? Disentangling Traces with Measurable Features
Dayeon Ki, Kevin Duh, Marine Carpuat
Large Reasoning Models (LRMs) still exhibit large performance gaps between English and other languages, yet much current work assumes these gaps can be closed simply by making reas…
Pragmatics Meets Culture: Culturally-adapted Artwork Description Generation and Evaluation
Lingjun Zhao, Dayeon Ki, Marine Carpuat +1
Language models are known to exhibit various forms of cultural bias in decision-making tasks, yet much less is known about their degree of cultural familiarity in open-ended text g…
Take Out Your Calculators: Estimating the Real Difficulty of Question Items with LLM Student Simulations
Christabel Acquaye, Yi Ting Huang, Marine Carpuat +1
Standardized math assessments require expensive human pilot studies to establish the difficulty of test items. We investigate the predictive value of open-source large language mod…
Toward Machine Translation Literacy: How Lay Users Perceive and Rely on Imperfect Translations
Yimin Xiao, Yongle Zhang, Dayeon Ki +5
As Machine Translation (MT) becomes increasingly commonplace, understanding how the general public perceives and relies on imperfect MT is crucial for contextualizing MT research i…
Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAG
Dayeon Ki, Marine Carpuat, Paul McNamee +4
Multilingual Retrieval-Augmented Generation (mRAG) systems enable language models to answer knowledge-intensive queries with citation-supported responses across languages. Despite…
An Interdisciplinary Approach to Human-Centered Machine Translation
Marine Carpuat, Omri Asscher, Kalika Bali +17
Machine Translation (MT) tools are widely used today, often in contexts where professional translators are not present. Despite progress in MT technology, a gap persists between sy…
Steering Large Language Models with Register Analysis for Arbitrary Style Transfer
Xinchen Yang, Marine Carpuat
Large Language Models (LLMs) have demonstrated strong capabilities in rewriting text across various styles. However, effectively leveraging this ability for example-based arbitrary…
Should I Share this Translation? Evaluating Quality Feedback for User Reliance on Machine Translation
Dayeon Ki, Kevin Duh, Marine Carpuat
As people increasingly use AI systems in work and daily life, feedback mechanisms that help them use AI responsibly are urgently needed, particularly in settings where users are no…
Multiple LLM Agents Debate for Equitable Cultural Alignment
Dayeon Ki, Rachel Rudinger, Tianyi Zhou +1
Large Language Models (LLMs) need to adapt their predictions to diverse cultural contexts to benefit diverse communities across the world. While previous efforts have focused on si…
VietMix: A Naturally-Occurring Parallel Corpus and Augmentation Framework for Vietnamese-English Code-Mixed Machine Translation
Hieu Tran, Phuong-Anh Nguyen-Le, Huy Nghiem +3
Machine translation (MT) systems universally degrade when faced with code-mixed text. This problem is more acute for low-resource languages that lack dedicated parallel corpora. Th…
AskQE: Question Answering as Automatic Evaluation for Machine Translation
Dayeon Ki, Kevin Duh, Marine Carpuat
How can a monolingual English speaker determine whether an automatic translation in French is good enough to be shared? Existing MT error detection and quality estimation (QE) tech…
Can you map it to English? The Role of Cross-Lingual Alignment in Multilingual Performance of LLMs
Kartik Ravisankar, Hyojung Han, Sarah Wiegreffe +1
Large language models (LLMs) can answer prompts in many languages, despite being trained predominantly on English; yet, the mechanisms driving this generalization remain poorly und…
Words as Bridges: Exploring Computational Support for Cross-Disciplinary Translation Work
Calvin Bao, Yow-Ting Shiue, Marine Carpuat +1
Scholars often explore literature outside of their home community of study. This exploration process is frequently hampered by field-specific jargon. Past computational work often…
Automatic Input Rewriting Improves Translation with Large Language Models
Dayeon Ki, Marine Carpuat
Can we improve machine translation (MT) with LLMs by rewriting their inputs automatically? Users commonly rely on the intuition that well-written text is easier to translate when u…
Findings of the IWSLT 2024 Evaluation Campaign
Ibrahim Said Ahmad, Antonios Anastasopoulos, Ondřej Bojar +42
This paper reports on the shared tasks organized by the 21st IWSLT Conference. The shared tasks address 7 scientific challenges in spoken language translation: simultaneous and off…