11 citations · 33 across the 48 of their papers we have counts for
54 papers
RedVox: Safety and Fairness Gaps in Speech Models Across Languages
Beatrice Savoldi, Sara Papi, Wafa Aissa +2
Speech-capable models are increasingly deployed in real-world applications across languages. Yet their safety and fairness beyond English settings and under naturalistic conditions…
FBK's Long-form SpeechLLMs for IWSLT 2026 Instruction Following
Zhihang Xie, Marco Gaido, Sara Papi +2
This paper describes our submission to the IWSLT 2026 Instruction Following shared task. SpeechLLMs are developed for both short-form and long-form speech instruction following und…
DOA: Training-Free Decoder-Only Attention Policy for Long-Form Simultaneous Translation with SpeechLLMs
Sara Papi, Luisa Bentivogli
Simultaneous speech-to-text translation (SimulST) generates translations while speech is still unfolding, requiring a streaming policy that decides when to read and when to write.…
Do What I Say: A Spoken Prompt Dataset for Instruction-Following
Maike Züfle, Sara Papi, Fabian Retkowski +5
Speech Large Language Models (SLLMs) have rapidly expanded, supporting a wide range of tasks. These models are typically evaluated using text prompts, which may not reflect real-wo…
SimulU: Training-free Policy for Long-form Simultaneous Speech-to-Speech Translation
Amirbek Djanibekov, Luisa Bentivogli, Matteo Negri +1
Simultaneous speech-to-speech translation (SimulS2S) is essential for real-time multilingual communication, with increasing integration into meeting and streaming platforms. Despit…
Challenging the Abilities of Large Language Models in Italian: a Community Initiative
Malvina Nissim, Danilo Croce, Viviana Patti +78
The rapid progress of Large Language Models (LLMs) has transformed natural language processing and broadened its impact across research and society. Yet, systematic evaluation of t…
Simulstream: Open-Source Toolkit for Evaluation and Demonstration of Streaming Speech-to-Text Translation Systems
Marco Gaido, Sara Papi, Mauro Cettolo +2
Streaming Speech-to-Text Translation (StreamST) requires producing translations concurrently with incoming speech under strict latency constraints, demanding models that balance lo…
Generative AI Practices, Literacy, and Divides: An Empirical Analysis in the Italian Context
Beatrice Savoldi, Giuseppe Attanasio, Olga Gorodetskaya +10
The rise of generative AI (GenAI) chatbots accessible via conversational interfaces is transforming digital interactions and holds economic promise. However, these tools might deep…
Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation
Lina Conti, Dennis Fucci, Marco Gaido +3
Unlike text, speech conveys information about the speaker, such as gender, through acoustic cues like pitch. This gives rise to modality-specific bias concerns. For example, in spe…
How to Evaluate Speech Translation with Source-Aware Neural MT Metrics
Mauro Cettolo, Marco Gaido, Matteo Negri +2
Automatic evaluation of ST systems is typically performed by comparing translation hypotheses with one or more reference translations. While effective to some extent, this approach…
The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models
Lina Conti, Dennis Fucci, Marco Gaido +3
Contrastive explanations, which indicate why an AI system produced one output (the target) instead of another (the foil), are widely regarded in explainable AI as more informative…
Gender-Neutral Rewriting in Italian: Models, Approaches, and Trade-offs
Andrea Piergentili, Beatrice Savoldi, Matteo Negri +1
Gender-neutral rewriting (GNR) aims to reformulate text to eliminate unnecessary gender specifications while preserving meaning, a particularly challenging task in grammatical-gend…
Cross-Attention is Half Explanation in Speech-to-Text Models
Sara Papi, Dennis Fucci, Marco Gaido +2
Cross-attention is a core mechanism in encoder-decoder architectures, widespread in many fields, including speech-to-text (S2T) processing. Its scores have been repurposed for vari…
Better Late Than Never: Meta-Evaluation of Latency Metrics for Simultaneous Speech-to-Text Translation
Peter Polák, Sara Papi, Luisa Bentivogli +1
Simultaneous speech-to-text translation systems must balance translation quality with latency. Although quality evaluation is well established, latency measurement remains a challe…
MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks
Sara Papi, Maike Züfle, Marco Gaido +5
Recent advances in large language models have laid the foundation for multimodal LLMs (MLLMs), which unify text, speech, and vision within a single framework. As these models are r…
An Interdisciplinary Approach to Human-Centered Machine Translation
Marine Carpuat, Omri Asscher, Kalika Bali +17
Machine Translation (MT) tools are widely used today, often in contexts where professional translators are not present. Despite progress in MT technology, a gap persists between sy…
Echoes of Phonetics: Unveiling Relevant Acoustic Cues for ASR via Feature Attribution
Dennis Fucci, Marco Gaido, Matteo Negri +2
Despite significant advances in ASR, the specific acoustic cues models rely on remain unclear. Prior studies have examined such cues on a limited set of phonemes and outdated model…
Different Speech Translation Models Encode and Translate Speaker Gender Differently
Dennis Fucci, Marco Gaido, Matteo Negri +3
Recent studies on interpreting the hidden states of speech models have shown their ability to capture speaker-specific features, including gender. Does this finding also hold for s…
FAMA: The First Large-Scale Open-Science Speech Foundation Model for English and Italian
Sara Papi, Marco Gaido, Luisa Bentivogli +6
The development of speech foundation models (SFMs) like Whisper and SeamlessM4T has significantly advanced the field of speech processing. However, their closed nature--with inacce…
The Warmup Dilemma: How Learning Rate Strategies Impact Speech-to-Text Model Convergence
Marco Gaido, Sara Papi, Luisa Bentivogli +6
Training large-scale models presents challenges not only in terms of resource requirements but also in terms of their convergence. For this reason, the learning rate (LR) is often…
An LLM-as-a-judge Approach for Scalable Gender-Neutral Translation Evaluation
Andrea Piergentili, Beatrice Savoldi, Matteo Negri +1
Gender-neutral translation (GNT) aims to avoid expressing the gender of human referents when the source text lacks explicit cues about the gender of those referents. Evaluating GNT…
Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison
Tsz Kin Lam, Marco Gaido, Sara Papi +2
Following the remarkable success of Large Language Models (LLMs) in NLP tasks, there is increasing interest in extending their capabilities to speech -- the most common form of com…
NUTSHELL: A Dataset for Abstract Generation from Scientific Talks
Maike Züfle, Sara Papi, Beatrice Savoldi +3
Scientific communication is receiving increasing attention in natural language processing, especially to help researches access, summarize, and generate content. One emerging appli…
Translation in the Hands of Many:Centering Lay Users in Machine Translation Interactions
Beatrice Savoldi, Alan Ramponi, Matteo Negri +1
Converging societal and technical factors have transformed language technologies into user-facing applications used by the general public across languages. Machine Translation (MT)…