22 citations · 79 across the 14 of their papers we have counts for
14 papers
F-Actor: Controllable Conversational Behaviour in Full-Duplex Models
Maike Züfle, Ondrej Klejch, Nicholas Sanders +3
Spoken conversational systems require more than accurate speech generation to have human-like conversations: to feel natural and engaging, they must produce conversational behaviou…
The Prosody of Emojis
Giulio Zhou, Tsz Kin Lam, Alexandra Birch +1
Prosodic features such as pitch, timing, and intonation are central to spoken communication, conveying emotion, intent, and discourse structure. In text-based settings, where these…
From TOWER to SPIRE: Adding the Speech Modality to a Translation-Specialist LLM
Kshitij Ambilduke, Ben Peters, Sonal Sannigrahi +5
We introduce Spire, a speech-augmented language model (LM) capable of both translating and transcribing speech input from English into 10 other languages as well as translating tex…
Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison
Tsz Kin Lam, Marco Gaido, Sara Papi +2
Following the remarkable success of Large Language Models (LLMs) in NLP tasks, there is increasing interest in extending their capabilities to speech -- the most common form of com…
Pitfalls and Outlooks in Using COMET
Vilém Zouhar, Pinzhen Chen, Tsz Kin Lam +2
The COMET metric has blazed a trail in the machine translation community, given its strong correlation with human judgements of translation quality. Its success stems from being a…
Compact Speech Translation Models via Discrete Speech Units Pretraining
Tsz Kin Lam, Alexandra Birch, Barry Haddow
We propose a pretraining method to use Self-Supervised Speech (SSS) model to creating more compact Speech-to-text Translation. In contrast to using the SSS model for initialization…
Prosody in Cascade and Direct Speech-to-Text Translation: a case study on Korean Wh-Phrases
Giulio Zhou, Tsz Kin Lam, Alexandra Birch +1
Speech-to-Text Translation (S2TT) has typically been addressed with cascade systems, where speech recognition systems generate a transcription that is subsequently passed to a tran…
Analyzing the Use of Influence Functions for Instance-Specific Data Filtering in Neural Machine Translation
Tsz Kin Lam, Eva Hasler, Felix Hieber
Customer feedback can be an important signal for improving commercial machine translation systems. One solution for fixing specific translation errors is to remove the related erro…
Make More of Your Data: Minimal Effort Data Augmentation for Automatic Speech Recognition and Translation
Tsz Kin Lam, Shigehiko Schamoni, Stefan Riezler
Data augmentation is a technique to generate new training data based on existing data. We evaluate the simple and cost-effective method of concatenating the original data examples…
Sample, Translate, Recombine: Leveraging Audio Alignments for Data Augmentation in End-to-end Speech Translation
Tsz Kin Lam, Shigehiko Schamoni, Stefan Riezler
End-to-end speech translation relies on data that pair source-language speech inputs with corresponding translations into a target language. Such data are notoriously scarce, makin…
On-the-Fly Aligned Data Augmentation for Sequence-to-Sequence ASR
Tsz Kin Lam, Mayumi Ohta, Shigehiko Schamoni +1
We propose an on-the-fly data augmentation method for automatic speech recognition (ASR) that uses alignment information to generate effective training samples. Our method, called…
Cascaded Models With Cyclic Feedback For Direct Speech Translation
Tsz Kin Lam, Shigehiko Schamoni, Stefan Riezler
Direct speech translation describes a scenario where only speech inputs and corresponding translations are available. Such data are notoriously limited. We present a technique that…
Interactive-Predictive Neural Machine Translation through Reinforcement and Imitation
Tsz Kin Lam, Shigehiko Schamoni, Stefan Riezler
We propose an interactive-predictive neural machine translation framework for easier model personalization using reinforcement and imitation learning. During the interactive transl…
A Reinforcement Learning Approach to Interactive-Predictive Neural Machine Translation
Tsz Kin Lam, Julia Kreutzer, Stefan Riezler
We present an approach to interactive-predictive neural machine translation that attempts to reduce human effort from three directions: Firstly, instead of requiring humans to sele…