8 citations · 9 across the 10 of their papers we have counts for
6 papers · 1 filter
Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models
Atsumoto Ohashi, Neil Zeghidour, Alexandre Défossez +1
Full-duplex spoken dialogue models can listen and speak simultaneously, making them a promising architecture for natural conversation. However, current models are trained solely wi…
MoshiRAG: Asynchronous Knowledge Retrieval for Full-Duplex Speech Language Models
Chung-Ming Chien, Manu Orsini, Eugene Kharitonov +3
Speech-to-speech language models have recently emerged to enhance the naturalness of conversational AI. In particular, full-duplex models are distinguished by their real-time inter…
Simultaneous Speech-to-Speech Translation Without Aligned Data
Tom Labiausse, Romain Fabre, Yannick Estève +2
Simultaneous speech translation requires translating source speech into a target language in real-time while handling non-monotonic word dependencies. Traditional approaches rely o…
Streaming Sequence-to-Sequence Learning with Delayed Streams Modeling
Neil Zeghidour, Eugene Kharitonov, Manu Orsini +6
We introduce Delayed Streams Modeling (DSM), a flexible formulation for streaming, multimodal sequence-to-sequence learning. Sequence-to-sequence generation is often cast in an off…
Aligning Spoken Dialogue Models from User Interactions
Anne Wu, Laurent Mazaré, Neil Zeghidour +1
We propose a novel preference alignment framework for improving spoken dialogue models on real-time conversations from user interactions. Current preference learning methods primar…
High-Fidelity Simultaneous Speech-To-Speech Translation
Tom Labiausse, Laurent Mazaré, Edouard Grave +3
We introduce Hibiki, a decoder-only model for simultaneous speech translation. Hibiki leverages a multistream language model to synchronously process source and target speech, and…