12 citations · 38 across the 18 of their papers we have counts for
20 papers
A Study of Data Selection Strategies for Pre-training Self-Supervised Speech Models
Ryan Whetten, Titouan Parcollet, Marco Dinarelli +1
Self-supervised learning (SSL) has transformed speech processing, yet its reliance on massive pre-training datasets remains a bottleneck. While robustness is often attributed to sc…
Towards Early Prediction of Self-Supervised Speech Model Performance
Ryan Whetten, Lucas Maison, Titouan Parcollet +2
In Self-Supervised Learning (SSL), pre-training and evaluation are resource intensive. In the speech domain, current indicators of the quality of SSL models during pre-training, su…
An Analysis of Linear Complexity Attention Substitutes with BEST-RQ
Ryan Whetten, Titouan Parcollet, Adel Moumen +2
Self-Supervised Learning (SSL) has proven to be effective in various domains, including speech processing. However, SSL is computationally and memory expensive. This is in part due…
Open Implementation and Study of BEST-RQ for Speech Processing
Ryan Whetten, Titouan Parcollet, Marco Dinarelli +1
Self-Supervised Learning (SSL) has proven to be useful in various speech tasks. However, these methods are generally very demanding in terms of data, memory, and computational reso…
LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speech
Titouan Parcollet, Ha Nguyen, Solene Evain +19
Self-supervised learning (SSL) is at the origin of unprecedented improvements in many different domains including computer vision and natural language processing. Speech processing…
Encoding Sentence Position in Context-Aware Neural Machine Translation with Concatenation
Lorenzo Lupo, Marco Dinarelli, Laurent Besacier
Context-aware translation can be achieved by processing a concatenation of consecutive sentences with the standard Transformer architecture. This paper investigates the intuitive i…
Focused Concatenation for Context-Aware Neural Machine Translation
Lorenzo Lupo, Marco Dinarelli, Laurent Besacier
A straightforward approach to context-aware neural machine translation consists in feeding the standard encoder-decoder architecture with a window of consecutive sentences, formed…
TArC: Tunisian Arabish Corpus First complete release
Elisa Gugliotta, Marco Dinarelli
In this paper we present the final result of a project on Tunisian Arabic encoded in Arabizi, the Latin-based writing system for digital conversations. The project led to the creat…
Toward Low-Cost End-to-End Spoken Language Understanding
Marco Dinarelli, Marco Naguib, François Portet
Recent advances in spoken language understanding benefited from Self-Supervised models trained on large speech corpora. For French, the LeBenchmark project has made such models ava…
Vers la compréhension automatique de la parole bout-en-bout à moindre effort
Marco Naguib, François Portet, Marco Dinarelli
Recent advances in spoken language understanding benefited from Self-Supervised models trained on large speech corpora. For French, the LeBenchmark project has made such models ava…
LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speech
Solene Evain, Ha Nguyen, Hang Le +15
Self-Supervised Learning (SSL) using huge unlabeled data has been successfully explored for image and natural language processing. Recent works also investigated SSL from speech. T…
Divide and Rule: Effective Pre-Training for Context-Aware Multi-Encoder Translation Models
Lorenzo Lupo, Marco Dinarelli, Laurent Besacier
Multi-encoder models are a broad family of context-aware neural machine translation systems that aim to improve translation quality by encoding document-level contextual informatio…
Multi-Task Sequence Prediction For Tunisian Arabizi Multi-Level Annotation
Elisa Gugliotta, Marco Dinarelli, Olivier Kraif
In this paper we propose a multi-task sequence prediction system, based on recurrent neural networks and used to annotate on multiple levels an Arabizi Tunisian corpus. The annotat…
TArC: Incrementally and Semi-Automatically Collecting a Tunisian Arabish Corpus
Elisa Gugliotta, Marco Dinarelli
This article describes the constitution process of the first morpho-syntactically annotated Tunisian Arabish Corpus (TArC). Arabish, also known as Arabizi, is a spontaneous coding…
A Data Efficient End-To-End Spoken Language Understanding Architecture
Marco Dinarelli, Nikita Kapoor, Bassam Jabaian +1
End-to-end architectures have been recently proposed for spoken language understanding (SLU) and semantic parsing. Based on a large amount of data, those models learn jointly acous…
Hybrid Neural Models For Sequence Modelling: The Best Of Three Worlds
Marco Dinarelli, Loïc Grobol
We propose a neural architecture with the main characteristics of the most successful neural models of the last years: bidirectional RNNs, encoder-decoder, and the Transformer mode…
Seq2Biseq: Bidirectional Output-wise Recurrent Neural Networks for Sequence Modelling
Marco Dinarelli, Loïc Grobol
During the last couple of years, Recurrent Neural Networks (RNN) have reached state-of-the-art performances on most of the sequence modelling problems. In particular, the "sequence…
Effective Spoken Language Labeling with Deep Recurrent Neural Networks
Marco Dinarelli, Yoann Dupont, Isabelle Tellier
Understanding spoken language is a highly complex problem, which can be decomposed into several simpler tasks. In this paper, we focus on Spoken Language Understanding (SLU), the m…
Label-Dependencies Aware Recurrent Neural Networks
Yoann Dupont, Marco Dinarelli, Isabelle Tellier
In the last few years, Recurrent Neural Networks (RNNs) have proved effective on several NLP tasks. Despite such great success, their ability to model \emph{sequence labeling} is s…
Improving Recurrent Neural Networks For Sequence Labelling
Marco Dinarelli, Isabelle Tellier
In this paper we study different types of Recurrent Neural Networks (RNN) for sequence labeling tasks. We propose two new variants of RNNs integrating improvements for sequence lab…