20 citations · 40 across the 9 of their papers we have counts for
9 papers
Closing the Gap Between Text and Speech Understanding in LLMs
Santiago Cuervo, Skyler Seto, Maureen de Seyssel +5
Large Language Models (LLMs) can be adapted to extend their text capabilities to speech inputs. However, these speech-adapted LLMs consistently underperform their text-based counte…
LFAR: Accounting for Layerwise Dynamics to Improve Multimodal Adaptation of Language Models
Santiago Cuervo, Adel Moumen, Yanis Labrak +5
Text-pretrained language models (LMs) encode rich world knowledge, but adapting them to process and generate perceptual modalities such as audio and images while effectively levera…
Scaling Properties of Speech Language Models
Santiago Cuervo, Ricard Marxer
Speech Language Models (SLMs) aim to learn language from raw audio, without textual resources. Despite significant advances, our current models exhibit weak syntax and semantic abi…
Transfer Learning from Whisper for Microscopic Intelligibility Prediction
Paul Best, Santiago Cuervo, Ricard Marxer
Macroscopic intelligibility models predict the expected human word-error-rate for a given speech-in-noise stimulus. In contrast, microscopic intelligibility models aim to make fine…
Speech foundation models on intelligibility prediction for hearing-impaired listeners
Santiago Cuervo, Ricard Marxer
Speech foundation models (SFMs) have been benchmarked on many speech processing tasks, often achieving state-of-the-art performance with minimal adaptation. However, the SFM paradi…
Variable-rate hierarchical CPC leads to acoustic unit discovery in speech
Santiago Cuervo, Adrian Łańcucki, Ricard Marxer +2
The success of deep learning comes from its ability to capture the hierarchical structure of data by learning high-level representations defined in terms of low-level ones. In this…
Contrastive prediction strategies for unsupervised segmentation and categorization of phonemes and words
Santiago Cuervo, Maciej Grabias, Jan Chorowski +4
We investigate the performance on phoneme categorization and phoneme and word segmentation of several self-supervised learning (SSL) methods based on Contrastive Predictive Coding…
PAMELI: A Meta-Algorithm for Computationally Expensive Multi-Objective Optimization Problems
Santiago Cuervo, Miguel Melgarejo, Angie Blanco-Cañon +2
We present an algorithm for multi-objective optimization of computationally expensive problems. The proposed algorithm is based on solving a set of surrogate problems defined by mo…
Emergent cooperation through mutual information maximization
Santiago Cuervo, Marco Alzate
With artificial intelligence systems becoming ubiquitous in our society, its designers will soon have to start to consider its social dimension, as many of these systems will have…