336 citations · 1.4k across the 70 of their papers we have counts for
78 papers
GLaS-JEPA: Gaussian-Regularized Speech SSL without Engineered Prediction Targets
Gaspard Botté, Séverin Baroudi, Samir Sadok +5
Speech self-supervised learning aims to learn general-purpose representations for downstream speech tasks. However, current approaches rely on complex, carefully designed predictio…
Test-time adaptation for speech enhancement with an autoregressive speech prior
Sofiene Kammoun, Simon Leglaive, Xavier Alameda-Pineda +1
Test-time adaptation (TTA) offers a promising direction for improving speech enhancement models under mismatched acoustic conditions, without requiring access to labeled target dat…
EMODY Flow: Emotion-Aware Audio-Driven Full-Body Motion Generation
Harsh Kumar Agarwal, Xavier Alameda-Pineda, Olivier Perrotin
Embodied conversational agents require synchronized full-body motion (body gestures and facial expressions) that aligns with speech and emotional state. Omni-modal large language m…
InsideSSL: Understanding Self-Supervised Speech Representations using a Model-Centric Perspective
Samir Sadok, Xavier Alameda-Pineda
Self-supervised learning (SSL) models, such as Wav2Vec2, HuBERT, and WavLM, have become foundational across a wide range of speech and audio tasks. Despite their success, understan…
The Equalizer: Introducing Shape-Gain Decomposition in Neural Audio Codecs
Samir Sadok, Laurent Girin, Xavier Alameda-Pineda
Neural audio codecs (NACs) typically encode the short-term energy (gain) and normalized structure (shape) of speech/audio signals jointly within the same latent space. As a result,…
Residual Tokens Enhance Masked Autoencoders for Speech Modeling
Samir Sadok, Stéphane Lathuilière, Xavier Alameda-Pineda
Recent speech modeling relies on explicit attributes such as pitch, content, and speaker identity, but these alone cannot capture the full richness of natural speech. We introduce…
OpenSocInt: A Multi-modal Training Environment for Human-Aware Social Navigation
Victor Sanchez, Chris Reinke, Ahamed Mohamed +1
In this paper, we introduce OpenSocInt, an open-source software package providing a simulator for multi-modal social interactions and a modular architecture to train social agents.…
Diffusion-based Frameworks for Unsupervised Speech Enhancement
Jean-Eudes Ayilo, Mostafa Sadeghi, Romain Serizel +1
This paper addresses unsupervised diffusion-based single-channel speech enhancement (SE). Prior work in this direction combines a score-based diffusion model trained on clean speec…
Modeling strategies for speech enhancement in the latent space of a neural audio codec
Sofiene Kammoun, Xavier Alameda-Pineda, Simon Leglaive
Neural audio codecs (NACs) provide compact latent speech representations in the form of sequences of continuous vectors or discrete tokens. In this work, we investigate how these t…
Posterior Transition Modeling for Unsupervised Diffusion-Based Speech Enhancement
Mostafa Sadeghi, Jean-Eudes Ayilo, Romain Serizel +1
We explore unsupervised speech enhancement using diffusion models as expressive generative priors for clean speech. Existing approaches guide the reverse diffusion process using no…
AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder
Samir Sadok, Simon Leglaive, Laurent Girin +2
This article introduces AnCoGen, a novel method that leverages a masked autoencoder to unify the analysis, control, and generation of speech signals within a single model. AnCoGen…
Diffusion-based Unsupervised Audio-visual Speech Enhancement
Jean-Eudes Ayilo, Mostafa Sadeghi, Romain Serizel +1
This paper proposes a new unsupervised audio-visual speech enhancement (AVSE) approach that combines a diffusion-based audio-visual speech generative model with a non-negative matr…
MEGA: Masked Generative Autoencoder for Human Mesh Recovery
Guénolé Fiche, Simon Leglaive, Xavier Alameda-Pineda +1
Human Mesh Recovery (HMR) from a single RGB image is a highly ambiguous problem, as an infinite set of 3D interpretations can explain the 2D observation equally well. Nevertheless,…
Socially Pertinent Robots in Gerontological Healthcare
Xavier Alameda-Pineda, Angus Addlesee, Daniel Hernández García +41
Despite the many recent achievements in developing and deploying social robotics, there are still many underexplored environments and applications for which systematic evaluation o…
Mixture of Dynamical Variational Autoencoders for Multi-Source Trajectory Modeling and Separation
Xiaoyu Lin, Laurent Girin, Xavier Alameda-Pineda
In this paper, we propose a latent-variable generative model called mixture of dynamical variational autoencoders (MixDVAE) to model the dynamics of a system composed of multiple m…
VQ-HPS: Human Pose and Shape Estimation in a Vector-Quantized Latent Space
Guénolé Fiche, Simon Leglaive, Xavier Alameda-Pineda +2
Previous works on Human Pose and Shape Estimation (HPSE) from RGB images can be broadly categorized into two main groups: parametric and non-parametric approaches. Parametric techn…
Univariate Radial Basis Function Layers: Brain-inspired Deep Neural Layers for Low-Dimensional Inputs
Daniel Jost, Basavasagar Patil, Xavier Alameda-Pineda +1
Deep Neural Networks (DNNs) became the standard tool for function approximation with most of the introduced architectures being developed for high-dimensional input data. However,…
On the Effectiveness of LayerNorm Tuning for Continual Learning in Vision Transformers
Thomas De Min, Massimiliano Mancini, Karteek Alahari +2
State-of-the-art rehearsal-free continual learning methods exploit the peculiarities of Vision Transformers to learn task-specific prompts, drastically reducing catastrophic forget…
A Comprehensive Multi-scale Approach for Speech and Dynamics Synchrony in Talking Head Generation
Louis Airale, Dominique Vaufreydaz, Xavier Alameda-Pineda
Animating still face images with deep generative models using a speech input signal is an active research topic and has seen important recent progress.However, much of the effort h…
Unsupervised speech enhancement with deep dynamical generative speech and noise models
Xiaoyu Lin, Simon Leglaive, Laurent Girin +1
This work builds on a previous work on unsupervised speech enhancement using a dynamical variational autoencoder (DVAE) as the clean speech model and non-negative matrix factorizat…
Semi-supervised learning made simple with self-supervised clustering
Enrico Fini, Pietro Astolfi, Karteek Alahari +4
Self-supervised learning models have been shown to learn rich visual representations without requiring human annotations. However, in many real-world scenarios, labels are partiall…
Motion-DVAE: Unsupervised learning for fast human motion denoising
Guénolé Fiche, Simon Leglaive, Xavier Alameda-Pineda +1
Pose and motion priors are crucial for recovering realistic and accurate human motion from noisy observations. Substantial progress has been made on pose and shape estimation from…
A multimodal dynamical variational autoencoder for audiovisual speech representation learning
Samir Sadok, Simon Leglaive, Laurent Girin +2
In this paper, we present a multimodal and dynamical VAE (MDVAE) applied to unsupervised audio-visual speech representation learning. The latent space is structured to dissociate t…
Speech Modeling with a Hierarchical Transformer Dynamical VAE
Xiaoyu Lin, Xiaoyu Bie, Simon Leglaive +2
The dynamical variational autoencoders (DVAEs) are a family of latent-variable deep generative models that extends the VAE to model a sequence of observed data and a corresponding…