47 citations · 115 across the 22 of their papers we have counts for
14 papers · 1 filter
Conversational Voice Aesthetic Model with Reinforcement Learning from Human Listeners
Xilin Jiang, Shun Zhang, Tejas Jayashankar +2
We introduce Conversational Voice Aesthetic Model, a speech large language model for describing the voice aesthetics of real or synthetic speech responses in natural conversational…
MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flow
Riki Shimizu, Xilin Jiang, Nima Mesgarani
Target speaker extraction (TSE) aims to isolate a desired speaker's voice from a multi-speaker mixture using auxiliary information such as a reference utterance. Although recent ad…
DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesis
Yinghao Aaron Li, Xilin Jiang, Fei Tao +4
Diffusion-based text-to-speech (TTS) systems have made remarkable progress in zero-shot speech synthesis, yet optimizing all components for perceptual metrics remains challenging.…
ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
Zhongweiyang Xu, Xulin Fan, Zhong-Qiu Wang +2
Blind Speech Separation (BSS) aims to separate multiple speech sources from audio mixtures recorded by a microphone array. The problem is challenging because it is a blind inverse…
Exploring Finetuned Audio-LLM on Heart Murmur Features
Adrian Florea, Xilin Jiang, Nima Mesgarani +1
Large language models (LLMs) for audio have excelled in recognizing and analyzing human speech, music, and environmental sounds. However, their potential for understanding other ty…
StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusion
Yinghao Aaron Li, Xilin Jiang, Cong Han +1
The rapid development of large-scale text-to-speech (TTS) models has led to significant advancements in modeling diverse speaker prosody and voices. However, these models often fac…
Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesis
Xilin Jiang, Yinghao Aaron Li, Adrian Nicolas Florea +2
It is too early to conclude that Mamba is a better alternative to transformers for speech before comparing Mamba with transformers in terms of both performance and efficiency in mu…
SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Model
Siavash Shams, Sukru Samet Dindar, Xilin Jiang +1
Transformers have revolutionized deep learning across various tasks, including audio representation learning, due to their powerful modeling capabilities. However, they often suffe…
Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation
Xilin Jiang, Cong Han, Nima Mesgarani
Transformers have been the most successful architecture for various speech modeling tasks, including speech separation. However, the self-attention mechanism in transformers with q…
Listen, Chat, and Remix: Text-Guided Soundscape Remixing for Enhanced Auditory Experience
Xilin Jiang, Cong Han, Yinghao Aaron Li +1
In daily life, we encounter a variety of sounds, both desirable and undesirable, with limited control over their presence and volume. Our work introduces "Listen, Chat, and Remix"…
Exploring Self-Supervised Contrastive Learning of Spatial Sound Event Representation
Xilin Jiang, Cong Han, Yinghao Aaron Li +1
In this study, we present a simple multi-channel framework for contrastive learning (MC-SimCLR) to encode 'what' and 'where' of spatial audios. MC-SimCLR learns joint spectral and…
HiFTNet: A Fast High-Quality Neural Vocoder with Harmonic-plus-Noise Filter and Inverse Short Time Fourier Transform
Yinghao Aaron Li, Cong Han, Xilin Jiang +1
Recent advancements in speech synthesis have leveraged GAN-based networks like HiFi-GAN and BigVGAN to produce high-fidelity waveforms from mel-spectrograms. However, these network…
DeCoR: Defy Knowledge Forgetting by Predicting Earlier Audio Codes
Xilin Jiang, Yinghao Aaron Li, Nima Mesgarani
Lifelong audio feature extraction involves learning new sound classes incrementally, which is essential for adapting to new data distributions over time. However, optimizing the mo…
Learning Representations for New Sound Classes With Continual Self-Supervised Learning
Zhepei Wang, Cem Subakan, Xilin Jiang +4
In this paper, we work on a sound recognition system that continually incorporates new sound classes. Our main goal is to develop a framework where the model can be updated without…