28 citations · 47 across the 12 of their papers we have counts for
13 papers
Prevailing Research Areas for Music AI in the Era of Foundation Models
Megan Wei, Mateusz Modrzejewski, Aswin Sivaraman +1
Parallel to rapid advancements in foundation model research, the past few years have witnessed a surge in music AI applications. As AI-generated and AI-augmented music become incre…
Multimodal Large Language Models with Fusion Low Rank Adaptation for Device Directed Speech Detection
Shruti Palaskar, Oggi Rudovic, Sameer Dharur +7
Although Large Language Models (LLMs) have shown promise for human-like conversations, they are primarily pre-trained on text data. Incorporating audio or video improves performanc…
The Potential of Neural Speech Synthesis-based Data Augmentation for Personalized Speech Enhancement
Anastasia Kuznetsova, Aswin Sivaraman, Minje Kim
With the advances in deep learning, speech enhancement systems benefited from large neural network architectures and achieved state-of-the-art quality. However, speaker-agnostic me…
Adapting Speech Separation to Real-World Meetings Using Mixture Invariant Training
Aswin Sivaraman, Scott Wisdom, Hakan Erdogan +1
The recently-proposed mixture invariant training (MixIT) is an unsupervised method for training single-channel sound separation models in the sense that it does not require ground-…
Zero-Shot Personalized Speech Enhancement through Speaker-Informed Model Selection
Aswin Sivaraman, Minje Kim
This paper presents a novel zero-shot learning approach towards personalized speech enhancement through the use of a sparsely active ensemble model. Optimizing speech denoising sys…
Personalized Speech Enhancement through Self-Supervised Data Augmentation and Purification
Aswin Sivaraman, Sunwoo Kim, Minje Kim
Training personalized speech enhancement models is innately a no-shot learning problem due to privacy constraints and limited access to noise-free speech from the target user. If t…
Efficient Personalized Speech Enhancement through Self-Supervised Learning
Aswin Sivaraman, Minje Kim
This work presents self-supervised learning methods for developing monaural speaker-specific (i.e., personalized) speech enhancement models. While generalist models must broadly ad…
Detecting Extraneous Content in Podcasts
Sravana Reddy, Yongze Yu, Aasish Pappu +3
Podcast episodes often contain material extraneous to the main content, such as advertisements, interleaved within the audio and the written descriptions. We present classifiers th…
Self-Supervised Learning from Contrastive Mixtures for Personalized Speech Enhancement
Aswin Sivaraman, Minje Kim
This work explores how self-supervised learning can be universally used to discover speaker-specific features towards enabling personalized speech enhancement models. We specifical…
Sparse Mixture of Local Experts for Efficient Speech Enhancement
Aswin Sivaraman, Minje Kim
In this paper, we investigate a deep learning approach for speech denoising through an efficient ensemble of specialist neural networks. By splitting up the speech denoising task i…
Deep Autotuner: A Data-Driven Approach to Natural-Sounding Pitch Correction for Singing Voice in Karaoke Performances
Sanna Wager, George Tzanetakis, Cheng-i Wang +3
We describe a machine-learning approach to pitch correcting a solo singing performance in a karaoke setting, where the solo voice and accompaniment are on separate tracks. The prop…
A Data-Driven Approach to Smooth Pitch Correction for Singing Voice in Pop Music
Sanna Wager, Lijiang Guo, Aswin Sivaraman +1
In this paper, we present a machine-learning approach to pitch correction for voice in a karaoke setting, where the vocals and accompaniment are on separate tracks and time-aligned…
On Psychoacoustically Weighted Cost Functions Towards Resource-Efficient Deep Neural Networks for Speech Denoising
Kai Zhen, Aswin Sivaraman, Jongmo Sung +1
We present a psychoacoustically enhanced cost function to balance network complexity and perceptual performance of deep neural networks for speech denoising. While training the net…