18 citations · 18 across the 8 of their papers we have counts for
10 papers
Rubric-Based Optimization for Text-to-Music Generation
Ping Wang, Guang Yang, Shao-Rong Su +3
Post-training text-to-music generation requires reward signals that capture multiple aspects of musical quality beyond what any single automatic metric can measure. We study struct…
Cocktail-Talker: Multi-Speaker Dialog Modeling in Noisy Social Environments with Turn Action GRPO
Xilin Jiang, Riki Shimizu, Sukru Samet Dindar +3
Spoken dialog systems are typically designed for clean, dyadic interactions in which a single user and an assistant take turns speaking. Real-world social conversations, however, a…
AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
Xilin Jiang, Qiaolin Wang, Junkai Wu +30
Internet audio-visual clips convey meaning through time-varying sound and motion, which extend beyond what text alone can represent. To examine whether AI models can understand suc…
SightSound-R1: Cross-Modal Reasoning Distillation from Vision to Audio Language Models
Qiaolin Wang, Xilin Jiang, Linyang He +2
While large audio-language models (LALMs) have demonstrated state-of-the-art audio understanding, their reasoning capability in complex soundscapes still falls behind large vision-…
Bridging Ears and Eyes: Analyzing Audio and Visual Large Language Models to Humans in Visible Sound Recognition and Reducing Their Sensory Gap via Cross-Modal Distillation
Xilin Jiang, Junkai Wu, Vishal Choudhari +1
Audio large language models (LLMs) are considered experts at recognizing sound objects, yet their performance relative to LLMs in other sensory modalities, such as visual or audio-…
Just ASR + LLM? A Study on Speech Large Language Models' Ability to Identify and Understand Speaker in Spoken Dialogue
Junkai Wu, Xulin Fan, Bo-Ru Lu +4
In recent years, we have observed a rapid advancement in speech language models (SpeechLLMs), catching up with humans' listening and reasoning abilities. SpeechLLMs have demonstrat…
Meta-AF Echo Cancellation for Improved Keyword Spotting
Jonah Casebeer, Junkai Wu, Paris Smaragdis
Adaptive filters (AFs) are vital for enhancing the performance of downstream tasks, such as speech recognition, sound event detection, and keyword spotting. However, traditional AF…
Unsupervised Improvement of Audio-Text Cross-Modal Representations
Zhepei Wang, Cem Subakan, Krishna Subramani +4
Recent advances in using language models to obtain cross-modal audio-text representations have overcome the limitations of conventional training approaches that use predefined labe…
Meta-Learning for Adaptive Filters with Higher-Order Frequency Dependencies
Junkai Wu, Jonah Casebeer, Nicholas J. Bryan +1
Adaptive filters are applicable to many signal processing tasks including acoustic echo cancellation, beamforming, and more. Adaptive filters are typically controlled using algorit…
Learning Representations for New Sound Classes With Continual Self-Supervised Learning
Zhepei Wang, Cem Subakan, Xilin Jiang +4
In this paper, we work on a sound recognition system that continually incorporates new sound classes. Our main goal is to develop a framework where the model can be updated without…