6 papers
Randomized Scores and Diverse Timbres: Augmenting Automatic Music Transcription with Online-Generated Data
Haiwen Xia, Chao Zhang, Qiuqiang Kong
Automatic music transcription (AMT) is limited by the scarcity of audio recordings paired with precise symbolic annotations. Synthetic data can provide supervision at scale, but it…
OmniVChat: Synthesizing, Benchmarking, and Training for Native Audio-Visual Dialogue
Haolin He, Yunfei Chu, Qi Chen +15
We define OmniVChat (Omni Video Chat) as the task of native audio-visual dialogue between a user and an omni model. In OmniVChat, omni models directly and simultaneously receive au…
Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction
Qi Chen, Yunfei Chu, Haolin He +15
Natural audio-visual interaction is emerging as an important interface for AI assistants, allowing users to communicate through speech and vision rather than carefully composed tex…
MelTok: 2D Tokenization for Single-Codebook Audio Compression
Jingyi Li, Zhiyuan Zhao, Zhisheng Zhang +6
Large Audio Language Models (LALMs) have emerged with strong performance across diverse audio understanding tasks and can be further enhanced by neural audio codecs. Transitioning…
CLEAR: Continuous Latent Autoregressive Modeling for High-quality and Low-latency Speech Synthesis
Chun Yat Wu, Jiajun Deng, Guinan Li +2
Autoregressive (AR) language models have emerged as powerful solutions for zero-shot text-to-speech (TTS) synthesis, capable of generating natural speech from a few seconds of audi…
Extract and Diffuse: Latent Integration for Improved Diffusion-based Speech and Vocal Enhancement
Yudong Yang, Zhan Liu, Wenyi Yu +3
Diffusion-based generative models have recently achieved remarkable results in speech and vocal enhancement due to their ability to model complex speech data distributions. While t…