138 citations · 1.2k across the 72 of their papers we have counts for
74 papers
Behavior-Conditioned Animal Vocalization Generation: Event-Aware Modeling and Multidimensional Evaluation
Xinlong Deng, Yong Tian, Jian Guan +3
Behavior-conditioned animal vocalization generation aims to synthesize vocalizations from natural-language descriptions of animal behaviors, social interactions, and contexts. This…
SAID: Semantic Acoustic Imaging Detector for Sound Event Localization and Detection
Runbang Wang, Zining Liang, Yin Cao +1
In daily life, people hear speech, footsteps, and music around them. We can often recognize these sounds and judge where they come from. Each sound source can be shown on a separat…
Region-Specific Audio Tagging for Spatial Sound
Jinzheng Zhao, Yong Xu, Haohe Liu +6
Audio tagging aims to label sound events appearing in an audio recording. In this paper, we propose region-specific audio tagging, a new task which labels sound events in a given r…
Universal Sound Separation with Self-Supervised Audio Masked Autoencoder
Junqi Zhao, Xubo Liu, Jinzheng Zhao +4
Universal sound separation (USS) is a task of separating mixtures of arbitrary sound sources. Typically, universal separation models are trained from scratch in a supervised manner…
WavCraft: Audio Editing and Generation with Large Language Models
Jinhua Liang, Huan Zhang, Haohe Liu +7
We introduce WavCraft, a collective system that leverages large language models (LLMs) to connect diverse task-specific models for audio content creation and editing. Specifically,…
Selective-Memory Meta-Learning with Environment Representations for Sound Event Localization and Detection
Jinbo Hu, Yin Cao, Ming Wu +4
Environment shifts and conflicts present significant challenges for learning-based sound event localization and detection (SELD) methods. SELD systems, when trained in particular a…
MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning
Dichucheng Li, Yinghao Ma, Weixing Wei +6
Instrument playing techniques (IPTs) constitute a pivotal component of musical expression. However, the development of automatic IPT detection methods suffers from limited labeled…
Transformer-based Autoencoder with ID Constraint for Unsupervised Anomalous Sound Detection
Jian Guan, Youde Liu, Qiuqiang Kong +4
Unsupervised anomalous sound detection (ASD) aims to detect unknown anomalous sounds of devices when only normal sound data is available. The autoencoder (AE) and self-supervised l…
AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining
Haohe Liu, Yi Yuan, Xubo Liu +7
Although audio generation shares commonalities across different types of audio, such as speech, music, and sound effects, designing models for each type requires careful considerat…
Separate Anything You Describe
Xubo Liu, Qiuqiang Kong, Yan Zhao +7
Language-queried audio source separation (LASS) is a new paradigm for computational auditory scene analysis (CASA). LASS aims to separate a target sound from an audio mixture given…
WavJourney: Compositional Audio Creation with Large Language Models
Xubo Liu, Zhongkai Zhu, Haohe Liu +8
Despite breakthroughs in audio generation models, their capabilities are often confined to domain-specific conditions such as speech transcriptions and audio captions. However, rea…
Universal Source Separation with Weakly Labelled Data
Qiuqiang Kong, Ke Chen, Haohe Liu +4
Universal source separation (USS) is a fundamental research task for computational auditory scene analysis, which aims to separate mono recordings into individual source tracks. Th…
A unified front-end framework for English text-to-speech synthesis
Zelin Ying, Chen Li, Yu Dong +4
The front-end is a critical component of English text-to-speech (TTS) systems, responsible for extracting linguistic features that are essential for a text-to-speech model to synth…
Multi-level Temporal-channel Speaker Retrieval for Zero-shot Voice Conversion
Zhichao Wang, Liumeng Xue, Qiuqiang Kong +4
Zero-shot voice conversion (VC) converts source speech into the voice of any desired speaker using only one utterance of the speaker without requiring additional model updates. Typ…
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research
Xinhao Mei, Chutong Meng, Haohe Liu +6
The advancement of audio-language (AL) multimodal learning tasks has been significant in recent years. However, researchers face challenges due to the costly and time-consuming col…
Jointist: Simultaneous Improvement of Multi-instrument Transcription and Music Source Separation via Joint Training
Kin Wai Cheuk, Keunwoo Choi, Qiuqiang Kong +5
In this paper, we introduce Jointist, an instrument-aware multi-instrument framework that is capable of transcribing, recognizing, and separating multiple musical instruments from…
Ontology-aware Learning and Evaluation for Audio Tagging
Haohe Liu, Qiuqiang Kong, Xubo Liu +3
This study defines a new evaluation metric for audio tagging tasks to overcome the limitation of the conventional mean average precision (mAP) metric, which treats different kinds…
Binaural Rendering of Ambisonic Signals by Neural Networks
Yin Zhu, Qiuqiang Kong, Junjie Shi +4
Binaural rendering of ambisonic signals is of broad interest to virtual reality and immersive media. Conventional methods often require manually measured Head-Related Transfer Func…
Streaming Voice Conversion Via Intermediate Bottleneck Features And Non-streaming Teacher Guidance
Yuanzhe Chen, Ming Tu, Tang Li +7
Streaming voice conversion (VC) is the task of converting the voice of one person to another in real-time. Previous streaming VC methods use phonetic posteriorgrams (PPGs) extracte…
Neural Sound Field Decomposition with Super-resolution of Sound Direction
Qiuqiang Kong, Shilei Liu, Junjie Shi +5
Sound field decomposition predicts waveforms in arbitrary directions using signals from a limited number of microphones as inputs. Sound field decomposition is fundamental to downs…
Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention
Xubo Liu, Qiushi Huang, Xinhao Mei +10
Audio captioning aims to generate text descriptions of audio clips. In the real world, many objects produce similar sounds. How to accurately recognize ambiguous sounds is a major…
Learning Temporal Resolution in Spectrogram for Audio Classification
Haohe Liu, Xubo Liu, Qiuqiang Kong +2
The audio spectrogram is a time-frequency representation that has been widely used for audio classification. One of the key attributes of the audio spectrogram is the temporal reso…
Simple Pooling Front-ends For Efficient Audio Classification
Xubo Liu, Haohe Liu, Qiuqiang Kong +3
Recently, there has been increasing interest in building efficient audio neural networks for on-device scenarios. Most existing approaches are designed to reduce the size of audio…
Sound Event Localization and Detection for Real Spatial Sound Scenes: Event-Independent Network and Data Augmentation Chains
Jinbo Hu, Yin Cao, Ming Wu +4
Sound event localization and detection (SELD) is a joint task of sound event detection and direction-of-arrival estimation. In DCASE 2022 Task 3, types of data transform from compu…