57 citations · 384 across the 43 of their papers we have counts for
25 papers · 1 filter
DiveSound: LLM-Assisted Automatic Taxonomy Construction for Diverse Audio Generation
Baihan Li, Zeyu Xie, Xuenan Xu +5
Audio generation has attracted significant attention. Despite remarkable enhancement in audio quality, existing models overlook diversity evaluation. This is partially due to the l…
PicoAudio: Enabling Precise Timestamp and Frequency Controllability of Audio Events in Text-to-audio Generation
Zeyu Xie, Xuenan Xu, Zhizheng Wu +1
Recently, audio generation tasks have attracted considerable research interests. Precise temporal controllability is essential to integrate audio generation with real applications.…
AudioTime: A Temporally-aligned Audio-text Benchmark Dataset
Zeyu Xie, Xuenan Xu, Zhizheng Wu +1
Recent advancements in audio generation have enabled the creation of high-fidelity audio clips from free-form textual descriptions. However, temporal relationships, a critical feat…
FakeSound: Deepfake General Audio Detection
Zeyu Xie, Baihan Li, Xuenan Xu +3
With the advancement of audio generation, generative models can produce highly realistic audios. However, the proliferation of deepfake general audio can pose negative consequences…
SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound
Haohe Liu, Xuenan Xu, Yi Yuan +3
Large language models (LLMs) have significantly advanced audio processing through audio codecs that convert audio into discrete tokens, enabling the application of language modelli…
A Detailed Audio-Text Data Simulation Pipeline using Single-Event Sounds
Xuenan Xu, Xiaohang Xu, Zeyu Xie +3
Recently, there has been an increasing focus on audio-text cross-modal learning. However, most of the existing audio-text datasets contain only simple descriptions of sound events.…
Enhancing Audio Generation Diversity with Visual Information
Zeyu Xie, Baihan Li, Xuenan Xu +2
Audio and sound generation has garnered significant attention in recent years, with a primary focus on improving the quality of generated audios. However, there has been limited re…
Towards Weakly Supervised Text-to-Audio Grounding
Xuenan Xu, Ziyang Ma, Mengyue Wu +1
Text-to-audio grounding (TAG) task aims to predict the onsets and offsets of sound events described by natural language. This task can facilitate applications such as multimodal in…
Towards Lexical Analysis of Dog Vocalizations via Online Videos
Yufei Wang, Chunhao Zhang, Jieyi Huang +2
Deciphering the semantics of animal language has been a grand challenge. This study presents a data-driven investigation into the semantics of dog vocalizations via correlating dif…
Does My Dog ''Speak'' Like Me? The Acoustic Correlation between Pet Dogs and Their Human Owners
Jieyi Huang, Chunhao Zhang, Yufei Wang +2
How hosts language influence their pets' vocalization is an interesting yet underexplored problem. This paper presents a preliminary investigation into the possible correlation bet…
Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning
Luoyi Sun, Xuenan Xu, Mengyue Wu +1
Recently, the AI community has made significant strides in developing powerful foundation models, driven by large-scale multimodal datasets. However, for audio representation learn…
Enhance Temporal Relations in Audio Captioning with Sound Event Detection
Zeyu Xie, Xuenan Xu, Mengyue Wu +1
Automated audio captioning aims at generating natural language descriptions for given audio clips, not only detecting and classifying sounds, but also summarizing the relationships…
Diverse and Vivid Sound Generation from Text Descriptions
Guangwei Li, Xuenan Xu, Lingfeng Dai +2
Previous audio generation mainly focuses on specified sound classes such as speech or music, whose form and content are greatly restricted. In this paper, we go beyond specific aud…
BLAT: Bootstrapping Language-Audio Pre-training based on AudioSet Tag-guided Synthetic Data
Xuenan Xu, Zhiling Zhang, Zelin Zhou +4
Compared with ample visual-text pre-training research, few works explore audio-text pre-training, mostly due to the lack of sufficient parallel audio-text data. Most existing metho…
Beyond the Status Quo: A Contemporary Survey of Advances and Challenges in Audio Captioning
Xuenan Xu, Zeyu Xie, Mengyue Wu +1
Automated audio captioning (AAC), a task that mimics human perception as well as innovatively links audio processing and natural language processing, has overseen much progress ove…
Audio-text Retrieval in Context
Siyu Lou, Xuenan Xu, Mengyue Wu +1
Audio-text retrieval based on natural language descriptions is a challenging task. It involves learning cross-modality alignments between long sequences under inadequate data condi…
Enriching Ontology with Temporal Commonsense for Low-Resource Audio Tagging
Zhiling Zhang, Zelin Zhou, Haifeng Tang +3
Audio tagging aims at predicting sound events occurred in a recording. Traditional models require enormous laborious annotations, otherwise performance degeneration will be the nor…
Can Audio Captions Be Evaluated with Image Caption Metrics?
Zelin Zhou, Zhiling Zhang, Xuenan Xu +3
Automated audio captioning aims at generating textual descriptions for an audio clip. To evaluate the quality of generated audio captions, previous works directly adopt image capti…
Voice activity detection in the wild: A data-driven approach using teacher-student training
Heinrich Dinkel, Shuai Wang, Xuenan Xu +2
Voice activity detection is an essential pre-processing component for speech-related tasks such as automatic speech recognition (ASR). Traditional supervised VAD systems obtain fra…
Text-to-Audio Grounding: Building Correspondence Between Captions and Sound Events
Xuenan Xu, Heinrich Dinkel, Mengyue Wu +1
Automated Audio Captioning is a cross-modal task, generating natural language descriptions to summarize the audio clips' sound events. However, grounding the actual sound events in…
Investigating Local and Global Information for Automated Audio Captioning with Transfer Learning
Xuenan Xu, Heinrich Dinkel, Mengyue Wu +2
Automated audio captioning (AAC) aims at generating summarizing descriptions for audio clips. Multitudinous concepts are described in an audio caption, ranging from local informati…
Towards duration robust weakly supervised sound event detection
Heinrich Dinkel, Mengyue Wu, Kai Yu
Sound event detection (SED) is the task of tagging the absence or presence of audio events and their corresponding interval within a given audio clip. While SED can be done using s…
Voice activity detection in the wild via weakly supervised sound event detection
Heinrich Dinkel, Yefei Chen, Mengyue Wu +1
Traditional supervised voice activity detection (VAD) methods work well in clean and controlled scenarios, with performance severely degrading in real-world applications. One possi…
Audio Caption in a Car Setting with a Sentence-Level Loss
Xuenan Xu, Heinrich Dinkel, Mengyue Wu +1
Captioning has attracted much attention in image and video understanding while a small amount of work examines audio captioning. This paper contributes a Mandarin-annotated dataset…