11 citations · 30 across the 24 of their papers we have counts for
26 papers
AlignAudio: A Unified Dual-Alignment Framework for Noise-Robust Speech-Conditioned Generation
Yixuan Li, Xuenan Xu, Zeyu Xie +1
Speech-conditioned generation maps speech signals to target audio outputs. Prior end-to-end speech-conditioned generative models achieve high fidelity in clean acoustic conditions,…
AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing
Ziyang Ma, Zhikang Niu, Wenming Tu +30
We introduce AuK, an open-source foundational model that unifies speech generation and editing through a common interface of natural-language instructions and audio context. To sup…
MMAE: A Massive Multitask Audio Editing Benchmark
Ziyang Ma, Ruiqi Yan, Ruiyang Xu +35
We introduce MMAE, a Massive Multitask Audio Editing benchmark, serving as the first comprehensive evaluation testbed designed for general-purpose instruction-based audio editing.…
WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modeling
Guanrou Yang, Tian Tan, Qian Chen +12
Integrating speech understanding and generation is a pivotal step toward building unified speech models. However, the different representations required for these two tasks current…
Covo-Audio Technical Report
Wenfu Wang, Chenxing Li, Liqiang Zhang +23
In this work, we present Covo-Audio, a 7B-parameter end-to-end LALM that directly processes continuous audio inputs and generates audio outputs within a single unified architecture…
SemanticVocoder: Bridging Audio Generation and Audio Understanding via Semantic Latents
Zeyu Xie, Chenxing Li, Qiao Jin +6
Recent audio generation models typically rely on Variational Autoencoders (VAEs) and perform generation within the VAE latent space. Although VAEs excel at compression and reconstr…
PicoAudio2: Temporal Controllable Text-to-Audio Generation with Natural Language Description
Zihao Zheng, Zeyu Xie, Xuenan Xu +3
While recent work in controllable text-to-audio (TTA) generation has achieved fine-grained control through timestamp conditioning, its scope remains limited by audio quality and in…
LARA-Gen: Enabling Continuous Emotion Control for Music Generation Models via Latent Affective Representation Alignment
Jiahao Mei, Xuenan Xu, Shimeng Li +7
Recent advances in text-to-music models have enabled coherent music generation from text prompts, yet fine-grained emotional control remains unresolved. We introduce LARA-Gen, a fr…
When Audio Generators Become Good Listeners: Generative Features for Understanding Tasks
Zeyu Xie, Chenxing Li, Xuenan Xu +6
This work pioneers the utilization of generative features in enhancing audio understanding. Unlike conventional discriminative features that directly optimize posterior and thus em…
UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalities
Xuenan Xu, Jiahao Mei, Zihao Zheng +9
Audio generation, including speech, music and sound effects, has advanced rapidly in recent years. These tasks can be divided into two categories: time-aligned (TA) tasks, where ea…
FakeSound2: A Benchmark for Explainable and Generalizable Deepfake Sound Detection
Zeyu Xie, Yaoyun Zhang, Xuenan Xu +4
The rapid development of generative audio raises ethical and security concerns stemming from forged data, making deepfake sound detection an important safeguard against the malicio…
STAR: Speech-to-Audio Generation via Representation Learning
Zeyu Xie, Xuenan Xu, Yixuan Li +2
This work presents STAR, the first end-to-end speech-to-audio generation framework, designed to enhance efficiency and address error propagation inherent in cascaded systems. Unlik…
Overview of the Amphion Toolkit (v0.2)
Jiaqi Li, Xueyao Zhang, Yuancheng Wang +9
Amphion is an open-source toolkit for Audio, Music, and Speech Generation, designed to lower the entry barrier for junior researchers and engineers in these fields. It provides a v…
DiveSound: LLM-Assisted Automatic Taxonomy Construction for Diverse Audio Generation
Baihan Li, Zeyu Xie, Xuenan Xu +5
Audio generation has attracted significant attention. Despite remarkable enhancement in audio quality, existing models overlook diversity evaluation. This is partially due to the l…
PicoAudio: Enabling Precise Timestamp and Frequency Controllability of Audio Events in Text-to-audio Generation
Zeyu Xie, Xuenan Xu, Zhizheng Wu +1
Recently, audio generation tasks have attracted considerable research interests. Precise temporal controllability is essential to integrate audio generation with real applications.…
AudioTime: A Temporally-aligned Audio-text Benchmark Dataset
Zeyu Xie, Xuenan Xu, Zhizheng Wu +1
Recent advancements in audio generation have enabled the creation of high-fidelity audio clips from free-form textual descriptions. However, temporal relationships, a critical feat…
FakeSound: Deepfake General Audio Detection
Zeyu Xie, Baihan Li, Xuenan Xu +3
With the advancement of audio generation, generative models can produce highly realistic audios. However, the proliferation of deepfake general audio can pose negative consequences…
A Detailed Audio-Text Data Simulation Pipeline using Single-Event Sounds
Xuenan Xu, Xiaohang Xu, Zeyu Xie +3
Recently, there has been an increasing focus on audio-text cross-modal learning. However, most of the existing audio-text datasets contain only simple descriptions of sound events.…
Enhancing Audio Generation Diversity with Visual Information
Zeyu Xie, Baihan Li, Xuenan Xu +2
Audio and sound generation has garnered significant attention in recent years, with a primary focus on improving the quality of generated audios. However, there has been limited re…
Phonetic and Lexical Discovery of a Canine Language using HuBERT
Xingyuan Li, Sinong Wang, Zeyu Xie +2
This paper delves into the pioneering exploration of potential communication patterns within dog vocalizations and transcends traditional linguistic analysis barriers, which heavil…
Improving Audio Caption Fluency with Automatic Error Correction
Hanxue Zhang, Zeyu Xie, Xuenan Xu +2
Automated audio captioning (AAC) is an important cross-modality translation task, aiming at generating descriptions for audio clips. However, captions generated by previous AAC mod…
Enhance Temporal Relations in Audio Captioning with Sound Event Detection
Zeyu Xie, Xuenan Xu, Mengyue Wu +1
Automated audio captioning aims at generating natural language descriptions for given audio clips, not only detecting and classifying sounds, but also summarizing the relationships…
BLAT: Bootstrapping Language-Audio Pre-training based on AudioSet Tag-guided Synthetic Data
Xuenan Xu, Zhiling Zhang, Zelin Zhou +4
Compared with ample visual-text pre-training research, few works explore audio-text pre-training, mostly due to the lack of sufficient parallel audio-text data. Most existing metho…
Beyond the Status Quo: A Contemporary Survey of Advances and Challenges in Audio Captioning
Xuenan Xu, Zeyu Xie, Mengyue Wu +1
Automated audio captioning (AAC), a task that mimics human perception as well as innovatively links audio processing and natural language processing, has overseen much progress ove…