138 citations · 792 across the 57 of their papers we have counts for
32 papers · 1 filter
FlowSep 2: Self-Supervised Flow Matching for Language-Queried Audio Source Separation
Yi Yuan, Xubo Liu, Haohe Liu +3
Language-queried audio source separation (LASS) aims to extract target sources from audio mixtures according to natural language descriptions, offering a flexible and scalable inte…
DreamAudio: Customized Text-to-Audio Generation with Diffusion Models
Yi Yuan, Xubo Liu, Haohe Liu +5
With the development of large-scale diffusion-based and language-modeling-based generative models, impressive progress has been achieved in text-to-audio generation. Despite produc…
AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion
Junqi Zhao, Jinzheng Zhao, Haohe Liu +5
Diffusion models have significantly improved the quality and diversity of audio generation but are hindered by slow inference speed. Rectified flow enhances inference speed by lear…
EnvSDD: Benchmarking Environmental Sound Deepfake Detection
Han Yin, Yang Xiao, Rohan Kumar Das +4
Audio generation systems now create very realistic soundscapes that can enhance media production, but also pose potential risks. Several studies have examined deepfakes in speech o…
FlowSep: Language-Queried Sound Separation with Rectified Flow Matching
Yi Yuan, Xubo Liu, Haohe Liu +2
Language-queried audio source separation (LASS) focuses on separating sounds using textual descriptions of the desired sources. Current methods mainly use discriminative approaches…
Efficient Audio Captioning with Encoder-Level Knowledge Distillation
Xuenan Xu, Haohe Liu, Mengyue Wu +2
Significant improvement has been achieved in automated audio captioning (AAC) with recent models. However, these models have become increasingly large as their performance is enhan…
Zero-Shot Audio Captioning Using Soft and Hard Prompts
Yiming Zhang, Xuenan Xu, Ruoyi Du +5
In traditional audio captioning methods, a model is usually trained in a fully supervised manner using a human-annotated dataset containing audio-text pairs and then evaluated on t…
SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound
Haohe Liu, Xuenan Xu, Yi Yuan +3
Large language models (LLMs) have significantly advanced audio processing through audio codecs that convert audio into discrete tokens, enabling the application of language modelli…
T-CLAP: Temporal-Enhanced Contrastive Language-Audio Pretraining
Yi Yuan, Zhuo Chen, Xubo Liu +6
Contrastive language-audio pretraining~(CLAP) has been developed to align the representations of audio and language, achieving remarkable performance in retrieval and classificatio…
Balanced SNR-Aware Distillation for Guided Text-to-Audio Generation
Bingzhi Liu, Yin Cao, Haohe Liu +1
Diffusion models have demonstrated promising results in text-to-audio generation tasks. However, their practical usability is hindered by slow sampling speeds, limiting their appli…
First-Shot Unsupervised Anomalous Sound Detection With Unknown Anomalies Estimated by Metadata-Assisted Audio Generation
Hejing Zhang, Qiaoxi Zhu, Jian Guan +6
First-shot (FS) unsupervised anomalous sound detection (ASD) is a brand-new task introduced in DCASE 2023 Challenge Task 2, where the anomalous sounds for the target machine types…
Synth-AC: Enhancing Audio Captioning with Synthetic Supervision
Feiyang Xiao, Qiaoxi Zhu, Jian Guan +4
Data-driven approaches hold promise for audio captioning. However, the development of audio captioning methods can be biased due to the limited availability and quality of text-aud…
AudioSR: Versatile Audio Super-resolution at Scale
Haohe Liu, Ke Chen, Qiao Tian +2
Audio super-resolution is a fundamental task that predicts high-frequency components for low-resolution audio, enhancing audio quality in digital applications. Previous methods hav…
Retrieval-Augmented Text-to-Audio Generation
Yi Yuan, Haohe Liu, Xubo Liu +3
Despite recent progress in text-to-audio (TTA) generation, we show that the state-of-the-art models, such as AudioLDM, trained on datasets with an imbalanced class distribution, su…
Multimodal Fish Feeding Intensity Assessment in Aquaculture
Meng Cui, Xubo Liu, Haohe Liu +5
Fish feeding intensity assessment (FFIA) aims to evaluate fish appetite changes during feeding, which is crucial in industrial aquaculture applications. Existing FFIA methods are l…
MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies
Ke Chen, Yusong Wu, Haohe Liu +3
Diffusion models have shown promising results in cross-modal generation tasks, including text-to-image and text-to-audio generation. However, generating music, as a special type of…
AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining
Haohe Liu, Yi Yuan, Xubo Liu +7
Although audio generation shares commonalities across different types of audio, such as speech, music, and sound effects, designing models for each type requires careful considerat…
WavJourney: Compositional Audio Creation with Large Language Models
Xubo Liu, Zhongkai Zhu, Haohe Liu +8
Despite breakthroughs in audio generation models, their capabilities are often confined to domain-specific conditions such as speech transcriptions and audio captions. However, rea…
Text-Driven Foley Sound Generation With Latent Diffusion Model
Yi Yuan, Haohe Liu, Xubo Liu +4
Foley sound generation aims to synthesise the background sound for multimedia content. Previous models usually employ a large development set with labels as input (e.g., single num…
E-PANNs: Sound Recognition Using Efficient Pre-trained Audio Neural Networks
Arshdeep Singh, Haohe Liu, Mark D. Plumbley
Sounds carry an abundance of information about activities and events in our everyday environment, such as traffic noise, road works, music, or people talking. Recent machine learni…
Learning to detect an animal sound from five examples
Inês Nolasco, Shubhr Singh, Veronica Morfi +14
Automatic detection and classification of animal sounds has many applications in biodiversity monitoring and animal behaviour. In the past twenty years, the volume of digitised wil…
Universal Source Separation with Weakly Labelled Data
Qiuqiang Kong, Ke Chen, Haohe Liu +4
Universal source separation (USS) is a fundamental research task for computational auditory scene analysis, which aims to separate mono recordings into individual source tracks. Th…
Latent Diffusion Model Based Foley Sound Generation System For DCASE Challenge 2023 Task 7
Yi Yuan, Haohe Liu, Xubo Liu +3
Foley sound presents the background sound for multimedia content and the generation of Foley sound involves computationally modelling sound effects with specialized techniques. In…
Leveraging Pre-trained AudioLDM for Sound Generation: A Benchmark Study
Yi Yuan, Haohe Liu, Jinhua Liang +3
Deep neural networks have recently achieved breakthroughs in sound generation. Despite the outstanding sample quality, current sound generation models face issues on small-scale da…