95 citations · 559 across the 53 of their papers we have counts for
23 papers · 1 filter
Universal Sound Separation with Self-Supervised Audio Masked Autoencoder
Junqi Zhao, Xubo Liu, Jinzheng Zhao +4
Universal sound separation (USS) is a task of separating mixtures of arbitrary sound sources. Typically, universal separation models are trained from scratch in a supervised manner…
WavCraft: Audio Editing and Generation with Large Language Models
Jinhua Liang, Huan Zhang, Haohe Liu +7
We introduce WavCraft, a collective system that leverages large language models (LLMs) to connect diverse task-specific models for audio content creation and editing. Specifically,…
Acoustic Prompt Tuning: Empowering Large Language Models with Audition Capabilities
Jinhua Liang, Xubo Liu, Wenwu Wang +3
The auditory system plays a substantial role in shaping the overall human perceptual experience. While prevailing large language models (LLMs) and visual language models (VLMs) hav…
Separate Anything You Describe
Xubo Liu, Qiuqiang Kong, Yan Zhao +7
Language-queried audio source separation (LASS) is a new paradigm for computational auditory scene analysis (CASA). LASS aims to separate a target sound from an audio mixture given…
Knowledge Distillation for Efficient Audio-Visual Video Captioning
Özkan Çaylı, Xubo Liu, Volkan Kılıç +1
Automatically describing audio-visual content with texts, namely video captioning, has received significant attention due to its potential applications across diverse fields. Deep…
Dual Transformer Decoder based Features Fusion Network for Automated Audio Captioning
Jianyuan Sun, Xubo Liu, Xinhao Mei +3
Automated audio captioning (AAC) which generates textual descriptions of audio content. Existing AAC models achieve good results but only use the high-dimensional representation of…
Adapting Language-Audio Models as Few-Shot Audio Learners
Jinhua Liang, Xubo Liu, Haohe Liu +4
We presented the Treff adapter, a training-efficient adapter for CLAP, to boost zero-shot classification performance by making use of a small set of labelled data. Specifically, we…
Ontology-aware Learning and Evaluation for Audio Tagging
Haohe Liu, Qiuqiang Kong, Xubo Liu +3
This study defines a new evaluation metric for audio tagging tasks to overcome the limitation of the conventional mean average precision (mAP) metric, which treats different kinds…
Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention
Xubo Liu, Qiushi Huang, Xinhao Mei +10
Audio captioning aims to generate text descriptions of audio clips. In the real world, many objects produce similar sounds. How to accurately recognize ambiguous sounds is a major…
Simple Pooling Front-ends For Efficient Audio Classification
Xubo Liu, Haohe Liu, Qiuqiang Kong +3
Recently, there has been increasing interest in building efficient audio neural networks for on-device scenarios. Most existing approaches are designed to reduce the size of audio…
Segment-level Metric Learning for Few-shot Bioacoustic Event Detection
Haohe Liu, Xubo Liu, Xinhao Mei +3
Few-shot bioacoustic event detection is a task that detects the occurrence time of a novel sound given a few examples. Previous methods employ metric learning to build a latent spa…
Automated Audio Captioning: An Overview of Recent Progress and New Challenges
Xinhao Mei, Xubo Liu, Mark D. Plumbley +1
Automated audio captioning is a cross-modal translation task that aims to generate natural language descriptions for given audio clips. This task has received increasing attention…
VoiceFixer: A Unified Framework for High-Fidelity Speech Restoration
Haohe Liu, Xubo Liu, Qiuqiang Kong +5
Speech restoration aims to remove distortions in speech signals. Prior methods mainly focus on a single type of distortion, such as speech denoising or dereverberation. However, sp…
Separate What You Describe: Language-Queried Audio Source Separation
Xubo Liu, Haohe Liu, Qiuqiang Kong +5
In this paper, we introduce the task of language-queried audio source separation (LASS), which aims to separate a target source from an audio mixture based on a natural language qu…
Neural Vocoder is All You Need for Speech Super-resolution
Haohe Liu, Woosung Choi, Xubo Liu +3
Speech super-resolution (SR) is a task to increase speech sampling rate by generating high-frequency components. Existing speech SR methods are trained in constrained experimental…
Leveraging Pre-trained BERT for Audio Captioning
Xubo Liu, Xinhao Mei, Qiushi Huang +6
Audio captioning aims at using natural language to describe the content of an audio clip. Existing audio captioning systems are generally based on an encoder-decoder architecture,…
Deep Neural Decision Forest for Acoustic Scene Classification
Jianyuan Sun, Xubo Liu, Xinhao Mei +4
Acoustic scene classification (ASC) aims to classify an audio clip based on the characteristic of the recording environment. In this regard, deep learning based approaches have eme…
On Metric Learning for Audio-Text Cross-Modal Retrieval
Xinhao Mei, Xubo Liu, Jianyuan Sun +2
Audio-text retrieval aims at retrieving a target audio clip or caption from a pool of candidates given a query in another modality. Solving such cross-modal retrieval task is chall…
Diverse Audio Captioning via Adversarial Training
Xinhao Mei, Xubo Liu, Jianyuan Sun +2
Audio captioning aims at generating natural language descriptions for audio clips automatically. Existing audio captioning models have shown promising improvement in recent years.…
An Encoder-Decoder Based Audio Captioning System With Transfer and Reinforcement Learning
Xinhao Mei, Qiushi Huang, Xubo Liu +10
Automated audio captioning aims to use natural language to describe the content of audio data. This paper presents an audio captioning system with an encoder-decoder architecture,…
Audio Captioning Transformer
Xinhao Mei, Xubo Liu, Qiushi Huang +2
Audio captioning aims to automatically generate a natural language description of an audio clip. Most captioning models follow an encoder-decoder architecture, where the decoder pr…
Conditional Sound Generation Using Neural Discrete Time-Frequency Representation Learning
Xubo Liu, Turab Iqbal, Jinzheng Zhao +3
Deep generative models have recently achieved impressive performance in speech and music synthesis. However, compared to the generation of those domain-specific sounds, generating…
CL4AC: A Contrastive Loss for Audio Captioning
Xubo Liu, Qiushi Huang, Xinhao Mei +4
Automated Audio captioning (AAC) is a cross-modal translation task that aims to use natural language to describe the content of an audio clip. As shown in the submissions received…