138 citations · 371 across the 57 of their papers we have counts for
57 papers
ML-LMCL: Mutual Learning and Large-Margin Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding
Xuxin Cheng, Bowen Cao, Qichen Ye +3
Spoken language understanding (SLU) is a fundamental task in the task-oriented dialogue systems. However, the inevitable errors from automatic speech recognition (ASR) usually impa…
NADiffuSE: Noise-aware Diffusion-based Model for Speech Enhancement
Wen Wang, Dongchao Yang, Qichen Ye +2
The goal of speech enhancement (SE) is to eliminate the background interference from the noisy speech signal. Generative models such as diffusion models (DM) have been applied to t…
MultiCapCLIP: Auto-Encoding Prompts for Zero-Shot Multilingual Visual Captioning
Bang Yang, Fenglin Liu, Xian Wu +3
Supervised visual captioning models typically require a large scale of images or videos paired with descriptions in a specific language (i.e., the vision-caption pairs) for trainin…
Multimodal Prompt Learning for Product Title Generation with Extremely Limited Labels
Bang Yang, Fenglin Liu, Zheng Li +4
Generating an informative and attractive title for the product is a crucial task for e-commerce. Most existing works follow the standard multimodal natural language generation appr…
Improving Audio-Text Retrieval via Hierarchical Cross-Modal Interaction and Auxiliary Captions
Yifei Xin, Yuexian Zou
Most existing audio-text retrieval (ATR) methods focus on constructing contrastive pairs between whole audio clips and complete caption sentences, while ignoring fine-grained cross…
G2L: Semantically Aligned and Uniform Video Grounding via Geodesic and Game Theory
Hongxiang Li, Meng Cao, Xuxin Cheng +3
The recent video grounding works attempt to introduce vanilla contrastive learning into video grounding. However, we claim that this naive solution is suboptimal. Contrastive learn…
Customizing General-Purpose Foundation Models for Medical Report Generation
Bang Yang, Asif Raza, Yuexian Zou +1
Medical caption prediction which can be regarded as a task of medical report generation (MRG), requires the automatic generation of coherent and accurate captions for the given med…
HiFi-Codec: Group-residual Vector quantization for High Fidelity Audio Codec
Dongchao Yang, Songxiang Liu, Rongjie Huang +3
Audio codec models are widely used in audio communication as a crucial technique for compressing audio into discrete representations. Nowadays, audio codec models are increasingly…
TLAG: An Informative Trigger and Label-Aware Knowledge Guided Model for Dialogue-based Relation Extraction
Hao An, Dongsheng Chen, Weiyuan Xu +2
Dialogue-based Relation Extraction (DRE) aims to predict the relation type of argument pairs that are mentioned in dialogue. The latest trigger-enhanced methods propose trigger pre…
Improving Text-Audio Retrieval by Text-aware Attention Pooling and Prior Matrix Revised Loss
Yifei Xin, Dongchao Yang, Yuexian Zou
In text-audio retrieval (TAR) tasks, due to the heterogeneity of contents between text and audio, the semantic information contained in the text is only similar to certain frames w…
PoseRAC: Pose Saliency Transformer for Repetitive Action Counting
Ziyu Yao, Xuxin Cheng, Yuexian Zou
This paper presents a significant contribution to the field of repetitive action counting through the introduction of a new approach called Pose Saliency Representation. The propos…
FTM: A Frame-level Timeline Modeling Method for Temporal Graph Representation Learning
Bowen Cao, Qichen Ye, Weiyuan Xu +1
Learning representations for graph-structured data is essential for graph analytical tasks. While remarkable progress has been made on static graphs, researches on temporal graphs…
FiTs: Fine-grained Two-stage Training for Knowledge-aware Question Answering
Qichen Ye, Bowen Cao, Nuo Chen +2
Knowledge-aware question answering (KAQA) requires the model to answer questions over a knowledge base, which is essential for both open-domain QA and domain-specific QA, especiall…
Improve Retrieval-based Dialogue System via Syntax-Informed Attention
Tengtao Song, Nuo Chen, Ji Jiang +2
Multi-turn response selection is a challenging task due to its high demands on efficient extraction of the matching features from abundant information provided by context utterance…
Improving Weakly Supervised Sound Event Detection with Causal Intervention
Yifei Xin, Dongchao Yang, Fan Cui +2
Existing weakly supervised sound event detection (WSSED) work has not explored both types of co-occurrences simultaneously, i.e., some sound events often co-occur, and their occurr…
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research
Xinhao Mei, Chutong Meng, Haohe Liu +6
The advancement of audio-language (AL) multimodal learning tasks has been significant in recent years. However, researchers face challenges due to the costly and time-consuming col…
Unify, Align and Refine: Multi-Level Semantic Alignment for Radiology Report Generation
Yaowei Li, Bang Yang, Xuxin Cheng +3
Automatic radiology report generation has attracted enormous research interest due to its practical value in reducing the workload of radiologists. However, simultaneously establis…
ZeroNLG: Aligning and Autoencoding Domains for Zero-Shot Multimodal and Multilingual Natural Language Generation
Bang Yang, Fenglin Liu, Yuexian Zou +3
Natural Language Generation (NLG) accepts input data in the form of images, videos, or text and generates corresponding natural language text as output. Existing NLG methods mainly…
SSVMR: Saliency-based Self-training for Video-Music Retrieval
Xuxin Cheng, Zhihong Zhu, Hongxiang Li +2
With the rise of short videos, the demand for selecting appropriate background music (BGM) for a video has increased significantly, video-music retrieval (VMR) task gradually draws…
Exploiting Auxiliary Caption for Video Grounding
Hongxiang Li, Meng Cao, Xuxin Cheng +3
Video grounding aims to locate a moment of interest matching the given query sentence from an untrimmed video. Previous works ignore the {sparsity dilemma} in video annotations, wh…
Towards Unified All-Neural Beamforming for Time and Frequency Domain Speech Separation
Rongzhi Gu, Shi-Xiong Zhang, Yuexian Zou +1
Recently, frequency domain all-neural beamforming methods have achieved remarkable progress for multichannel speech separation. In parallel, the integration of time domain network…
M3ST: Mix at Three Levels for Speech Translation
Xuxin Cheng, Qianqian Dong, Fengpeng Yue +3
How to solve the data scarcity problem for end-to-end speech-to-text translation (ST)? It's well known that data augmentation is an efficient method to improve performance for many…
Aligning Source Visual and Target Language Domains for Unpaired Video Captioning
Fenglin Liu, Xian Wu, Chenyu You +3
Training supervised video captioning model requires coupled video-caption pairs. However, for many targeted languages, sufficient paired data are not available. To this end, we int…
A Dynamic Graph Interactive Framework with Label-Semantic Injection for Spoken Language Understanding
Zhihong Zhu, Weiyuan Xu, Xuxin Cheng +2
Multi-intent detection and slot filling joint models are gaining increasing traction since they are closer to complicated real-world scenarios. However, existing approaches (1) foc…