23 citations · 37 across the 19 of their papers we have counts for
20 papers
From Content to Knowledge: Lightning Fast Long-Video Understanding with Neural Knowledge Representations
Yuchen Guan, Xiao Li, Zongyu Guo +4
We propose a new paradigm for long video understanding by treating a long video as a Neural Knowledge Representation (NKR). NKR represents video contents neither as a stream of tok…
Hierarchical Long Video Understanding with Audiovisual Entity Cohesion and Agentic Search
Xinlei Yin, Xiulian Peng, Xiao Li +2
Long video understanding presents significant challenges for vision-language models due to extremely long context windows. Existing solutions relying on naive chunking strategies w…
Bitrate-Controlled Diffusion for Disentangling Motion and Content in Video
Xiao Li, Qi Chen, Xiulian Peng +3
We propose a novel and general framework to disentangle video data into its dynamic motion and static content components. Our proposed method is a self-supervised pipeline with les…
Text-Queried Audio Source Separation via Hierarchical Modeling
Xinlei Yin, Xiulian Peng, Xue Jiang +2
Target audio source separation with natural language queries presents a promising paradigm for extracting arbitrary audio events through arbitrary text descriptions. Existing metho…
Convert and Speak: Zero-shot Accent Conversion with Minimum Supervision
Zhijun Jia, Huaying Xue, Xiulian Peng +1
Low resource of parallel data is the key challenge of accent conversion(AC) problem in which both the pronunciation units and prosody pattern need to be converted. We propose a two…
Masked Audio Modeling with CLAP and Multi-Objective Learning
Yifei Xin, Xiulian Peng, Yan Lu
Most existing masked audio modeling (MAM) methods learn audio representations by masking and reconstructing local spectrogram patches. However, the reconstruction loss mainly accou…
Low-latency Speech Enhancement via Speech Token Generation
Huaying Xue, Xiulian Peng, Yan Lu
Existing deep learning based speech enhancement mainly employ a data-driven approach, which leverage large amounts of data with a variety of noise types to achieve noise removal fr…
QDFormer: Towards Robust Audiovisual Segmentation in Complex Environments with Quantization-based Semantic Decomposition
Xiang Li, Jinglu Wang, Xiaohao Xu +4
Audiovisual segmentation (AVS) is a challenging task that aims to segment visual objects in videos according to their associated acoustic cues. With multiple sound sources and back…
ABC-KD: Attention-Based-Compression Knowledge Distillation for Deep Learning-Based Noise Suppression
Yixin Wan, Yuan Zhou, Xiulian Peng +2
Noise suppression (NS) models have been widely applied to enhance speech quality. Recently, Deep Learning-Based NS, which we denote as Deep Noise Suppression (DNS), became the main…
DasFormer: Deep Alternating Spectrogram Transformer for Multi/Single-Channel Speech Separation
Shuo Wang, Xiangyu Kong, Xiulian Peng +3
For the task of speech separation, previous study usually treats multi-channel and single-channel scenarios as two research tracks with specialized solutions developed respectively…
Contrast-PLC: Contrastive Learning for Packet Loss Concealment
Huaying Xue, Xiulian Peng, Yan Lu
Packet loss concealment (PLC) is challenging in concealing missing contents both plausibly and naturally when there are only limited available context to use. Recently deep-learnin…
Time-Variance Aware Real-Time Speech Enhancement
Chengyu Zheng, Yuan Zhou, Xiulian Peng +2
Time-variant factors often occur in real-world full-duplex communication applications. Some of them are caused by the complex environment such as non-stationary environmental noise…
Improving Speech Enhancement via Event-based Query
Yifei Xin, Xiulian Peng, Yan Lu
Existing deep learning based speech enhancement (SE) methods either use blind end-to-end training or explicitly incorporate speaker embedding or phonetic information into the SE ne…
Real-time speech enhancement with dynamic attention span
Chengyu Zheng, Yuan Zhou, Xiulian Peng +2
For real-time speech enhancement (SE) including noise suppression, dereverberation and acoustic echo cancellation, the time-variance of the audio signals becomes a severe challenge…
Disentangled Feature Learning for Real-Time Neural Speech Coding
Xue Jiang, Xiulian Peng, Yuan Zhang +1
Recently end-to-end neural audio/speech coding has shown its great potential to outperform traditional signal analysis based audio codecs. This is mostly achieved by following the…
Cross-Scale Vector Quantization for Scalable Neural Speech Coding
Xue Jiang, Xiulian Peng, Huaying Xue +2
Bitrate scalability is a desirable feature for audio coding in real-time communications. Existing neural audio codecs usually enforce a specific bitrate during training, so differe…
Multi-Modal Multi-Correlation Learning for Audio-Visual Speech Separation
Xiaoyu Wang, Xiangyu Kong, Xiulian Peng +1
In this paper we propose a multi-modal multi-correlation learning framework targeting at the task of audio-visual speech separation. Although previous efforts have been extensively…
Towards Error-Resilient Neural Speech Coding
Huaying Xue, Xiulian Peng, Xue Jiang +1
Neural audio coding has shown very promising results recently in the literature to largely outperform traditional codecs but limited attention has been paid on its error resilience…
Latent-Domain Predictive Neural Speech Coding
Xue Jiang, Xiulian Peng, Huaying Xue +2
Neural audio/speech coding has recently demonstrated its capability to deliver high quality at much lower bitrates than traditional methods. However, existing neural audio/speech c…
End-to-End Neural Speech Coding for Real-Time Communications
Xue Jiang, Xiulian Peng, Chengyu Zheng +3
Deep-learning based methods have shown their advantages in audio coding over traditional ones but limited attention has been paid on real-time communications (RTC). This paper prop…