10 citations · 30 across the 45 of their papers we have counts for
46 papers
Realizing Immersive Volumetric Video: A Multimodal Framework for 6-DoF VR Engagement
Zhengxian Yang, Shengqi Wang, Shi Pan +8
Fully immersive experiences that tightly integrate 6-DoF visual and auditory interaction are essential for virtual and augmented reality. While such experiences can be achieved thr…
Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs
Ruihan Jin, Pengpeng Shao, Zhengqi Wen +5
Knowledge distillation has emerged as a pivotal technique for transferring knowledge from stronger large language models (LLMs) to smaller, more efficient models. However, traditio…
Edit Content, Preserve Acoustics: Imperceptible Text-Based Speech Editing via Self-Consistency Rewards
Yong Ren, Jiangyan Yi, Jianhua Tao +3
Imperceptible text-based speech editing modifies spoken content through transcript manipulation while preserving acoustic continuity. Prior acoustic-space approaches suffer from co…
OV-InstructTTS: Towards Open-Vocabulary Instruct Text-to-Speech
Yong Ren, Jiangyan Yi, Jianhua Tao +5
Instruct Text-to-Speech (InstructTTS) leverages natural language descriptions as style prompts to guide speech synthesis. However, existing InstructTTS methods mainly rely on a dir…
Atlas: Orchestrating Heterogeneous Models and Tools for Multi-Domain Complex Reasoning
Jinyang Wu, Guocheng Zhai, Ruihan Jin +5
The integration of large language models (LLMs) with external tools has significantly expanded the capabilities of AI agents. However, as the diversity of both LLMs and tools incre…
PSA-MF: Personality-Sentiment Aligned Multi-Level Fusion for Multimodal Sentiment Analysis
Heng Xie, Kang Zhu, Zhengqi Wen +4
Multimodal sentiment analysis (MSA) is a research field that recognizes human sentiments by combining textual, visual, and audio modalities. The main challenge lies in integrating…
SecoustiCodec: Cross-Modal Aligned Streaming Single-Codecbook Speech Codec
Chunyu Qiang, Haoyu Wang, Cheng Gong +10
Speech codecs serve as a crucial bridge in unifying speech and text language models. Existing codec methods face several challenges in semantic encoding, such as residual paralingu…
Manipulated Regions Localization For Partially Deepfake Audio: A Survey
Jiayi He, Jiangyan Yi, Jianhua Tao +2
With the development of audio deepfake techniques, attacks with partially deepfake audio are beginning to rise. Compared to fully deepfake, it is much harder to be identified by th…
Debunk and Infer: Multimodal Fake News Detection via Diffusion-Generated Evidence and LLM Reasoning
Kaiying Yan, Moyang Liu, Yukun Liu +4
The rapid spread of fake news across multimedia platforms presents serious challenges to information credibility. In this paper, we propose a Debunk-and-Infer framework for Fake Ne…
P2Mark: Plug-and-play Parameter-level Watermarking for Neural Speech Generation
Yong Ren, Jiangyan Yi, Tao Wang +7
Neural speech generation (NSG) has rapidly advanced as a key component of artificial intelligence-generated content, enabling the generation of high-quality, highly realistic speec…
ALLM4ADD: Unlocking the Capabilities of Audio Large Language Models for Audio Deepfake Detection
Hao Gu, Jiangyan Yi, Chenglong Wang +6
Audio deepfake detection (ADD) has grown increasingly important due to the rise of high-fidelity audio generative models and their potential for misuse. Given that audio large lang…
ImViD: Immersive Volumetric Videos for Enhanced VR Engagement
Zhengxian Yang, Shi Pan, Shengqi Wang +7
User engagement is greatly enhanced by fully immersive multi-modal experiences that combine visual and auditory stimuli. Consequently, the next frontier in VR/AR technologies lies…
MTPareto: A MultiModal Targeted Pareto Framework for Fake News Detection
Kaiying Yan, Moyang Liu, Yukun Liu +5
Multimodal fake news detection is essential for maintaining the authenticity of Internet multimedia information. Significant differences in form and content of multimodal informati…
Region-Based Optimization in Continual Learning for Audio Deepfake Detection
Yujie Chen, Jiangyan Yi, Cunhang Fan +10
Rapid advancements in speech synthesis and voice conversion bring convenience but also new security risks, creating an urgent need for effective audio deepfake detection. Although…
Reject Threshold Adaptation for Open-Set Model Attribution of Deepfake Audio
Xinrui Yan, Jiangyan Yi, Jianhua Tao +6
Open environment oriented open set model attribution of deepfake audio is an emerging research topic, aiming to identify the generation models of deepfake audio. Most previous work…
Multimodal Diffusion Transformer with Memory Bank for Scalable Long-Duration Talking Video Generation
Haojie Zhang, Zhihao Liang, Ruibo Fu +5
Long-duration talking video synthesis faces enduring challenges in achieving high video quality, portrait consistency, temporal coherence, and computational efficiency. As video le…
Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.0
Zhiyong Wang, Ruibo Fu, Zhengqi Wen +10
Speech synthesis technology has posed a serious threat to speaker verification systems. Currently, the most effective fake audio detection methods utilize pretrained models, and in…
DPI-TTS: Directional Patch Interaction for Fast-Converging and Style Temporal Modeling in Text-to-Speech
Xin Qi, Ruibo Fu, Zhengqi Wen +12
In recent years, speech diffusion models have advanced rapidly. Alongside the widely used U-Net architecture, transformer-based models such as the Diffusion Transformer (DiT) have…
Text Prompt is Not Enough: Sound Event Enhanced Prompt Adapter for Target Style Audio Generation
Chenxu Xiong, Ruibo Fu, Shuchen Shi +9
Current mainstream audio generation methods primarily rely on simple text prompts, often failing to capture the nuanced details necessary for multi-style audio generation. To addre…
WMCodec: End-to-End Neural Speech Codec with Deep Watermarking for Authenticity Verification
Junzuo Zhou, Jiangyan Yi, Yong Ren +3
Recent advances in speech spoofing necessitate stronger verification mechanisms in neural speech codecs to ensure authenticity. Current methods embed numerical watermarks before co…
Utilizing Speaker Profiles for Impersonation Audio Detection
Hao Gu, JiangYan Yi, Chenglong Wang +5
Fake audio detection is an emerging active topic. A growing number of literatures have aimed to detect fake utterance, which are mostly generated by Text-to-speech (TTS) or voice c…
Exploring the Role of Audio in Multimodal Misinformation Detection
Moyang Liu, Yukun Liu, Ruibo Fu +4
With the rapid development of deepfake technology, especially the deep audio fake technology, misinformation detection on the social media scene meets a great challenge. Social med…
Does Current Deepfake Audio Detection Model Effectively Detect ALM-based Deepfake Audio?
Yuankun Xie, Chenxu Xiong, Xiaopeng Wang +9
Currently, Audio Language Models (ALMs) are rapidly advancing due to the developments in large language models and audio neural codecs. These ALMs have significantly lowered the ba…
EELE: Exploring Efficient and Extensible LoRA Integration in Emotional Text-to-Speech
Xin Qi, Ruibo Fu, Zhengqi Wen +10
In the current era of Artificial Intelligence Generated Content (AIGC), a Low-Rank Adaptation (LoRA) method has emerged. It uses a plugin-based approach to learn new knowledge with…