1 citations · 4 across the 18 of their papers we have counts for
25 papers
SemiCodec: Bridging Speech Tokenization and High-fidelity Reconstruction with Semi-discrete Representations
Kecan Mao, Nan Xu, Zhaoyang Meng +4
Speech representation learning requires a discrete modeling interface while preserving high waveform fidelity. Existing approaches rarely achieve both: discrete tokens ease modelin…
VoTSpeech: Decoupling Voice Design from Speech Generation in Instruction-Following TTS
Wenbing Yang, Qihang Lu, Zihan Sun +4
Instruction-following text-to-speech (TTS) typically leaves voice design implicit, as conventional speaker embeddings encode reference audio rather than text-driven targets. We pro…
From Reliable Text to Real Voices: Trust-Aware Progressive Adaptation for Low-Resource TTS
Jiayi Lu, Yizhong Geng, Jinghan Yang +4
Low-resource text-to-speech (TTS) adaptation is constrained by scarce paired data and costly manual transcription. Existing fixed-voice TTS systems can provide relatively accurate…
Stabilizing Instruction Supervision for Instruct-TTS via Controllable Diversification and Drift Filtering
Yizhong Geng, Kecan Mao, Qifei Li +6
Instruct-TTS systems expand structured style labels into natural-language training instructions through LLM rewriting, yet we find that over 40% of unconstrained rewrites contain s…
CLASVS: Continuous-Latent Autoregression for Melody-Preserving Lyric Editing in Singing Voice Synthesis
Yizhong Geng, Tian-Hao Zhang, Chunfeng Wang +7
Reference-conditioned melody-preserving lyric editing replaces words while retaining a performance's timing, singer identity, and naturalness. Continuous-latent autoregression avoi…
SceneTTS-Bench: A Benchmark for Scene-Level TTS in Drama Dubbing
Yizhong Geng, Yanliang Li, Jinghan Yang +3
Text-to-speech systems are increasingly used for drama dubbing, yet evaluation protocols remain sentence-level, leaving critical scene-level behaviors insufficiently measured. We p…
MIDAS: Mutual Information Disentanglement with Uncertainty-Aware Fusion for Incomplete Multimodal Sentiment Analysis
Yuhua Wen, Yingying Zhou, Qifei Li +4
Most existing multimodal sentiment analysis approaches assume access to complete multimodal inputs. However, real-world applications frequently encounter incomplete or corrupted mo…
MeloCodec: Harnessing Melodic Priors for High-Fidelity Singing Voice Representation
Yizhong Geng, Wenxin Fu, Kecan Mao +7
Neural audio codecs serve as fundamental tokenizers for LLM-based audio generation. While semantic priors are widely exploited to enhance linguistic intelligibility, the integratio…
AffectCodec: Emotion-Preserving Neural Speech Codec with Block-Diagonal Residual FSQ
Zhaoyang Meng, Zhengyao Ma, Kecan Mao +2
Neural speech codecs have become the discrete interface between raw audio and speech language models, yet they remain optimized primarily for acoustic reconstruction fidelity, whic…
Hello-Chat: Towards Realistic Social Audio Interactions
Yueran Hou, Peilei Jia, Zihan Sun +5
Recent advancements in Large Audio Language Models (LALMs) have demonstrated exceptional performance in speech recognition and translation. However, existing models often suffer fr…
DashFusion: Dual-stream Alignment with Hierarchical Bottleneck Fusion for Multimodal Sentiment Analysis
Yuhua Wen, Qifei Li, Yingying Zhou +4
Multimodal sentiment analysis (MSA) integrates various modalities, such as text, image, and audio, to provide a more comprehensive understanding of sentiment. However, effective MS…
RRPO: Robust Reward Policy Optimization for LLM-based Emotional TTS
Cong Wang, Changfeng Gao, Yang Xiang +7
Differentiable reinforcement learning (RL) frameworks like DiffRO offer a powerful approach for controllable text-to-speech (TTS), but are vulnerable to reward hacking, particularl…
Multi-Loss Learning for Speech Emotion Recognition with Energy-Adaptive Mixup and Frame-Level Attention
Cong Wang, Yizhong Geng, Yuhua Wen +7
Speech emotion recognition (SER) is an important technology in human-computer interaction. However, achieving high performance is challenging due to emotional complexity and scarce…
HQ-SVC: Towards High-Quality Zero-Shot Singing Voice Conversion in Low-Resource Scenarios
Bingsong Bai, Yizhong Geng, Fengping Wang +4
Zero-shot singing voice conversion (SVC) transforms a source singer's timbre to an unseen target speaker's voice while preserving melodic content without fine-tuning. Existing meth…
SynParaSpeech: Automated Synthesis of Paralinguistic Datasets for Speech Generation and Understanding
Bingsong Bai, Qihang Lu, Wenbing Yang +8
Paralinguistic sounds, like laughter and sighs, are crucial for synthesizing more realistic and engaging speech. However, existing methods typically depend on proprietary datasets,…
Deep Learning Approaches for Multimodal Intent Recognition: A Survey
Jingwei Zhao, Yuhua Wen, Qifei Li +8
Intent recognition aims to identify users' underlying intentions, traditionally focusing on text in natural language processing. With growing demands for natural human-computer int…
Psy-Copilot: Visual Chain of Thought for Counseling
Keqi Chen, Zekai Sun, Huijun Lian +2
Large language models (LLMs) are becoming increasingly popular in the field of psychological counseling. However, when human therapists work with LLMs in therapy sessions, it is ha…
Psy-Insight: Explainable Multi-turn Bilingual Dataset for Mental Health Counseling
Keqi Chen, Zekai Sun, Yuhua Wen +3
The in-context learning capabilities of large language models (LLMs) show great potential in mental health support. However, the lack of counseling datasets, particularly in Chines…
Enhancing Modal Fusion by Alignment and Label Matching for Multimodal Emotion Recognition
Qifei Li, Yingming Gao, Yuhua Wen +2
To address the limitation in multimodal emotion recognition (MER) performance arising from inter-modal information fusion, we propose a novel MER framework based on multitask learn…
SPA-SVC: Self-supervised Pitch Augmentation for Singing Voice Conversion
Bingsong Bai, Fengping Wang, Yingming Gao +1
Diffusion-based singing voice conversion (SVC) models have shown better synthesis quality compared to traditional methods. However, in cross-domain SVC scenarios, where there is a…
Retrieval Augmented Generation in Prompt-based Text-to-Speech Synthesis with Context-Aware Contrastive Language-Audio Pretraining
Jinlong Xue, Yayue Deng, Yingming Gao +1
Recent prompt-based text-to-speech (TTS) models can clone an unseen speaker using only a short speech prompt. They leverage a strong in-context ability to mimic the speech prompts,…
Improving Audio Codec-based Zero-Shot Text-to-Speech Synthesis with Multi-Modal Context and Large Language Model
Jinlong Xue, Yayue Deng, Yicheng Han +2
Recent advances in large language models (LLMs) and development of audio codecs greatly propel the zero-shot TTS. They can synthesize personalized speech with only a 3-second speec…
Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation
Jinlong Xue, Yayue Deng, Yingming Gao +1
Recent advancements in diffusion models and large language models (LLMs) have significantly propelled the field of AIGC. Text-to-Audio (TTA), a burgeoning AIGC application designed…
Frame-level emotional state alignment method for speech emotion recognition
Qifei Li, Yingming Gao, Cong Wang +4
Speech emotion recognition (SER) systems aim to recognize human emotional state during human-computer interaction. Most existing SER systems are trained based on utterance-level la…