10 papers
Ex-Omni-2D: Expressive Omni-Modal Dialogue Models with Native Visual Presence
Haoyu Zhang, Zhipeng Li, Xiaoying Tang +2
Omni-modal dialogue models can understand multimodal inputs and synthesize spoken replies, but a spoken answer still leaves the agent visually absent. We introduce \textbf{Ex-Omni-…
EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesis
Tianchi Liu, Zeyang Song, Tianrui Wang +3
Psychological research on emotion dynamics has established that human affect is a continuous, evolving process: emotions rise, decay, and transition within seconds. Current emotion…
Ex-Omni: Enabling 3D Facial Animation Generation for Omni-modal Large Language Models
Haoyu Zhang, Zhipeng Li, Yiwen Guo +1
Omni-modal large language models (OLLMs) aim to unify multimodal understanding and generation, yet extending them to jointly produce speech and 3D facial animation remains largely…
BridgeCode: A Dual Speech Representation Paradigm for Autoregressive Zero-Shot Text-to-Speech Synthesis
Jingyuan Xing, Mingru Yang, Zhipeng Li +2
Autoregressive (AR) frameworks have recently achieved remarkable progress in zero-shot text-to-speech (TTS) by leveraging discrete speech tokens and large language model techniques…
Parallel GPT: Harmonizing the Independence and Interdependence of Acoustic and Semantic Information for Zero-Shot Text-to-Speech
Jingyuan Xing, Zhipeng Li, Jialong Mai +2
Advances in speech representation and large language models have enhanced zero-shot text-to-speech (TTS) performance. However, existing zero-shot TTS models face challenges in capt…
Long-Context Speech Synthesis with Context-Aware Memory
Zhipeng Li, Xiaofen Xing, Jingyuan Xing +3
In long-text speech synthesis, current approaches typically convert text to speech at the sentence-level and concatenate the results to form pseudo-paragraph-level speech. These me…
Dynamic Parameter Memory: Temporary LoRA-Enhanced LLM for Long-Sequence Emotion Recognition in Conversation
Jialong Mai, Xiaofen Xing, Yawei Li +4
Recent research has focused on applying speech large language model (SLLM) to improve speech emotion recognition (SER). However, the inherently high frame rate in speech modality s…
Multi-Scale Temporal Transformer For Speech Emotion Recognition
Zhipeng Li, Xiaofen Xing, Yuanbo Fang +3
Speech emotion recognition plays a crucial role in human-machine interaction systems. Recently various optimized Transformers have been successfully applied to speech emotion recog…
AS-Speech: Adaptive Style For Speech Synthesis
Zhipeng Li, Xiaofen Xing, Jun Wang +4
In recent years, there has been significant progress in Text-to-Speech (TTS) synthesis technology, enabling the high-quality synthesis of voices in common scenarios. In unseen situ…
LLM-for-X: Application-agnostic Integration of Large Language Models to Support Personal Writing Workflows
Lukas Teufelberger, Xintong Liu, Zhipeng Li +2
To enhance productivity and to streamline workflows, there is a growing trend to embed large language model (LLM) functionality into applications, from browser-based web apps to na…