11 papers
FireRedTTS3: Unified Speech Generation and Editing with Semantically Enriched Speech Representations
Feiyu Shen, Kun Xie, Yichen Wu +8
Recent continuous autoregressive TTS models operate directly on continuous speech representations, preserving rich acoustic details while leveraging the instruction-following capab…
OSUM-Pangu: An Open-Source Multidimension Speech Understanding Foundation Model Built upon OpenPangu on Ascend NPUs
Yujie Liao, Xuelong Geng, Hongfei Xue +2
Recent advancements in Speech Large Language Models have significantly enhanced multi-dimensional speech understanding. However, the majority of high-performance frameworks are pre…
Seeing the Context: Rich Visual Context-Aware Speech Recognition via Multimodal Reasoning
Wenjie Tian, Mingchen Shao, Bingshen Mu +8
Audio-visual speech recognition (AVSR) is an extension of ASR that incorporates visual signals. Current AVSR approaches primarily focus on lip motion, largely overlooking rich cont…
dLLM-ASR: A Faster Diffusion LLM-based Framework for Speech Recognition
Wenjie Tian, Bingshen Mu, Guobin Ma +3
Automatic speech recognition (ASR) systems based on large language models (LLMs) achieve superior performance by leveraging pretrained LLMs as decoders, but their token-by-token ge…
Steering Language Model to Stable Speech Emotion Recognition via Contextual Perception and Chain of Thought
Zhixian Zhao, Xinfa Zhu, Xinsheng Wang +4
Large-scale audio language models (ALMs), such as Qwen2-Audio, are capable of comprehending diverse audio signal, performing audio analysis and generating textual responses. Howeve…
WEST: LLM based Speech Toolkit for Speech Understanding, Generation, and Interaction
Binbin Zhang, Chengdong Liang, Shuai Wang +8
In this paper, we present WEST(WE Speech Toolkit), a speech toolkit based on a large language model (LLM) for speech understanding, generation, and interaction. There are three key…