5 papers
SoulX-Transcriber: A Robust End-to-End Framework for Multi-Speaker Speech Transcription
Yuhang Dai, Haopeng Lin, Zhennan Lin +10
Recent advances in Automatic Speech Recognition (ASR) and Large Language Models (LLMs) have significantly improved speech understanding capabilities. However, multi-speaker speech…
Aligning Language Models for Lyric-to-Melody Generation with Rule-Based Musical Constraints
Hao Meng, Siyuan Zheng, Shuran Zhou +2
Large Language Models (LLMs) show promise in lyric-to-melody generation, but models trained with Supervised Fine-Tuning (SFT) often produce musically implausible melodies with issu…
Joint Learning Global-Local Speaker Classification to Enhance End-to-End Speaker Diarization and Recognition
Yuhang Dai, Haopeng Lin, Jiale Qian +9
Large Audio-Language Models (LALMs) have demonstrated remarkable performance in end-to-end speaker diarization and recognition. However, their speaker discriminability remains limi…
SoulX-Duplug: Plug-and-Play Streaming State Prediction Module for Realtime Full-Duplex Speech Conversation
Ruiqi Yan, Wenxi Chen, Zhanxun Liu +17
Recent advances in spoken dialogue systems have brought increased attention to human-like full-duplex voice interactions. However, our comprehensive review of this field reveals se…
Beyond Text: Utilizing Vocal Cues to Improve Decision Making in LLMs for Robot Navigation Tasks
Xingpeng Sun, Haoming Meng, Souradip Chakraborty +2
While LLMs excel in processing text in these human conversations, they struggle with the nuances of verbal instructions in scenarios like social navigation, where ambiguity and unc…