9 papers
Grounded Decoding for Autoregressive Speech Enhancement via Adaptive Code-Space Grounding and Local LLM Refinement
Hao Shi, Yuan Gao, Zhaoheng Ni +4
Large language model (LLM)-based autoregressive speech enhancement (SE) produces natural speech using learned clean-speech priors, but may hallucinate content unsupported by the in…
Sarashina2.2-TTS: Tackling Kanji Polyphony in Japanese Speech Generation via Data Scaling and Targeted Data Synthesis
Lianbo Liu, Shiao Zhu, Kai Washizaki +10
While large language model (LLM)-based text-to-speech (TTS) systems have achieved high-quality speech synthesis, most existing systems focus on English and Chinese. Japanese, howev…
Speech-Worthy Alignment for Japanese SpeechLLMs via Direct Preference Optimization
Mengjie Zhao, Lianbo Liu, Yusuke Fujita +4
SpeechLLMs typically combine ASR-trained encoders with text-based LLM backbones, leading them to inherit written-style output patterns unsuitable for text-to-speech synthesis. This…
Distilling LLM Semantic Priors into Encoder-Only Multi-Talker ASR with Talker-Count Routing
Hao Shi, Yusuke Fujita, Roman Koshkin +4
Large language models (LLMs) provide strong semantic priors that can improve multi-talker automatic speech recognition (MT-ASR), but using an LLM as an autoregressive decoder is co…
Paralinguistic Emotion-Aware Validation Timing Detection in Japanese Empathetic Spoken Dialogue
Zi Haur Pang, Yahui Fu, Yuan Gao +1
Emotional Validation is a psychotherapy communication technique that involves recognizing, understanding, and explicitly acknowledging another person's feelings and actions, which…
Beyond Acoustic Prefixes: Persistent Access to Serialized Acoustic Memory for LLM-Based Multi-Talker Speech Recognition
Hao Shi, Yuan Gao, Xugang Lu +1
Large language models (LLMs) provide strong linguistic priors for serialized output training (SOT), yet LLM-based multi-talker ASR degrades substantially as the number of overlappi…
SONAR: Self-Distilled Continual Pre-training for Domain Adaptive Audio Representation
Yizhou Zhang, Yuan Gao, Wangjin Zhou +3
Self-supervised learning (SSL) on large-scale datasets like AudioSet has become the dominant paradigm for audio representation learning. While the continuous influx of new, unlabel…
Bridging Speech Emotion Recognition and Personality: Dataset and Temporal Interaction Condition Network
Yuan Gao, Hao Shi, Yahui Fu +2
This study investigates the interaction between personality traits and emotion expression, exploring how personality information can improve speech emotion recognition (SER). We co…
Serialized Speech Information Guidance with Overlapped Encoding Separation for Multi-Speaker Automatic Speech Recognition
Hao Shi, Yuan Gao, Zhaoheng Ni +1
Serialized output training (SOT) attracts increasing attention due to its convenience and flexibility for multi-speaker automatic speech recognition (ASR). However, it is not easy…