5 papers · 1 filter
PhonoVoxtLM: Exploring Single-Codebook Phonological Tokens for Speech Understanding and Generation
Mingrui Liang, Kentaro Onda, Shinji Watanabe
We investigate how speech token representations can support both understanding and generation within a unified speech language model. We present PhonoVoxtLM, built on the existing…
How Should We Evaluate Naturalness for Synthetic Speech with a Foreign Accent?
Kentaro Onda, Satoru Fukayama, Daisuke Saito +1
Recent advances in speech synthesis technology have enabled the generation of speech with diverse characteristics. One such characteristic is a foreign accent. In our preliminary e…
Leveraging Soft Distributions of SSL-Derived Discrete Speech Tokens for Downstream Inference
Kentaro Onda, Satoru Fukayama, Daisuke Saito +1
Discrete speech tokens obtained from self-supervised learning (SSL) models provide efficient data compression while maintaining strong performance, and have been widely used as int…
Phonological Tokenizer: Prosody-Aware Phonetic Token via Multi-Objective Fine-Tuning with Differentiable K-Means
Kentaro Onda, Hayato Futami, Yosuke Kashiwagi +2
In recent years, there has been growing interest in representing speech with discrete tokens, which serve as pseudo-text for speech language models (speechLMs) and as efficient int…
Advanced Modeling of Interlanguage Speech Intelligibility Benefit with L1-L2 Multi-Task Learning Using Differentiable K-Means for Accent-Robust Discrete Token-Based ASR
Kentaro Onda, Satoru Fukayama, Daisuke Saito +1
Building ASR systems robust to foreign-accented speech is an important challenge in today's globalized world. A prior study explored the way to enhance the performance of phonetic…