10 papers
PhonoVoxtLM: Exploring Single-Codebook Phonological Tokens for Speech Understanding and Generation
Mingrui Liang, Kentaro Onda, Shinji Watanabe
We investigate how speech token representations can support both understanding and generation within a unified speech language model. We present PhonoVoxtLM, built on the existing…
How Should We Evaluate Naturalness for Synthetic Speech with a Foreign Accent?
Kentaro Onda, Satoru Fukayama, Daisuke Saito +1
Recent advances in speech synthesis technology have enabled the generation of speech with diverse characteristics. One such characteristic is a foreign accent. In our preliminary e…
Leveraging Soft Distributions of SSL-Derived Discrete Speech Tokens for Downstream Inference
Kentaro Onda, Satoru Fukayama, Daisuke Saito +1
Discrete speech tokens obtained from self-supervised learning (SSL) models provide efficient data compression while maintaining strong performance, and have been widely used as int…
Phonological Tokenizer: Prosody-Aware Phonetic Token via Multi-Objective Fine-Tuning with Differentiable K-Means
Kentaro Onda, Hayato Futami, Yosuke Kashiwagi +2
In recent years, there has been growing interest in representing speech with discrete tokens, which serve as pseudo-text for speech language models (speechLMs) and as efficient int…
Advanced Modeling of Interlanguage Speech Intelligibility Benefit with L1-L2 Multi-Task Learning Using Differentiable K-Means for Accent-Robust Discrete Token-Based ASR
Kentaro Onda, Satoru Fukayama, Daisuke Saito +1
Building ASR systems robust to foreign-accented speech is an important challenge in today's globalized world. A prior study explored the way to enhance the performance of phonetic…
Benchmarking Prosody Encoding in Discrete Speech Tokens
Kentaro Onda, Satoru Fukayama, Daisuke Saito +1
Recently, discrete tokens derived from self-supervised learning (SSL) models via k-means clustering have been actively studied as pseudo-text in speech language models and as effic…
Differentiable K-means for Fully-optimized Discrete Token-based ASR
Kentaro Onda, Yosuke Kashiwagi, Emiru Tsunoo +2
Recent studies have highlighted the potential of discrete tokens derived from self-supervised learning (SSL) models for various speech-related tasks. These tokens serve not only as…
Prosodically Enhanced Foreign Accent Simulation by Discrete Token-based Resynthesis Only with Native Speech Corpora
Kentaro Onda, Keisuke Imoto, Satoru Fukayama +2
Recently, a method for synthesizing foreign-accented speech only with native speech data using discrete tokens obtained from self-supervised learning (SSL) models was proposed. Con…
Discrete Tokens Exhibit Interlanguage Speech Intelligibility Benefit: an Analytical Study Towards Accent-robust ASR Only with Native Speech Data
Kentaro Onda, Keisuke Imoto, Satoru Fukayama +2
In this study, we gained insight that contributes to achieving accent-robust ASR using only native speech data. In human perception of non-native speech, the phenomenon known as "i…
A Pilot Study of GSLM-based Simulation of Foreign Accentuation Only Using Native Speech Corpora
Kentaro Onda, Joonyong Park, Nobuaki Minematsu +1
We propose a method of simulating the human process of foreign accentuation using Generative Spoken Language Model (GSLM) only with native speech corpora. When one listens to spoke…