Multilingual Multi-Speaker Unit Vocoders: A Systematic Analysis of Discrete Speech Representations

Naman Kothari, Arjun Gangwar, Adarsh Arigala +1

Discrete speech units obtained via k-means clustering of self supervised embeddings entangle phonetic, speaker, and language information, causing speaker mixing and cross-lingual i…