An Information-Theoretic Analysis of Self-supervised Discrete Representations of Speech

Badr M. Abdullah, Mohammed Maqsood Shaik, Bernd Möbius +1

Self-supervised representation learning for speech often involves a quantization step that transforms the acoustic input into discrete units. However, it remains unclear how to cha…