Audio ALBERT: A Lite BERT for Self-supervised Learning of Audio Representation

Po-Han Chi, Pei-Hung Chung, Tsung-Han Wu +4

For self-supervised speech processing, it is crucial to use pretrained models as speech representation extractors. In recent works, increasing the size of the model has been utiliz…