169 citations · 818 across the 32 of their papers we have counts for
4 papers · 1 filter
An overview of neural architectures for self-supervised audio representation learning from masked spectrograms
Sarthak Yadav, Sergios Theodoridis, Zheng-Hua Tan
In recent years, self-supervised learning has amassed significant interest for training deep neural representations without labeled data. One such self-supervised learning approach…
AudioMAE++: learning better masked audio representations with SwiGLU FFNs
Sarthak Yadav, Sergios Theodoridis, Zheng-Hua Tan
Masked Autoencoders (MAEs) trained on audio spectrogram patches have emerged as a prominent approach for learning self-supervised audio representations. While several recent papers…
AxLSTMs: learning self-supervised audio representations with xLSTMs
Sarthak Yadav, Sergios Theodoridis, Zheng-Hua Tan
While the transformer has emerged as the eminent neural architecture, several independent lines of research have emerged to address its limitations. Recurrent neural approaches hav…
Masked Autoencoders with Multi-Window Local-Global Attention Are Better Audio Learners
Sarthak Yadav, Sergios Theodoridis, Lars Kai Hansen +1
In this work, we propose a Multi-Window Masked Autoencoder (MW-MAE) fitted with a novel Multi-Window Multi-Head Attention (MW-MHA) module that facilitates the modelling of local-gl…