1.1k citations · 1.7k across the 22 of their papers we have counts for
5 papers · 1 filter
MAViL: Masked Audio-Video Learners
Po-Yao Huang, Vasu Sharma, Hu Xu +7
We present Masked Audio-Video Learners (MAViL) to train audio-visual representations. Our approach learns with three complementary forms of self-supervision: (1) reconstruction of…
Masked Autoencoders that Listen
Po-Yao Huang, Hu Xu, Juncheng Li +5
This paper studies a simple extension of image-based Masked Autoencoders (MAE) to self-supervised representation learning from audio spectrograms. Following the Transformer encoder…
On Adversarial Robustness of Large-scale Audio Visual Learning
Juncheng B Li, Shuhui Qu, Xinjian Li +2
As audio-visual systems are being deployed for safety-critical tasks such as surveillance and malicious content filtering, their robustness remains an under-studied area. Existing…
AudioTagging Done Right: 2nd comparison of deep learning methods for environmental sound classification
Juncheng B Li, Shuhui Qu, Po-Yao Huang +1
After its sweeping success in vision and language tasks, pure attention-based neural architectures (e.g. DeiT) are emerging to the top of audio tagging (AT) leaderboards, which see…
CM3: A Causal Masked Multimodal Model of the Internet
Armen Aghajanyan, Bernie Huang, Candace Ross +8
We introduce CM3, a family of causally masked generative models trained over a large corpus of structured multi-modal documents that can contain both text and image tokens. Our new…