Towards Attention-based Contrastive Learning for Audio Spoof Detection

Abstract

Vision transformers (ViT) have made substantial progress for classification tasks in computer vision. Recently, Gong et. al. '21, introduced attention-based modeling for several audio tasks. However, relatively unexplored is the use of a ViT for audio spoof detection task. We bridge this gap and introduce ViTs for this task. A vanilla baseline built on fine-tuning the SSAST (Gong et. al. '22) audio ViT model achieves sub-optimal equal error rates (EERs). To improve performance, we propose a novel attention-based contrastive learning framework (SSAST-CL) that uses cross-attention to aid the representation learning. Experiments show that our framework successfully disentangles the bonafide and spoof classes and helps learn better classifiers for the task. With appropriate data augmentations policy, a model trained on our framework achieves competitive performance on the ASVSpoof 2021 challenge. We provide comparisons and ablation studies to justify our claim.

Proc. INTERSPEECH 2023

References in corpus (8)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• A Simple Framework for Contrastive Learning of Visual Representations
• A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild
• CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation
• Data Augmenting Contrastive Learning of Speech Representations in the Time Domain
• Global Context Vision Transformers
• Anti-Spoofing Using Transfer Learning with Variational Information Bottleneck
• RawBoost: A Raw Data Boosting and Augmentation Method applied to Automatic Speaker Verification Anti-Spoofing