Muting Tokens: Exploring Token Sparsification for Efficient Audio Transformers

Shreya Sajid, Fabrizio Julian Piva, Hiram Rayo Torres Rodriguez +3

Transformer-based models have shown strong performance across diverse tasks, but their reliance on self-attention mechanisms often leads to high computational costs. Token sparsifi…