VISION DIFFMASK: Faithful Interpretation of Vision Transformers with Differentiable Patch Masking

Angelos Nalmpantis, Apostolos Panagiotopoulos, John Gkountouras +2

The lack of interpretability of the Vision Transformer may hinder its use in critical real-world applications despite its effectiveness. To overcome this issue, we propose a post-h…