Adversarial Self-Attention for Language Understanding

Hongqiu Wu, Ruixue Ding, Hai Zhao +3

Deep neural models (e.g. Transformer) naturally learn spurious features, which create a ``shortcut'' between the labels and inputs, thus impairing the generalization and robustness…