Revealing the Dark Secrets of BERT

Olga Kovaleva, Alexey Romanov, Anna Rogers +1

BERT-based architectures currently give state-of-the-art performance on many NLP tasks, but little is known about the exact mechanisms that contribute to its success. In the curren…