BERT's output layer recognizes all hidden layers? Some Intriguing Phenomena and a simple way to boost BERT

Wei-Tsung Kao, Tsung-Han Wu, Po-Han Chi +2

Although Bidirectional Encoder Representations from Transformers (BERT) have achieved tremendous success in many natural language processing (NLP) tasks, it remains a black box. A…