Transformer-based Korean Pretrained Language Models: A Survey on Three Years of Progress

Kichang Yang

With the advent of Transformer, which was used in translation models in 2017, attention-based architectures began to attract attention. Furthermore, after the emergence of BERT, wh…