ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation

Long Phan, Hieu Tran, Hieu Nguyen +1

We present ViT5, a pretrained Transformer-based encoder-decoder model for the Vietnamese language. With T5-style self-supervised pretraining, ViT5 is trained on a large corpus of h…