Regularizing Transformers With Deep Probabilistic Layers

Aurora Cobo Aguilera, Pablo Martínez Olmos, Antonio Artés-Rodríguez +1

Language models (LM) have grown with non-stop in the last decade, from sequence-to-sequence architectures to the state-of-the-art and utter attention-based Transformers. In this wo…