752 citations · 836 across the 11 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.CL2024
CamemBERT 2.0: A Smarter French Language Model Aged to Perfection
Wissam Antoun, Francis Kulumba, Rian Touchent +3
French language models, such as CamemBERT, have been widely adopted across industries for natural language processing (NLP) tasks, with models like CamemBERT seeing over 4 million…
cs.CL2024
Anisotropy Is Inherent to Self-Attention in Transformers
Nathan Godey, Éric de la Clergerie, Benoît Sagot
The representation degeneration problem is a phenomenon that is widely observed among self-supervised learning methods based on Transformers. In NLP, it takes the form of anisotrop…