Improving Transformer Performance for French Clinical Notes Classification Using Mixture of Experts on a Limited Dataset

Thanh-Dung Le, Philippe Jouvet, Rita Noumeir

Transformer-based models have shown outstanding results in natural language processing but face challenges in applications like classifying small-scale clinical texts, especially w…