3 papers
Attention-Aware Routing: Coupling Routing and Attention in MoEs
Despoina Kosmopoulou, Anastasios Tsetsilas, Efthymios Georgiou +3
In Mixture-of-Experts language models, the router typically selects and weights experts based on the token's hidden state, utilizing limited contextual information. We propose Atte…
Masked Diffusion Language Models with Frequency-Informed Training
Despoina Kosmopoulou, Efthymios Georgiou, Vaggelis Dorovatas +2
We present a masked diffusion language modeling framework for data-efficient training for the BabyLM 2025 Challenge. Our approach applies diffusion training objectives to language…
MEDUSA: A Multimodal Deep Fusion Multi-Stage Training Framework for Speech Emotion Recognition in Naturalistic Conditions
Georgios Chatzichristodoulou, Despoina Kosmopoulou, Antonios Kritikos +5
SER is a challenging task due to the subjective nature of human emotions and their uneven representation under naturalistic conditions. We propose MEDUSA, a multimodal framework wi…