Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
DirMoE: Dirichlet-routed Mixture of Experts
Amirhossein Vahidi, Hesam Asadollahzadeh, Navid Akhavan Attar +4
Mixture-of-Experts (MoE) models have demonstrated exceptional performance in large-scale language models. Existing routers typically rely on non-differentiable Top-+Softmax, lim…
cs.LG2023
Probabilistic Self-supervised Learning via Scoring Rules Minimization
Amirhossein Vahidi, Simon Schoßer, Lisa Wimmer +4
In this paper, we propose a novel probabilistic self-supervised learning via Scoring Rule Minimization (ProSMIN), which leverages the power of probabilistic models to enhance repre…