2 papers
cs.LG2026
B-DENSE: Branching For Dense Ensemble Network Supervision Efficiency
Cherish Puniani, Tushar Kumar, Arnav Bendre +2
Inspired by non-equilibrium thermodynamics, diffusion models have achieved state-of-the-art performance in generative modeling. However, their iterative sampling nature results in…
cs.LG2025
Dirichlet-Prior Shaping: Guiding Expert Specialization in Upcycled MoEs
Leyla Mirvakhabova, Babak Ehteshami Bejnordi, Gaurav Kumar +3
Upcycling pre-trained dense models into sparse Mixture-of-Experts (MoEs) efficiently increases model capacity but often suffers from poor expert specialization due to naive weight…