2 papers
cs.LG2026
Barriers for Learning in an Evolving World: Mathematical Understanding of Loss of Plasticity
Amir Joudaki, Giulia Lanzillotta, Mohammad Samragh Razlighi +5
Deep learning models excel in stationary data but struggle in non-stationary environments due to a phenomenon known as loss of plasticity (LoP), the degradation of their ability to…
cs.LG2025
MoE-PHDS: One MoE checkpoint for flexible runtime sparsity
Lauren. A Hannah, Soheil Zibakhsh, Kumari Nishu +4
Sparse Mixtures of Experts (MoEs) are typically trained to operate at a fixed sparsity level, e.g. in a top- gating function. This global sparsity level determines an operat…