Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
AF-Muon: An AdamW-Free Muon Optimizer for Tied-Embedding Models
Arash Lagzian, Paniz Halvachi, Junming Zhang +2
Muon improves large-scale training by applying a spectral-norm steepest-descent update to matrix parameters, but practical models also contain parameter blocks that do not fit dens…
cs.LG2023
ELSA: Partial Weight Freezing for Overhead-Free Sparse Network Deployment
Paniz Halvachi, Alexandra Peste, Dan Alistarh +1
We present ELSA, a practical solution for creating deep networks that can easily be deployed at different levels of sparsity. The core idea is to embed one or more sparse networks…