MuonAll: Muon Variant for Efficient Finetuning of Large Language Models

Saurabh Page, Advait Joshi, S. S. Sonawane

Muon optimizer has demonstrated robust results in pretraining of language models but its performance in finetuning of existing public pretrained models is not yet explored. Current…