11 citations · 17 across the 5 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024
Sparse Upcycling: Inference Inefficient Finetuning
Sasha Doubov, Nikhil Sardana, Vitaliy Chiley
Small, highly trained, open-source large language models are widely used due to their inference efficiency, but further improving their quality remains a challenge. Sparse upcyclin…
cs.LG2024★ 3 cited
Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
Nikhil Sardana, Jacob Portes, Sasha Doubov +1
Large language model (LLM) scaling laws are empirical formulas that estimate changes in model quality as a result of increasing parameter count and training data. However, these fo…