Scaling Laws in the Tiny Regime: How Small Models Change Their Mistakes

Mohammed Alnemari, Rizwan Qureshi, Nader Begrazadah

Neural scaling laws describe how model performance improves as a power law with size, but existing work focuses on models above 100M parameters. The sub-20M regime -- where TinyML…