Squat: Quant Small Language Models on the Edge

Xuan Shen, Peiyan Dong, Zhenglun Kong +9

A growing trend has emerged in designing high-quality Small Language Models (SLMs) with a few million parameters. This trend is driven by the increasing concerns over cloud costs,…