Scaling Laws for Precision in High-Dimensional Linear Regression

Dechen Zhang, Xuan Tang, Yingyu Liang +1

Low-precision training is critical for optimizing the trade-off between model quality and training costs, necessitating the joint allocation of model size, dataset size, and numeri…