Parallel Scaling Law for Language Models

Mouxiang Chen, Binyuan Hui, Zeyu Cui +5

It is commonly believed that scaling language models should commit a significant space or time cost, by increasing the parameters (parameter scaling) or output tokens (inference-ti…