Towards Efficient Pre-training: Exploring FP4 Precision in Large Language Models

Jiecheng Zhou, Ding Tang, Rong Fu +8

The burgeoning computational demands for training large language models (LLMs) necessitate efficient methods, including quantized training, which leverages low-bit arithmetic opera…