LLMQ: Efficient Lower-Precision Pretraining for Consumer GPUs

Erik Schultheis, Dan Alistarh

We present LLMQ, an end-to-end CUDA/C++ implementation for medium-sized language-model training, e.g. 3B to 32B parameters, on affordable, commodity GPUs. These devices are charact…