SGEMM-cube: Precision-Recovery FP32 GEMM Approximation on Ascend NPUs with FP16 Matrix Engines

Abstract

Modern AI accelerators provide high-throughput low-precision matrix engines, but often lack efficient support for FP32 GEMM. This paper presents SGEMM-cube, an FP32-accuracy GEMM approximation for Ascend NPUs built on FP16 Cube units. Following the fixed-length two-word splitting of Ootomo and Yokota, each FP32 operand is represented by an FP16 high component and a scaled FP16 residual. The product is reconstructed from three FP16 GEMMs while omitting the residual-residual term; the method therefore targets FP32-level accuracy rather than bit-exact emulation. Under an RN FP32-accumulation model, we provide a componentwise error analysis showing that the omitted term is no larger than the rounding error of a short FP32 inner product and that, for practically relevant inner-product lengths, the overall error is dominated by ordinary FP32 accumulation. We further analyze residual underflow and scaling under round-to-nearest conversion, compare two accumulation orders, and adapt L1-aware blocking and double buffering to Ascend's software-managed memory hierarchy. On Ascend 910A, SGEMM-cube is substantially more accurate than native FP16 GEMM, is comparable to the tested OpenBLAS FP32 SGEMM baseline for the evaluated input distributions and exponent range, and reaches 65.3\,TFLOP/s, or 77\% of the three-GEMM FP32-equivalent peak.