Showing math.NAShow all
3 papers · 1 filter
math.NA2026
Automated Numerical Stability Analysis of Deep Learning Operators
Xinye Chen
Finite-precision arithmetic unavoidably introduces numerical approximation errors. Numerical computations may use insufficient precision or an improper formulation, which leads to…
math.NA2026
Computing k-means in mixed precision
Erin Carson, Xinye Chen, Xiaobo Liu
Motivated by the increasing availability of low- and mixed-precision arithmetic on modern hardware, we develop mixed-precision variants of Lloyd's algorithm for k-means clustering.…
math.NA2025
Mixed precision HODLR matrices
Erin Carson, Xinye Chen, Xiaobo Liu
Hierarchical matrix computations have attracted significant attention in the science and engineering community as exploiting data-sparse structures can significantly reduce the com…