Showing math.NAShow all
3 papers · 1 filter
math.NA2026
Automated Numerical Stability Analysis of Deep Learning Operators
Xinye Chen
Finite-precision arithmetic unavoidably introduces numerical approximation errors. Numerical computations may use insufficient precision or an improper formulation, which leads to…
math.NA2024
Mixed precision HODLR matrices
Erin Carson, Xinye Chen, Xiaobo Liu
Hierarchical matrix computations have attracted significant attention in the science and engineering community as exploiting data-sparse structures can significantly reduce the com…
math.NA2024
Computing k-means in mixed precision
Erin Carson, Xinye Chen, Xiaobo Liu
Motivated by the increasing availability of low- and mixed-precision arithmetic on modern hardware, we develop mixed-precision variants of Lloyd's algorithm for k-means clustering.…