AdaHOP: Fast and Accurate Low-Precision Training via Outlier-Pattern-Aware Rotation

Seonggon Kim, Alireza Khodamoradi, Pranathi Vasireddy +2

Hadamard transforms have become a key tool for stabilizing low-precision training, but existing methods apply them uniformly across tensors and computation paths. We show that this…