Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Dense Local Dependencies Induce Attention-Logit Explosion and Training Instability During Long-Sequence Transformer Training
Suvadeep Hajra
Autoregressive transformer language models frequently exhibit training instability when trained on long sequences, particularly under low-precision arithmetic. Although this instab…
cs.LG2019
Making Convex Loss Functions Robust to Outliers using -Exponentiated Transformation
Suvadeep Hajra
In this paper, we propose a novel {\em -exponentiated} transformation, , for loss functions. When the transformation is applied to a convex loss function, the transfo…