1 paper
Daryl Noupa Yongueng, Hamidou Tembine
Normalization is a key point in transformer training . In Dynamic Tanh (DyT), the author demonstrated that Tanh can be used as an alternative layer normalization (LN) and confirmed…