1 paper · 1 filter
Shikhar Srivastava, Christopher Kanan
As data propagates through a Transformer, the norm of its hidden states grows by orders of magnitude with depth, a phenomenon framed as 'curse of depth' and nearly universally trea…