Vardan Papyan

activity

20162026

most citedPrevalence of Neural Collapse during the terminal phase of deep learning training

374 citations · 941 across the 27 of their papers we have counts for

collaborators

Showing 2026Show all

2 papers · 1 filter

cs.LG2026

Gradient Smoothing: Coupling Layer-wise Updates for Improved Optimization

Haoming Meng, Anton Sugolov, Vardan Papyan

Deep neural networks with repeated architectural blocks, such as transformers, often exhibit structured relationships across layers that emerge during training. Motivated by this o…

cs.LG2026

Laplacian Heads Improve Transformers by Smoothing Token Representations

Yuchong Zhang, Vardan Papyan

Transformers update token representations through multi-head attention and residual connections as $X \leftarrow X + \sum_{i} P^{(i)} X W_{V_{i}} W_{o_{i}}$ , where $P^{(i)}$ is the softma…