most citedA Theory on Adam Instability in Large-Scale Machine Learning

2 citations · 2 across the 2 of their papers we have counts for

collaborators
Showing cs.LGShow all

2 papers · 1 filter

cs.LG2026

Scaling an Autoregressive Transformer for Single-Cell Generation

Aleksandr Sharipov, Yusif Mukhtarov, Igor Molybog

We study a self-supervised generation task for single-cell gene expression vectors: given a set of vectors from a cell type, we aim to generate additional gene expression vectors o…

cs.LG2023★ 2 cited

A Theory on Adam Instability in Large-Scale Machine Learning

Igor Molybog, Peter Albert, Moya Chen +14

We present a theory for the previously unexplained divergent behavior noticed in the training of large language models. We argue that the phenomenon is an artifact of the dominant…