13 citations · 18 across the 5 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024★ 1 cited
Next-token prediction capacity: general upper bounds and a lower bound for transformers
Liam Madden, Curtis Fox, Christos Thrampoulidis
Given a sequence of tokens, such as words, the task of next-token prediction is to predict the next-token conditional probability distribution. Decoder-only transformers have becom…
cs.LG2023★ 3 cited
Memory capacity of two layer neural networks with smooth activations
Liam Madden, Christos Thrampoulidis
Determining the memory capacity of two layer neural networks with hidden neurons and input dimension (i.e., total trainable parameters), which refers to the largest…