Liam Madden

most citedNext-token prediction capacity: general upper bounds and a lower bound for transformers

1 citations · 1 across the 2 of their papers we have counts for

collaborators

2 papers

cs.LG2024

Interpolation with deep neural networks with non-polynomial activations: necessary and sufficient numbers of neurons

Liam Madden

The minimal number of neurons required for a feedforward neural network to interpolate $n$ generic input-output pairs from $R^{d} \times R^{d^{'}}$ is $Θ (n d^{'})$ .…

cs.LG2024★ 1 cited

Next-token prediction capacity: general upper bounds and a lower bound for transformers

Liam Madden, Curtis Fox, Christos Thrampoulidis

Given a sequence of tokens, such as words, the task of next-token prediction is to predict the next-token conditional probability distribution. Decoder-only transformers have becom…