Double Descent as a Lens for Sample Efficiency in Autoregressive vs. Discrete Diffusion Models

Ahmad Fraij, Sam Dauncey

Data scarcity drives the need for more sample-efficient large language models. In this work, we use the double descent phenomenon to holistically compare the sample efficiency of d…