BeNice! Closing Looping's Inference-Memory Gap via KV-Sharing

Hayden Prairie, Jacob M. Ryan, Taylor Berg-Kirkpatrick +1

Looped architectures scale quality while keeping parameter count fixed by repeatedly passing activations through the same block of layers. During inference, each recurrence generat…