4 papers
BeNice! Closing Looping's Inference-Memory Gap via KV-Sharing
Hayden Prairie, Jacob M. Ryan, Taylor Berg-Kirkpatrick +1
Looped architectures scale quality while keeping parameter count fixed by repeatedly passing activations through the same block of layers. During inference, each recurrence generat…
Parcae: Scaling Laws For Stable Looped Language Models
Hayden Prairie, Zachary Novack, Taylor Berg-Kirkpatrick +1
Traditional fixed-depth architectures scale quality by increasing training FLOPs, typically through increased parameterization, at the expense of a higher memory footprint, or data…
Lost in Transmission: On the Impact of Networking Corruptions on Video Machine Learning Models
Trenton Chang, Daniel Y. Fu
We study how networking corruptions--data corruptions caused by networking errors--affect video machine learning (ML) models. We discover apparent networking corruptions in Kinetic…
Double Descent Optimization Pattern and Aliasing: Caveats of Noisy Labels
Florian Dubost, Erin Hong, Max Pike +5
Optimization plays a key role in the training of deep neural networks. Deciding when to stop training can have a substantial impact on the performance of the network during inferen…