22 citations · 29 across the 7 of their papers we have counts for
7 papers
WeightBridge: An Efficient Weight Transfer Library for Reinforcement Learning
Xuanlin Jiang, Samuel Hsia, Michael Kuchnik +3
Weight transfer - the propagation of updated parameters from trainers to rollout generators - is becoming an important performance bottleneck in reinforcement learning (RL) systems…
ZeroSumEval: An Extensible Framework For Scaling LLM Evaluation with Inter-Model Competition
Hisham A. Alyahya, Haidar Khan, Yazeed Alnumay +2
We introduce ZeroSumEval, a dynamic, competition-based, and evolving evaluation framework for Large Language Models (LLMs) that leverages competitive games. ZeroSumEval encompasses…
LLäMmlein: Transparent, Compact and Competitive German-Only Language Models from Scratch
Jan Pfister, Julia Wunderle, Andreas Hotho
We create two German-only decoder models, LLäMmlein 120M and 1B, transparently from scratch and publish them, along with the training data, for the German NLP research community to…
A Theory on Adam Instability in Large-Scale Machine Learning
Igor Molybog, Peter Albert, Moya Chen +14
We present a theory for the previously unexplained divergent behavior noticed in the training of large language models. We argue that the phenomenon is an artifact of the dominant…
Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python
James K. Reed, Zachary DeVito, Horace He +2
Modern deep learning frameworks provide imperative, eager execution programming interfaces embedded in Python to provide a productive development experience. However, deep learning…
Using Python for Model Inference in Deep Learning
Zachary DeVito, Jason Ansel, Will Constable +3
Python has become the de-facto language for training deep neural networks, coupling a large suite of scientific computing libraries with efficient libraries for tensor computation…
Ebb: A DSL for Physical Simulation on CPUs and GPUs
Gilbert Louis Bernstein, Chinmayee Shah, Crystal Lemire +4
Designing programming environments for physical simulation is challenging because simulations rely on diverse algorithms and geometric domains. These challenges are compounded when…