23 citations · 30 across the 4 of their papers we have counts for
4 papers
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
Tianzhe Chu, Yuexiang Zhai, Jihan Yang +6
Supervised fine-tuning (SFT) and reinforcement learning (RL) are widely used post-training techniques for foundation models. However, their roles in enhancing model generalization…
CodeInsight: A Curated Dataset of Practical Coding Solutions from Stack Overflow
Nathanaël Beau, Benoît Crabbé
We introduce a novel dataset tailored for code generation, aimed at aiding developers in common tasks. Our dataset provides examples that include a clarified intent, code snippets…
AutoNumerics-Zero: Automated Discovery of State-of-the-Art Mathematical Functions
Esteban Real, Mirko Rossini, Connal de Souza +7
Transcendental functions, such as the exponential, are central to scientific computing, yet they cannot be natively calculated by digital hardware. Instead, computers must approxim…
A Bayesian Perspective on Generalization and Stochastic Gradient Descent
Samuel L. Smith, Quoc V. Le
We consider two questions at the heart of machine learning; how can we predict if a minimum will generalize to the test set, and why does stochastic gradient descent find minima th…