4.3k citations · 5.2k across the 11 of their papers we have counts for
11 papers
Scaling laws for single-agent reinforcement learning
Jacob Hilton, Jie Tang, John Schulman
Recent work has shown that, in generative modeling, cross-entropy loss improves smoothly with model size and training compute, following a power law plus constant scaling law. One…
Scaling Laws for Reward Model Overoptimization
Leo Gao, John Schulman, Jacob Hilton
In reinforcement learning from human feedback, it is common to optimize against a reward model trained to predict human preferences. Because the reward model is an imperfect proxy,…
Teaching Models to Express Their Uncertainty in Words
Stephanie Lin, Jacob Hilton, Owain Evans
We show that a GPT-3 model can learn to express uncertainty about its own answers in natural language -- without use of model logits. When given a question, the model generates bot…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang +17
Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful, toxic,…
WebGPT: Browser-assisted question-answering with human feedback
Reiichiro Nakano, Jacob Hilton, Suchir Balaji +15
We fine-tune GPT-3 to answer long-form questions using a text-based web-browsing environment, which allows the model to search and navigate the web. By setting up the task so that…
Training Verifiers to Solve Math Word Problems
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian +9
State-of-the-art language models can match human performance on many tasks, but they still struggle to robustly perform multi-step mathematical reasoning. To diagnose the failures…
Batch size-invariance for policy optimization
Jacob Hilton, Karl Cobbe, John Schulman
We say an algorithm is batch size-invariant if changes to the batch size can largely be compensated for by changes to other hyperparameters. Stochastic gradient descent is well-kno…
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Stephanie Lin, Jacob Hilton, Owain Evans
We propose a benchmark to measure whether a language model is truthful in generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including…
Topological Ramsey numbers and countable ordinals
Andrés Eduardo Caicedo, Jacob Hilton
We study the topological version of the partition calculus in the setting of countable ordinals. Let and be ordinals and let be a positive integer. We write $β\to_{top}…
The topological pigeonhole principle for ordinals
Jacob Hilton
Given a cardinal and a sequence of ordinals, we determine the least ordinal (when one exists) such that the topological partition relation \[β\ri…