7 papers
Learning with a Single Rollout via Monte Carlo Pass@k Critic
Fengdi Che, Yang Liu, Lei Yu +4
Estimating token-level advantages in reinforcement learning (RL) for language models remains challenging because scaling up episodic experience collection is expensive. The difficu…
Universal computation is intrinsic to language model decoding
Alex Lewandowski, Marlos C. Machado, Dale Schuurmans
Language models now provide an interface to express and often solve general problems in natural language, yet their ultimate computational capabilities remain a major topic of scie…
The World Is Bigger! A Computationally-Embedded Perspective on the Big World Hypothesis
Alex Lewandowski, Adtiya A. Ramesh, Edan Meyer +2
Continual learning is often motivated by the idea, known as the big world hypothesis, that "the world is bigger" than the agent. Recent problem formulations capture this idea by ex…
Toward Understanding In-context vs. In-weight Learning
Bryan Chan, Xinyi Chen, András György +1
It has recently been demonstrated empirically that in-context learning emerges in transformers when certain distributional properties are present in the training data, but this abi…
Plastic Learning with Deep Fourier Features
Alex Lewandowski, Dale Schuurmans, Marlos C. Machado
Deep neural networks can struggle to learn continually in the face of non-stationarity. This phenomenon is known as loss of plasticity. In this paper, we identify underlying princi…
Learning Continually by Spectral Regularization
Alex Lewandowski, MichaÅ Bortkiewicz, Saurabh Kumar +4
Loss of plasticity is a phenomenon where neural networks can become more difficult to train over the course of learning. Continual learning algorithms seek to mitigate this effect…